Adding addtional disk to the existing disk group (Virtual Box)

1. Check the existing 
ls /dev/sd* 

brw-rw---- 1 root disk 8,  2 Apr 12 12:15 sda2
brw-rw---- 1 root disk 8, 32 Apr 12 12:15 sdc
brw-rw---- 1 root disk 8, 48 Apr 12 12:15 sdd
brw-rw---- 1 root disk 8,  1 Apr 12 12:15 sda1
brw-rw---- 1 root disk 8,  0 Apr 12 12:16 sda
brw-rw---- 1 root disk 8, 16 Apr 12 12:16 sdb
brw-rw---- 1 root disk 8, 64 Apr 12 12:16 sde
 

2. Shutdown the machine

3. create a new storage disk (new fixed size )

4. modify as shareable

5. Confirm that the new /dev/sd* has been attached sucessfully to the server

6. fdisk /dev/sdf -- follow the normal procedure as "n" "p" "1" return retun "w" cross reffer the commands and confirm the new disk is showing in /dev/sd*

7./sbin/scsi_id -g -u -d /dev/sdf (sdf is the newly added disk)

8.Configure udev vi /etc/udev/rules.d/99-oracle-asmdevices.rules


9. restart the machine it should have added now !!

Now Discover the asm disks from asm instance, so that it can be added to an existing disk group

column name format a30
column value format a20
SELECT group_number, name, value
FROM v$asm_attribute
ORDER BY group_number, name;
GROUP_NUMBER NAME  
SQL>   2    3

GROUP_NUMBER NAME                           VALUE
------------ ------------------------------ --------------------
           1 au_size                        1048576
           1 content.check                  FALSE
           1 content.type                   data
           1 disk_repair_time               3.6h
           1 failgroup_repair_time          24.0h
           1 idp.boundary                   auto
           1 idp.type                       dynamic
           1 phys_meta_replicated           true
           1 sector_size                    512
           1 thin_provisioned               FALSE

10 rows selected.
---------------------------------------------------------------------------------------

column path format a20
set lines 132
set pages 50
select path, group_number group_#, disk_number disk_#, mount_status,
header_status, state, total_mb, free_mb
from v$asm_disk
order by group_number;SQL> SQL> SQL>   2    3    4

PATH                    GROUP_#     DISK_# MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB
-------------------- ---------- ---------- ------- ------------ -------- ---------- ----------
/dev/asm-disk5                0          0 CLOSED  CANDIDATE    NORMAL            0          0
/dev/asm-disk4                1          3 CACHED  MEMBER       NORMAL         5114       2447
/dev/asm-disk1                1          0 CACHED  MEMBER       NORMAL         5114       2402
/dev/asm-disk3                1          2 CACHED  MEMBER       NORMAL         5114       2407
/dev/asm-disk2                1          1 CACHED  MEMBER       NORMAL         5114       2410

--------------------------------------------------------------------------------------------------

login to asm instance and add the new added disk to the existing disk group

alter diskgroup data add disk '/dev/asm-disk5';

And the new disk added sucessfully and visible as disk5 in the disk group.


column path format a20
set lines 132
set pages 50
select path, group_number group_#, disk_number disk_#, mount_status,
header_status, state, total_mb, free_mb
from v$asm_disk
order by group_numberSQL> SQL> SQL>   2    3    4
  5  /

PATH                    GROUP_#     DISK_# MOUNT_S HEADER_STATU STATE      TOTAL_MB    FREE_MB
-------------------- ---------- ---------- ------- ------------ -------- ---------- ----------
/dev/asm-disk3                1          2 CACHED  MEMBER       NORMAL         5114       2737
/dev/asm-disk5                1          4 CACHED  MEMBER       NORMAL        12284      10942
/dev/asm-disk1                1          0 CACHED  MEMBER       NORMAL         5114       2735
/dev/asm-disk2                1          1 CACHED  MEMBER       NORMAL         5114       2742
/dev/asm-disk4                1          3 CACHED  MEMBER       NORMAL         5114       2771


The below query can give us the detailed look on the asm disks/diskgroup etc....

REM ASM views: REM VIEW            |ASM INSTANCE                                     |DB INSTANCE REM ---------------------------------------------------------------------------------------------------------- REM V$ASM_DISKGROUP |Describes a disk group (number, name, size       |Contains one row for every open ASM REM                 |related info, state, and redundancy type)        |disk in the DB instance. REM V$ASM_CLIENT    |Identifies databases using disk groups           |Contains no rows. REM                 |managed by the ASM instance.                     | REM V$ASM_DISK      |Contains one row for every disk discovered       |Contains rows only for disks in the REM                 |by the ASM instance, including disks that        |disk groups in use by that DB instance. REM                 |are not part of any disk group.                  | REM V$ASM_FILE      |Contains one row for every ASM file in every     |Contains rows only for files that are REM                 |disk group mounted by the ASM instance.          |currently open in the DB instance. REM V$ASM_TEMPLATE  |Contains one row for every template present in   |Contains no rows. REM                 |every disk group mounted by the ASM instance.    | REM V$ASM_ALIAS     |Contains one row for every alias present in      |Contains no rows. REM                 |every disk group mounted by the ASM instance.    | REM v$ASM_OPERATION |Contains one row for every active ASM long       |Contains no rows. REM                 |running operation executing in the ASM instance. | set wrap off set lines 155 pages 9999 col "Group Name" for a6    Head "Group|Name" col "Disk Name"  for a10 col "State"      for a10 col "Type"       for a10   Head "Diskgroup|Redundancy" col "Total GB"   for 9,990 Head "Total|GB" col "Free GB"    for 9,990 Head "Free|GB" col "Imbalance"  for 99.9  Head "Percent|Imbalance" col "Variance"   for 99.9  Head "Percent|Disk Size|Variance" col "MinFree"    for 99.9  Head "Minimum|Percent|Free" col "MaxFree"    for 99.9  Head "Maximum|Percent|Free" col "DiskCnt"    for 9999  Head "Disk|Count" prompt prompt ASM Disk Groups prompt =============== SELECT g.group_number  "Group" ,      g.name          "Group Name" ,      g.state         "State" ,      g.type          "Type" ,      g.total_mb/1024 "Total GB" ,      g.free_mb/1024  "Free GB" ,      100*(max((d.total_mb-d.free_mb)/d.total_mb)-min((d.total_mb-d.free_mb)/d.total_mb))/max((d.total_mb-d.free_mb)/d.total_mb) "Imbalance" ,      100*(max(d.total_mb)-min(d.total_mb))/max(d.total_mb) "Variance" ,      100*(min(d.free_mb/d.total_mb)) "MinFree" ,      100*(max(d.free_mb/d.total_mb)) "MaxFree" ,      count(*)        "DiskCnt" FROM v$asm_disk d, v$asm_diskgroup g WHERE d.group_number = g.group_number and d.group_number <> 0 and d.state = 'NORMAL' and d.mount_status = 'CACHED' GROUP BY g.group_number, g.name, g.state, g.type, g.total_mb, g.free_mb ORDER BY 1; prompt ASM Disks In Use prompt ================ col "Group"          for 999 col "Disk"           for 999 col "Header"         for a9 col "Mode"           for a8 col "State"          for a8 col "Created"        for a10          Head "Added To|Diskgroup" --col "Redundancy"     for a10 --col "Failure Group"  for a10  Head "Failure|Group" col "Path"           for a19 --col "ReadTime"       for 999999990    Head "Read Time|seconds" --col "WriteTime"      for 999999990    Head "Write Time|seconds" --col "BytesRead"      for 999990.00    Head "GigaBytes|Read" --col "BytesWrite"     for 999990.00    Head "GigaBytes|Written" col "SecsPerRead"    for 9.000        Head "Seconds|PerRead" col "SecsPerWrite"   for 9.000        Head "Seconds|PerWrite" select group_number  "Group" ,      disk_number   "Disk" ,      header_status "Header" ,      mode_status   "Mode" ,      state         "State" ,      create_date   "Created" --,      redundancy    "Redundancy" ,      total_mb/1024 "Total GB" ,      free_mb/1024  "Free GB" ,      name          "Disk Name" --,      failgroup     "Failure Group" ,      path          "Path" --,      read_time     "ReadTime" --,      write_time    "WriteTime" --,      bytes_read/1073741824    "BytesRead" --,      bytes_written/1073741824 "BytesWrite" ,      read_time/reads "SecsPerRead" ,      write_time/writes "SecsPerWrite" from   v$asm_disk_stat where header_status not in ('FORMER','CANDIDATE') order by group_number ,        disk_number / Prompt File Types in Diskgroups Prompt ======================== col "File Type"      for a16 col "Block Size"     for a5    Head "Block|Size" col "Gb"             for 9990.00 col "Files"          for 99990 break on "Group Name" skip 1 nodup select g.name                                   "Group Name" ,      f.TYPE                                   "File Type" ,      f.BLOCK_SIZE/1024||'k'                   "Block Size" ,      f.STRIPED ,        count(*)                               "Files" ,      round(sum(f.BYTES)/(1024*1024*1024),2)   "Gb" from   v$asm_file f,v$asm_diskgroup g where  f.group_number=g.group_number group by g.name,f.TYPE,f.BLOCK_SIZE,f.STRIPED order by 1,2; clear break prompt Instances currently accessing these diskgroups prompt ============================================== col "Instance" form a8 select c.group_number  "Group" ,      g.name          "Group Name" ,      c.instance_name "Instance" from   v$asm_client c ,      v$asm_diskgroup g where  g.group_number=c.group_number / prompt Free ASM disks and their paths prompt ============================== col "Disk Size"    form a9 select header_status                   "Header" , mode_status                     "Mode" , path                            "Path" , lpad(round(os_mb/1024),7)||'Gb' "Disk Size" from   v$asm_disk where header_status in ('FORMER','CANDIDATE') order by path / prompt Current ASM disk operations prompt =========================== select * from   v$asm_operation /

How to Move OCR and Voting Disk to a new diskgroups

1. The current OCR and VD are in +OCRV diskgroup

As a Root user

$ ocrcheck
Status of Oracle Cluster Registry is as follows:
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3480
Available space (kbytes) : 258640
ID : 1318929051
Device/File Name : +OCRV
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user



$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 327b1f75f8374f1ebf8611c847ffbdad (ORCL:OCR1) [OCRV]
2. ONLINE 985958d30e4f4f45bf8134a9b07cae4f (ORCL:OCR2) [OCRV]
3. ONLINE 80650a28b2fa4ffdbf93716a8bc22668 (ORCL:OCR3) [OCRV]

2. Create the ASM diskgroup “VOCR”:

SQL>CREATE DISKGROUP VOCR NORMAL REDUNDANCY
FAILGROUP VOCRG1 DISK 'ORCL:VOCR1' name VOCR1
FAILGROUP VOCRG2 DISK 'ORCL:VOCR2' name VOCR2
FAILGROUP VOCRG3 DISK 'ORCL:VOCR3' name VOCR3;
SQL>alter diskgroup VOCR set attribute 'compatible.asm'=12.1.0.0.0';

3. Move OCR to the new ASM diskgroup:

Add the new ASM diskgroup for OCR
$ocrconfig -add +VOCR

Drop the old ASM diskgroup from OCR
$ocrconfig -delete +OCRV

4. Move the voting disk files from old ASM diskgroup to the new ASM diskgroup:
$ crsctl replace votedisk +VOCR
Successful addition of voting disk 29adbae485454f72bf9d66519c921e17.
Successful addition of voting disk 529b802332674f9fbf8543d5acd55672.
Successful addition of voting disk 017523650cd64f47bf65bb90e8ed98e6.
Successful deletion of voting disk 327b1f75f8374f1ebf8611c847ffbdad.
Successful deletion of voting disk 985958d30e4f4f45bf8134a9b07cae4f.
Successful deletion of voting disk 80650a28b2fa4ffdbf93716a8bc22668.
Successfully replaced voting disk group with +VOCR.
CRS-4266: Voting file(s) successfully replaced

$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 29adbae485454f72bf9d66519c921e17 (ORCL:VOCR1) [VOCR]
2. ONLINE 529b802332674f9fbf8543d5acd55672 (ORCL:VOCR2) [VOCR]
3. ONLINE 017523650cd64f47bf65bb90e8ed98e6 (ORCL:VOCR3) [VOCR]
Located 3 voting disk(s).

Make sure the /etc/oracle/ocr.loc file gets updated to point to the new ASM diskgroup:

$ more /etc/oracle/ocr.loc
ocrconfig_loc=+VOCR
local_only=FALSE

5. Shut down and restart CRS using the force option “the Clusterware on all the nodes”:
#./crsctl stop crs –f
# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

PRVF-9652 : Cluster Time Synchronization Services check failed

Cause - The plug-in failed in its perform method  Action - Refer to the logs or contact Oracle Support Services.  Log File Location
/u01/app/oraInventory/logs/installActions2014-02-01_12-58-20PM.log


INFO: Liveness check failed for "ntpd"
INFO: Check failed on nodes:
INFO:   ol6r01,ol6r02
INFO: PRVF-5494 : The NTP Daemon or Service was not alive on all nodes
INFO: PRVF-5415 : Check to see if NTP daemon or service is running failed
INFO: Clock synchronization check using Network Time Protocol(NTP) failed
INFO: PRVF-9652 : Cluster Time Synchronization Services check failed
INFO: Checking VIP configuration.
INFO: Checking VIP Subnet configuration.
INFO: Check for VIP Subnet configuration passed.
INFO: Checking VIP reachability
INFO: Check for VIP reachability passed.
INFO: Post-check for cluster services setup was unsuccessful on all the nodes.
INFO:
WARNING:
INFO: Completed Plugin named: Oracle Cluster Verification Utility

Solution :


INFO: Checking VIP Subnet configuration.
INFO: Check for VIP Subnet configuration passed.
INFO: Checking VIP reachability
INFO: Check for VIP reachability passed.
INFO: Post-check for cluster services setup was unsuccessful on all the no
INFO:
WARNING:
INFO: Completed Plugin named: Oracle Cluster Verification Utility
[root@ol6r01 install]# service ntpd status
ntpd is stopped
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
root     30831 19925  0 14:14 pts/0    00:00:00 grep ntp
[root@ol6r01 install]# grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
[root@ol6r01 install]# vi /etc/sysconfig/ntpd
[root@ol6r01 install]# grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
[root@ol6r01 install]# service ntpd stop
Shutting down ntpd:                                        [FAILED]
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
root     30877 19925  0 14:16 pts/0    00:00:00 grep ntp
[root@ol6r01 install]# service ntpd stop
Shutting down ntpd:                                        [FAILED]
[root@ol6r01 install]# service ntpd start
Starting ntpd:                                             [  OK  ]
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
ntp      30907     1  0 14:16 ?        00:00:00 ntpd -u ntp:ntp -p /var/ru                                                            n/ntpd.pid -x
root     30910 19925  0 14:16 pts/0    00:00:00 grep ntp


When i retried it was sucessfull ..

Thanks !!

Failed to execute command "/u01/app/12.1.0/grid/root.sh" as root within 3,600 seconds on nodes

And when the root.sh fails by the below cause

[root@ol6r01 ~]# /u01/app/12.1.0/grid/root.sh
Performing root user operation for Oracle 12c

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/12.1.0/grid
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsco                                                            nfig_params
2014/02/01 13:45:19 CLSRSC-363: User ignored prerequisites during installa                                                            tion

Failed to create keys in the OLR, rc = 127, Message:
  /u01/app/12.1.0/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory

2014/02/01 13:45:22 CLSRSC-188: Failed to create keys in Oracle Local Registry

Died at /u01/app/12.1.0/grid/crs/install/crsutils.pm line 6479.
The command '/u01/app/12.1.0/grid/perl/bin/perl -I/u01/app/12.1.0/grid/per                                                            l/lib -I/u01/app/12.1.0/grid/crs/install /u01/app/12.1.0/grid/crs/install/                                                            rootcrs.pl ' execution failed


Solution :
Install the below packages and update the failed servers probably u will see the first node on this failure since we prepare the servers similar to another one ..:)

[root@ol6r01 install]# yum install libcap.x86_64
Loaded plugins: security
public_ol6_UEKR3_latest                                                                                        | 1.2 kB     00:00
public_ol6_latest                                                                                              | 1.4 kB     00:00
Setting up Install Process
Package libcap-2.16-5.5.el6.x86_64 already installed and latest version
Nothing to do
[root@ol6r01 install]# yum install compat-libcap1.x86_64
Loaded plugins: security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package compat-libcap1.x86_64 0:1.10-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

======================================================================================================================================
 Package                            Arch                       Version                    Repository                             Size
======================================================================================================================================
Installing:
 compat-libcap1                     x86_64                     1.10-1                     public_ol6_latest                      17 k

Transaction Summary
======================================================================================================================================
Install       1 Package(s)

Total download size: 17 k
Installed size: 29 k
Is this ok [y/N]: y
Downloading Packages:
compat-libcap1-1.10-1.x86_64.rpm                                                                               |  17 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : compat-libcap1-1.10-1.x86_64                                                                                       1/1
  Verifying  : compat-libcap1-1.10-1.x86_64                                                                                       1/1

Installed:
  compat-libcap1.x86_64 0:1.10-1

Complete!
[root@ol6r01 install]# /u01/app/12.1.0/grid/root.sh
Performing root user operation for Oracle 12c

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/12.1.0/grid
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
2014/02/01 13:51:34 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2014/02/01 13:53:24 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.

How to copy/replicate/sync the sequences between two oracle databases

A combination of UltraCommits statements and a database link, in addition to a stored procedure that you can schedule to automatically run, would serve you well. and its a easiest way to keep this in sync.

--drop create db_link
DROP DATABASE LINK SOURCE_DB;

CREATE DATABASE LINK "SOURCE_DB"
  CONNECT TO USER IDENTIFIED BY password USING 'SOURCE_DB';

 --drop create sequences 
  DROP sequence target_seq;
CREATE sequence target_seq start with 6;

  --the next two lines run in source db
  DROP sequence source_seq;
CREATE sequence source_seq start with 6000;

--take a look at the sequences to get an idea of what to expect
SELECT source_schema.source_seq.nextval@SOURCE_DB source_seq,
  target_seq.nextval target_seq
FROM dual; 

--create procedure to reset target sequence that you can schedule to automatically run
CREATE OR REPLACE
PROCEDURE reset_sequence
AS
  l_source_sequence pls_integer;
  l_target_sequence pls_integer;
  l_sql VARCHAR2(100);
BEGIN
  SELECT source_schema.source_seq.nextval@SOURCE_DB,
    target_seq.nextval
  INTO l_source_sequence,
    l_target_sequence
  FROM dual;
  l_sql := 'alter sequence target_seq increment by '||to_number(l_source_sequence-l_target_sequence);
  EXECUTE immediate l_sql;
  SELECT target_seq.nextval INTO l_target_sequence FROM dual;
  l_sql := 'alter sequence target_seq increment by 1';
  EXECUTE immediate l_sql;
  COMMIT;
END reset_sequence;
/

--execute procedure to test it out
EXECUTE reset_sequence;

--review results; should be the same
SELECT source_schema.source_seq.nextval@SOURCE_DB, target_seq.nextval FROM dual;

How to find a Schema size in Oracle

SELECT sum(bytes) / (1024 * 1024 * 1024) size_in_gb
  FROM dba_segments
 WHERE owner = 'SCHEMA_NAME';