PRVF-9652 : Cluster Time Synchronization Services check failed

Cause - The plug-in failed in its perform method  Action - Refer to the logs or contact Oracle Support Services.  Log File Location
/u01/app/oraInventory/logs/installActions2014-02-01_12-58-20PM.log


INFO: Liveness check failed for "ntpd"
INFO: Check failed on nodes:
INFO:   ol6r01,ol6r02
INFO: PRVF-5494 : The NTP Daemon or Service was not alive on all nodes
INFO: PRVF-5415 : Check to see if NTP daemon or service is running failed
INFO: Clock synchronization check using Network Time Protocol(NTP) failed
INFO: PRVF-9652 : Cluster Time Synchronization Services check failed
INFO: Checking VIP configuration.
INFO: Checking VIP Subnet configuration.
INFO: Check for VIP Subnet configuration passed.
INFO: Checking VIP reachability
INFO: Check for VIP reachability passed.
INFO: Post-check for cluster services setup was unsuccessful on all the nodes.
INFO:
WARNING:
INFO: Completed Plugin named: Oracle Cluster Verification Utility

Solution :


INFO: Checking VIP Subnet configuration.
INFO: Check for VIP Subnet configuration passed.
INFO: Checking VIP reachability
INFO: Check for VIP reachability passed.
INFO: Post-check for cluster services setup was unsuccessful on all the no
INFO:
WARNING:
INFO: Completed Plugin named: Oracle Cluster Verification Utility
[root@ol6r01 install]# service ntpd status
ntpd is stopped
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
root     30831 19925  0 14:14 pts/0    00:00:00 grep ntp
[root@ol6r01 install]# grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -g"
[root@ol6r01 install]# vi /etc/sysconfig/ntpd
[root@ol6r01 install]# grep OPTIONS /etc/sysconfig/ntpd
OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid -x"
[root@ol6r01 install]# service ntpd stop
Shutting down ntpd:                                        [FAILED]
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
root     30877 19925  0 14:16 pts/0    00:00:00 grep ntp
[root@ol6r01 install]# service ntpd stop
Shutting down ntpd:                                        [FAILED]
[root@ol6r01 install]# service ntpd start
Starting ntpd:                                             [  OK  ]
[root@ol6r01 install]# ps -ef | grep ntp
oracle    3222     1  0 12:56 pts/0    00:00:10 /usr/bin/Xvnc :1 -desktop                                                             ol6r01.yuryffun.com:1 (oracle) -auth /home/oracle/.Xauthority -geometry 10                                                            24x768 -rfbwait 30000 -rfbauth /home/oracle/.vnc/passwd -rfbport 5901 -fp                                                             catalogue:/etc/X11/fontpath.d -pn
ntp      30907     1  0 14:16 ?        00:00:00 ntpd -u ntp:ntp -p /var/ru                                                            n/ntpd.pid -x
root     30910 19925  0 14:16 pts/0    00:00:00 grep ntp


When i retried it was sucessfull ..

Thanks !!

Failed to execute command "/u01/app/12.1.0/grid/root.sh" as root within 3,600 seconds on nodes

And when the root.sh fails by the below cause

[root@ol6r01 ~]# /u01/app/12.1.0/grid/root.sh
Performing root user operation for Oracle 12c

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/12.1.0/grid
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsco                                                            nfig_params
2014/02/01 13:45:19 CLSRSC-363: User ignored prerequisites during installa                                                            tion

Failed to create keys in the OLR, rc = 127, Message:
  /u01/app/12.1.0/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory

2014/02/01 13:45:22 CLSRSC-188: Failed to create keys in Oracle Local Registry

Died at /u01/app/12.1.0/grid/crs/install/crsutils.pm line 6479.
The command '/u01/app/12.1.0/grid/perl/bin/perl -I/u01/app/12.1.0/grid/per                                                            l/lib -I/u01/app/12.1.0/grid/crs/install /u01/app/12.1.0/grid/crs/install/                                                            rootcrs.pl ' execution failed


Solution :
Install the below packages and update the failed servers probably u will see the first node on this failure since we prepare the servers similar to another one ..:)

[root@ol6r01 install]# yum install libcap.x86_64
Loaded plugins: security
public_ol6_UEKR3_latest                                                                                        | 1.2 kB     00:00
public_ol6_latest                                                                                              | 1.4 kB     00:00
Setting up Install Process
Package libcap-2.16-5.5.el6.x86_64 already installed and latest version
Nothing to do
[root@ol6r01 install]# yum install compat-libcap1.x86_64
Loaded plugins: security
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package compat-libcap1.x86_64 0:1.10-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

======================================================================================================================================
 Package                            Arch                       Version                    Repository                             Size
======================================================================================================================================
Installing:
 compat-libcap1                     x86_64                     1.10-1                     public_ol6_latest                      17 k

Transaction Summary
======================================================================================================================================
Install       1 Package(s)

Total download size: 17 k
Installed size: 29 k
Is this ok [y/N]: y
Downloading Packages:
compat-libcap1-1.10-1.x86_64.rpm                                                                               |  17 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : compat-libcap1-1.10-1.x86_64                                                                                       1/1
  Verifying  : compat-libcap1-1.10-1.x86_64                                                                                       1/1

Installed:
  compat-libcap1.x86_64 0:1.10-1

Complete!
[root@ol6r01 install]# /u01/app/12.1.0/grid/root.sh
Performing root user operation for Oracle 12c

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/12.1.0/grid
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid/crs/install/crsconfig_params
2014/02/01 13:51:34 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2014/02/01 13:53:24 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.

How to copy/replicate/sync the sequences between two oracle databases

A combination of UltraCommits statements and a database link, in addition to a stored procedure that you can schedule to automatically run, would serve you well. and its a easiest way to keep this in sync.

--drop create db_link
DROP DATABASE LINK SOURCE_DB;

CREATE DATABASE LINK "SOURCE_DB"
  CONNECT TO USER IDENTIFIED BY password USING 'SOURCE_DB';

 --drop create sequences 
  DROP sequence target_seq;
CREATE sequence target_seq start with 6;

  --the next two lines run in source db
  DROP sequence source_seq;
CREATE sequence source_seq start with 6000;

--take a look at the sequences to get an idea of what to expect
SELECT source_schema.source_seq.nextval@SOURCE_DB source_seq,
  target_seq.nextval target_seq
FROM dual; 

--create procedure to reset target sequence that you can schedule to automatically run
CREATE OR REPLACE
PROCEDURE reset_sequence
AS
  l_source_sequence pls_integer;
  l_target_sequence pls_integer;
  l_sql VARCHAR2(100);
BEGIN
  SELECT source_schema.source_seq.nextval@SOURCE_DB,
    target_seq.nextval
  INTO l_source_sequence,
    l_target_sequence
  FROM dual;
  l_sql := 'alter sequence target_seq increment by '||to_number(l_source_sequence-l_target_sequence);
  EXECUTE immediate l_sql;
  SELECT target_seq.nextval INTO l_target_sequence FROM dual;
  l_sql := 'alter sequence target_seq increment by 1';
  EXECUTE immediate l_sql;
  COMMIT;
END reset_sequence;
/

--execute procedure to test it out
EXECUTE reset_sequence;

--review results; should be the same
SELECT source_schema.source_seq.nextval@SOURCE_DB, target_seq.nextval FROM dual;

How to find a Schema size in Oracle

SELECT sum(bytes) / (1024 * 1024 * 1024) size_in_gb
  FROM dba_segments
 WHERE owner = 'SCHEMA_NAME';

How to find an Index size

select segment_name,segment_type,bytes/1024/1024 MB
 from dba_segments
 where segment_type='INDEX' and segment_name='your_index_name';

How to find a table size

select segment_name,segment_type,bytes/1024/1024 MB
 from dba_segments
 where segment_type='TABLE' and segment_name='your_table_name';

Script to see Database Size

COLUMN "Total Mb" FORMAT 999,999,999.0
COLUMN "Redo Mb" FORMAT 999,999,999.0
COLUMN "Temp Mb" FORMAT 999,999,999.0
COLUMN "Data Mb" FORMAT 999,999,999.0


SELECT (SELECT Sum(bytes / 1048576)
        FROM   dba_data_files) "Data Mb",
       (SELECT Nvl(Sum(bytes / 1048576), 0)
        FROM   dba_temp_files) "Temp Mb",
       (SELECT Sum(bytes / 1048576) * Max(members)
        FROM   v$log)          "Redo Mb",
       (SELECT Sum(bytes / 1048576)
        FROM   dba_data_files)
       + (SELECT Nvl(Sum(bytes / 1048576), 0)
          FROM   dba_temp_files)
       + (SELECT Sum(bytes / 1048576) * Max(members)
          FROM   v$log)        "Total Mb"
FROM   dual;