Add Third Node To Existing Two Node Cluster
Feb 3, 2012
I'm trying to add the third node to my existing two node cluster. In fact i succeded adding the node but i have the following problem:
-when i start node 1 and 2, everything is ok, but on node 3 startup one of the other nodes fails
-this happens in any combination: start 1 and 3, everything is ok. when i start node 2, node 3 fails
The asm configuration (disks, permissions, etc) is the same on all three nodes.find below the asm and crsd log details from the failure time:
alert_ASM.log
NOTE: enlarging ACD for group 1/0x124851d1 (DATA)
WARNNING: cache read a corrupted block group=DATA dsk=4 blk=1 from disk 4
+NOTE: a corrupted block from group DATA was dumped to /u01/app/oracle/diag/asm/+asm/+ASM3/trace/+ASM3_rbal_4109.trc+
WARNNING: cache read(retry) a corrupted block group=DATA dsk=4 blk=1 from disk 4
[code]...
View 3 Replies
ADVERTISEMENT
Jun 15, 2012
current environment
Oracle VM 2 Node RAC cluster
Oracle 11G 11.2.0.3.0
Oracle 11g rel 2 GI
on current 2 node cluster we have GI and RAC db configured Nodes: vmorarac1,vmorarac2
we shutdown vmorarac1 to clone it to vmorarac5
on new node I have changed the hostname to vmorarac5 In /etc/sysconfig/network-scripts/ifcfg-eth0 change ip to new ip and same for /etc/sysconfig/network-scripts/ifcfg-eth1 152.144.199.210,152.144.199.211
mad echanges to /etc/hosts on vmorarac1/2 for 2 new IP address assigned to vmorarac5
152.144.199.171 vmorarac1.pbi.global.pvt vmorarac1
192.168.2.30 vmorarac1-priv.pbi.global.pvt vmorarac1-priv
152.144.199.184 vmorarac1-vip.pbi.global.pvt vmorarac1-vip
[Code]....
Query is is that all ok to reboot the new node to reflect changes as below and change the hostname to vmorarac2
152.144.199.210 vmorarac2.pbi.global.pvt vmorarac2
192.168.2.32 vmorarac2-priv.pbi.global.pvt vmorarac2-priv
152.144.199.211 vmorarac2-vip.pbi.global.pvt vmorarac2-vip
hostname change in /etc/sysconfig/network
On new node vmorarac5, there is alreay software for RAC DB and GUI cloned from vmorarac1
addnode.sh is used to add existing node, but as part of pre-requisite configuratoin is there any config step missing
View 3 Replies
View Related
Jun 24, 2013
We are going to setup a 4 node rac cluster. So i think we need to configure only one SCAN with 3IP's. And how the scan listeners are configured on these 4 nodes.
View 1 Replies
View Related
Apr 10, 2012
I have a 3 node RAC server on Windows Server 2008. Last week the hard drive went out on one of the nodes and I have had to rebuild as I could not recover anything.
I went through and deleted the old node and now I have just finished adding the new node back to my cluster via documentation. Once I created the new instance on that server DBCA attempted to start and it failed gaving me a crs error. I found out later that the other 2 nodes went down and the new one that did not start correctly was the only one up!! I went and stopped the new instance and restarted the first 2. The associated services did not start with the instance so I had to start each manually. The trace files show an ORA-29702 error with cluster group service and the instance being stopped on both of the existing nodes. No other error messages stood out.
Now I cannot get any crs services to start on that 3rd node even if I attempt to start manually. I have also tried stopping all and restarting and that does not work. I found another post on this forum from you and followed it. The ASM service was fine the entire time through all the logs and I don't know how to verify LMON in Windows but I didn't see any LMON errors in the alert log. Also, the voting disks are online. Each node has their own and they are mirrored. Where else to look?
View 7 Replies
View Related
Aug 16, 2013
I believe, in a multiple RAC Physical Standby nodes only one node applies the archivelogs to the standby database. How you can find which node is the APPLY NODE using a SQL query?
I know you can veiw this information by querying the standby configuration using DGMGRL but how would you find out this information from a remote client using SQLPlus/PLSQL?
View 2 Replies
View Related
Jan 24, 2011
We have oracle database 10g(10.2.0.4) running on AIX 5.1. we have installed 2-node RAC on another server AIX 6.1 . Now we want to migrate our single node database to new 2-node rac environment. how we can move/migrate/cluster-aware our single node database to 2-node rac?
View 3 Replies
View Related
Jul 18, 2013
I am installing Oracle 10g Clusterware in RHEL 5 Server. When i run root.sh in the second node the last step Running vipca(silent) for configuring nodeapps failed. Then i run VIPCA manually in second node and configured vip configuration successfully,but when i checked the post-checks for cluster services the Checking existence of VIP node application,Checking existence of ONS node application and Checking existence of GSD node application in second node failed. I am able to ping both the servers one another with vip name successfully and the CSS,CRS and EVM appears healthy in both nodes. 1. Does the vip configuration is proper? If no what is causing this error and how to rectify it? Find the output below
[root@Rhel52 bin]# ./crsctl check crsCSS appears healthyCRS appears healthyEVM appears healthy[root@Rhel52 bin]#[root@Rhel52 bin]# clear[root@Rhel52 bin]#[root@Rhel52 bin]# ./crs_stat -tName Type Target State Host------------------------------------------------------------------------------------ora.rhel5.gsd application ONLINE ONLINE rhel5ora.rhel5.ons application ONLINE ONLINE rhel5ora.rhel5.vip application ONLINE ONLINE rhel5ora.rhel52.gsd application ONLINE ONLINE rhel52ora.rhel52.ons application ONLINE ONLINE rhel52ora.rhel52.vip application ONLINE ONLINE rhel52 [root@Rhel52 bin]#./runcluvfy.sh stage -post crsinst -n
[code]....
View 0 Replies
View Related
Sep 13, 2012
I would like get more details about single node RAC.. What is single Node RAC?..
1). just configure only one server instead of two node... same RAC installation.. cluster installation and shared disk configuration in single server itself.. in future we can add node to this ...
2). in 11gR2 having new feature Single Node RAC.. Multiple instances running on same node and configure using Cluster...
is there any advantage using single node rac(other than online upgrade to full RAC).
View 3 Replies
View Related
Jul 25, 2012
I tried to clone a 2 node rac database to single instance non rac database using existing backup. I have not used connectivity to target or catalog. rman duplicate finished with below messages:
rman auxiliary sys/******@dbracdup
RMAN> duplicate database to dbrac spfile backup location '/oracle/backup';
...
...
Finished recover at 25-JUL-12
Segmentation fault
And the database was in mount stage, and when i tried to open database it failed with below error:
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-19838: Cannot use this control file to open database .
View 4 Replies
View Related
Feb 20, 2012
I have table property and flat , in both of these tables I have cvp_id common colomn .Now I want to add cluster on this colomn so how can I add cluster to table which is already exists.
View 3 Replies
View Related
Nov 19, 2013
there an option to install oracle grid infrastructure on one node only?just like oracle database one node once you have a second node extend the cluster to 2 nodes and maybe relocate the database to the other node ?or even go for full 2 node cluster
View 3 Replies
View Related
Jun 27, 2012
What is expected behavior for VIP on Linux servers? ( 2-node RAC)
1.When entire RAC stack is running, and one node is rebooted or crashed? Will VIP fail over to second node?
2.When one node RAC stack is contained (crsctl stop crs), shutting down RAC stack on remaining node? What happens to the VIP and Scan IP?
3.When RAC stacks are down on both nodes, one node has VIP on (eth0:1), reboot this node. What happens to the VIP?
We see some behavior in scenario 3, but don't know why the VIP was moved to second node as all Oracle processes are down.
View 2 Replies
View Related
Jul 14, 2011
I have 2 node 11gR1 running on Unix, I want to add 3 more RAC node to it. What are all the steps involved and should I first add 3rd node and then the 4th node or I should add both at the same time? is there any difference between these two methods?
View 1 Replies
View Related
May 23, 2013
I have recently installed ORACLE 11gr2 rac. After installation i am not able to connect to one node i am getting the below error
<msg time='2013-05-23T10:25:24.660+05:30' org_id='oracle' comp_id='tnslsnr'
type='UNKNOWN' level='16' host_id='DPDCNSFDB1'
host_addr='172.18.132.51'>
<txt>TNS-12518: TNS:listener could not hand off client connection
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
IBM/AIX RISC System/6000 Error: 32: Broken pipe
My Tns connects to only one node.And i tried to connect using the virtual IP but not able to connect. TNS poing works fine.
View 1 Replies
View Related
Jul 19, 2010
We are in the design phase of a project and wanting to implement a 4 node RAC with a DR site. However to get around some licensing costs, management are insisting that only 1 node be up accepting data at the DR site. In the event of a disaster, the remaining nodes will be switched on.
View 3 Replies
View Related
Nov 16, 2011
I need to change Virtual IP for one of my RAC node as it is conflicting with another server's IP.
View 1 Replies
View Related
Aug 8, 2011
When I added a new node to RAC (existng 2 windows server 2008 R2 nodes), using the command,
CODEaddNode -silent
"CLUSTER_NEW_NODES={node3}" "CLUSTER_NEW_PRIVATE_NODE_NAMES={node3}"
"CLUSTER_NEW_VIRTUAL_HOSTNAMES={node3-vip}"
the output was "The cluster node addition <path> was successful".
But when I tried the following command,
CODEcluvfy stage -post nodeadd -node3
I got an error stating
QUOTE "ERROR: Shared resources check for node addition failed
The following node is not in cluster: node3
Post-check for node addition was unsuccessful on all the nodes."
When I checked the inventory logs, I found the following entry
QUOTE SEVERE:
oracle.sysman.oii.oiix.OiixException: The Oracle home '<path>' could not be updated as it does not exist.
at oracle.sysman.oii.oiic.OiicBaseInventoryApp.getOracleHomeInfo(OiicBaseInventoryApp.java:645)
at oracle.sysman.oii.oiic.OiicUpdateNodeList.doOperation(OiicUpdateNodeList.java:206)
at oracle.sysman.oii.oiic.OiicBaseInventoryApp.main_helper(OiicBaseInventoryApp.java:790)
at oracle.sysman.oii.oiic.OiicUpdateNodeList.main(OiicUpdateNodeList.java:399)
View 1 Replies
View Related
Feb 27, 2012
How to make a node of a Hierarchical Tree BOLD without intervene java code ?
View 1 Replies
View Related
Aug 31, 2012
How can I add an attribute node to an existing xml in a xmltype?
Say I have
declare
t_xml xmltype;
begin
t_xml := xmltype( '<x a="1"/>' );
end;How can I manipulate this variable t_xml to become
<x a="1" b="2"/>Anton
View 14 Replies
View Related
Jan 4, 2013
1. i register scheme like this
DECLARE
schemaurl VARCHAR2(256);
xmlschema xmltype;
BEGIN
[code]...
/2. i try to insert XML
insert into TMP_SPM_ACTLOGLINE values (xmltype(
'<ACTLOGLINE xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
[code]...
The question is how should the scheme look like to insert this XML?
View 5 Replies
View Related
Dec 31, 2012
I want to find the path between two nodes from oracle Hierarchical Table.
Consider the following case ---
NodeId --- ParentId
=============
1 >>>>>> 0
2 >>>>>> 1
3 >>>>>> 2
4 >>>>>> 3
5 >>>>>> 0
6 >>>>>> 5
Here I want to query in the database table that whether there is a path between node 3 and node 5 ? The previous query you provided work up to root node.
Here my expected result is 3-->2-->1-->0-->5
Again if I query in the table to get the path between 1 and 3 , I want to get the output in the following way -
1-->2-->3
So the query need to work in both the cases. Where root nod can act as an intermediate node or not.
how I can get it ?
View 3 Replies
View Related
Aug 11, 2011
I am having an issue when trying to run a backup from node 207...Initially When trying to run the backup I got this error.
scrbdefrmr207:orarer 54> brbackup -u / -c force -t online -m all -p initRER_online.sap
BR0051I BRBACKUP 7.00 (40)
BR0055I Start of database backup: begnlswy.anf 2011-08-11 21.29.08
BR0484I BRBACKUP log file: /oracle/RER/sapbackup/begnlswy.anf
BR0477I Oracle pfile /oracle/RER/102_64/dbs/initRER001.ora created from spfile /oracle/RER/102_64/dbs/spfileRER001.ora
BR0280I BRBACKUP time stamp: 2011-08-11 21.29.13
BR1351I Configuration of database services:
RER001 PREF: RER001 AVAIL:
RER002 PREF: RER002 AVAIL:
DVEBMGS50_scrbdefrmr207 PREF: RER001 AVAIL: RER002
DVEBMGS50_scrbdefrmr208 PREF: RER002 AVAIL: RER001
[code]....
I have many other rac servers and sids setup this way and they all work,,I can login to sqlplus and do any changes .. no issues going into sqlplus in node 207 and node 208..I have checked the directory i.e mesg and teh files are there ..
scrbdefrmr207:orarer 56> pwd
/oracle/RER/102_64/sqlplus/mesg
scrbdefrmr207:orarer 57> ls
cpyus.msb cpyus.msg sp1us.msb sp1us.msg sp2us.msb sp2us.msg
I think the message is decieving as it complains about oracle home and sp1<lang>.msb not found ,, when i tis there.I have also checked my env variables .. they all there i.e
scrbdefrmr207:orarer 58> env |grep ORA
dbms_type=ORA
ORACLE_SID=RER001
ORACLE_HOME=/oracle/RER/102_64
ORACLE_BASE=/oracle
scrbdefrmr207:orarer 59> env | grep LD_
LD_LIBRARY_PATH=/usr/sap/RER/SYS/exe/run:/oracle/RER/102_64/lib
I an run archives log backups perfectly.. So its only when its trying to connect into the db.
View 4 Replies
View Related
Jun 1, 2013
We have an Oracle 9i (9.2.0.7) two-node RAC database running on Solaris 5.8. Recently we have switched our production database from no archive log mode to archive log. We changed the parameter CLUSTER_DATABASE=FALSE on one of the instances before converting and reverted back.why this parameter needs to be changed to false?
View 2 Replies
View Related
Mar 26, 2012
I had one of my RAC nodes go down due to a disk failure. I have a 3 node cluster running 10.2.0.4 on Dell 610's running Windows Server 2008. I have been running AWR reports this afternoon and am seeing CPU time as my top timed event. Here is the exerpt from the report I am looking at:
Cache Sizes
~~~~~~~~~~~ Begin End
---------- ----------
Buffer Cache: 20,032M 20,032M Std Block Size: 8K
Shared Pool Size: 12,688M 12,688M Log Buffer: 6,336K
[code]...
I wanted to ask if there was anything that I could be doing to alleviate the workload on the 2 remaining nodes right now? As far as I understand it there is no way to stop users from hitting the database and without my 3rd node to load balance the CPU will continue to be pegged until the end of the day as the users are logging off.
View 3 Replies
View Related
Apr 17, 2012
I have the following problem on a 10.2.0.4 Oracle on AIX 1.6..I have 2 RAC nodes as the node # 2 goes down, the VIP migrated correctly to the node # 1, but the listener # 1 do not see the change, and just resolve to the local ip.
The same does not apply in the reverse case, when it drops the node N 1, the listener attends to the fallen VIP node and migrated.The configuration is correct and identical in both nodes, according to the documentation, but I can not find a reasonable answer.
IP node # 1.
scm1or01 192.68.60.11
scm1or01-VIP 192.68.60.13
IP node # 2
scm1or02 192.168.60.12
scm1or01-VIP 192.168.60.14
when scm1or02 is down: scm1or01:/cots/oracle/app/oracle/product/10.2.0/db_1/network/admin> lsnrctl status
LSNRCTL for IBM/AIX RISC System/6000: Version 10.2.0.4.0 - Production on 11-APR-2012 19:56:46
Copyright � 1991, 2007, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER_SCM1OR01
Version TNSLSNR for IBM/AIX RISC System/6000: Version 10.2.0.4.0 - Production
Start Date 11-APR-2012 16:50:19
[code]...
I do not understand the reason, I've rebuilt the configuration of the listener, and netmgr netca tools, but the problem persists
View 3 Replies
View Related
Aug 12, 2010
I get the following output for node connectivity check as part of pre clusterware installation checks.oracle@cps-oracle-1.homer.com% runcluvfy.sh comp nodecon -n cps-oracle-1,cps-oracle-2 -verbose
Verifying node connectivity
Verification of node connectivity was unsuccessful on all the nodes.
The above output i get for node connectivity component check. I pinged each public and private interface on node and found them to be working.The CRS is for Oracle10Gr2.
View 3 Replies
View Related
Mar 19, 2012
A am new in RAC configuration.I configuring RAC with ASM diskgroup with the external redundency. So now i am facing problem to mount the ASM diskgroup. I am giving the the RAC configuration description below.
1. using two node rac1 and rac2
2. asm disk configuration 3 drive
/dev/raw/raw1
/dev/raw/raw2
/dev/raw/raw3
on both node
3. and i am using vmware to configure the both node asmdisk group
after configuring the os level and oracle clustring , i run dbca to configure asm disk group..so in the create disk screen i have found my previous create 3 disks
/dev/raw/raw1
/dev/raw/raw2
/dev/raw/raw3
so first two disks
/dev/raw/raw1
/dev/raw/raw2
i named it "dg1" then press ok , when i pressed ok a warring is appear that is "could not mount the diskgroup on remote node on rac2 using the connection service rac2:1521:+ASM2. Ensure that the listener is running on this node and the asm instance is registred to the listener. Receive the following error:
ORA-15032 : not all alterations performed
ORA-15063 : ASM discovered insufficient number of disk for diskgroup "DG1"
as the same way i create another diskgroup "fg" for flash recovery area using disk /dev/raw/raw3
so after creating the two diskgroup i saw the two diskgroup is appere in the "diskgroup appere screen" an saw that only one diskgroup is mounted but not second node i.e. rac2 diskgroup is not mounted . What should i do no .
View 2 Replies
View Related
Mar 21, 2012
I would like to implement Oracle RAC with 2 nodes for SE Licence. I did a lot when both this nodes with 3 NICs each were plugged at the same switch. Now I have a need to construct a RAC when two nodes will be in separate locations, abot 4 miles from each one. What should I explain to our network administrator he needs to do to implement this solution? I've been told that they can do a FO channel to each location. But don't have exact clear explicaton.
View 3 Replies
View Related
Jul 16, 2012
we had an issue with our 10.2.0.4.0 two node RAC Database where when running stats collection caused the database performance to go down and in AWR I noticed the following:
Top 5 Timed Events
Event Waits Time(s) Avg Wait(ms) % Total Call Time Wait Class
gc buffer busy 95,077 73,786 776 76.0 Cluster
cursor: pin S wait on X 3,524,808 54,467 15 56.1 Concurrency
library cache lock 30,223 13,660 452 14.1 Concurrency
gc cr request 2,876 3,594 1,250 3.7 Cluster
library cache pin 1,740 1,800 1,035 1.9 Concurrency
when looked under Library Cache Activity section in the AWR report, I see the following:
Namespace Get Requests Pct Miss Pin Requests Pct Miss Reloads Invali- dations
BODY 59 3.39 3,056 0.13 1 0
CLUSTER 70 0.00 73 0.00 0 0
INDEX 178 0.00 925 0.00 0 0
SQL AREA 423,064 50.25 2,465,209 2.37 182 119,671
TABLE/PROCEDURE 1,848 21.27 22,638 0.62 70 0
TRIGGER 14 0.00 1,801 0.06 1 0
does this mean, we are having memory crunch for this instance on node 1? we have sga_target=26gb and sga_max_size=32gb set for this instance.
View 2 Replies
View Related
Dec 28, 2010
There are 4 nodes in a RAC environment A,B,C,D..For instance i want to disable all the services for node D temporarily and hand over the Node D to the system admin guys and after the maintainance is done I want to restart the services again on node D..
View 1 Replies
View Related
