10G RAC刪除和新增節點
阿新 • • 發佈：2019-01-02
假設rac有兩個節點rac1和rac2，rac2發生物理損壞重灌系統，重新將rac2加入叢集
1. 在rac1節點執行對rac2節點資源的停止任務，一定要手動執行停止操作，即使顯示offline
[[email protected] ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.orcl.db application ONLINE ONLINE rac1 ora....l1.inst application ONLINE ONLINE rac1 ora....l2.inst application ONLINE OFFLINE ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application ONLINE OFFLINE ora....C2.lsnr application ONLINE OFFLINE ora.rac2.gsd application ONLINE OFFLINE ora.rac2.ons application ONLINE OFFLINE ora.rac2.vip application ONLINE ONLINE rac1
[[email protected] ~]$ srvctl stop inst -d orcl -i orcl2
[[email protected] ~]$ srvctl stop nodeapps -n rac2
[[email protected] ~]$ srvctl stop listener -n rac2
[[email protected] ~]$ srvctl stop asm -n rac22. 刪除群集中的rac2的資源 ---oracle
建議用圖形化dbca --> instance management --> delete an instance
這裡不建議用 srvctl remove instance -d orcl -i orcl2
因為 srvctl 僅僅是刪除了 OCR 中的配置資訊，但不會刪除節點 1 的 thread、redo groups 以及 undo tablespace，而這些 dbca 都會自動做了。
[[email protected] ~]$ srvctl remove asm -n rac23. 在rac1節點，手工刪除crs的遠端埠記錄[[email protected] ~]# cd /u01/app/oracle/product/10.2.0/crs_1/bin/ [[email protected] bin]# ./crs_unregister ora.rac2.LISTENER_RAC2.lsnr
[[email protected] ~]$ cat $ORA_CRS_HOME/opmn/conf/ons.config
localport=6113
remoteport=6200
loglevel=3
useocr=on
[[email protected] bin]# ./racgons remove_config rac2:6200
racgons: Existing key value on rac2 = 6200.
racgons: rac2:6200 removed from OCR.
[[email protected] bin]# ./srvctl remove nodeapps -n rac2
Please confirm that you intend to remove the node-level applications on node rac2 (y/[n]) y4. 在rac1節點，執行節點刪除指令碼
[[email protected] orcl]$ olsnodes -p -i -n --檢視下需要刪除的例項號 查詢結果的第二列
rac1 1 rac1-priv rac1-vip
rac2 2 rac2-priv rac2-vip[[email protected] bin]# cd ../install
[[email protected] install]# ./rootdeletenode.sh rac2,2 ----逗號後面點的2就是上面olsnodes 查出來的例項號
CRS-0210: Could not find resource 'ora.rac2.LISTENER_RAC2.lsnr'.
CRS-0210: Could not find resource 'ora.rac2.ons'.
CRS-0210: Could not find resource 'ora.rac2.vip'.
CRS-0210: Could not find resource 'ora.rac2.gsd'.
CRS-0210: Could not find resource ora.rac2.vip.
CRS nodeapps are deleted successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully deleted 14 values from OCR.
Key SYSTEM.css.interfaces.noderac2 marked for deletion is not there. Ignoring.
Successfully deleted 5 keys from OCR.
Node deletion operation successful.
'rac2,2' deleted successfully [[email protected] ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.orcl.db application ONLINE ONLINE rac1
ora....l1.inst application ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1 5. 執行資料級別和crs級別的清單刪除 --oracle使用者
[[email protected] ~]$ cd /u01/app/oracle/product/10.2.0/db_1/oui/bin/
[[email protected] bin]$ ./runInstaller -silent -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES=rac1"
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oracle/oraInventory
'UpdateNodeList' was successful.[[email protected] bin]$ cd /u01/app/oracle/product/10.2.0/crs_1/oui/bin/
[[email protected] bin]$ ./runInstaller -silent -updateNodeList ORACLE_HOME=$ORA_CRS_HOME "CLUSTER_NODES=rac1" CRS=TURE
Starting Oracle Universal Installer...
No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oracle/oraInventory
'UpdateNodeList' was successful.6. 執行crs的節點新增任務 (圖形化)
[[email protected] ~]$ $ORA_CRS_HOME/oui/bin/addNode.sh
[[email protected] ~]# /u01/app/oracle/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oracle/oraInventory to 770.
Changing groupname of /u01/app/oracle/oraInventory to oinstall.
The execution of the script is complete[[email protected] ~]# cd /u01/app/oracle/product/10.2.0/crs_1/install
[[email protected] install]# sh -x rootaddnode.sh
+ ORA_CRS_HOME=/u01/app/oracle/product/10.2.0/crs_1
+ export ORA_CRS_HOME
+ CH=/u01/app/oracle/product/10.2.0/crs_1
+ ORACLE_HOME=/u01/app/oracle/product/10.2.0/crs_1
+ export ORACLE_HOME
+ ORACLE_OWNER=oracle
+ CRS_NEW_HOST_NAME_LIST=rac2,2
+ CRS_NEW_NODE_NAME_LIST=rac2,2
+ CRS_NEW_PRIVATE_NAME_LIST=rac2-priv,2
+ CRS_NEW_NODEVIPS=rac2-vip
+ UNAME=/bin/uname
++ /bin/uname
+ PLATFORM=Linux
+ '[' -z '' ']'
+ AWK=/bin/awk
+ '[' -z '' ']'
+ SED=/bin/sed
+ '[' -z '' ']'
+ ECHO=/bin/echo
+ '[' -z '' ']'
+ CUT=/usr/bin/cut
+ '[' -z '' ']'
+ ID=/usr/bin/id
+ '[' -z '' ']'
+ SU=/bin/su
+ '[' -z '' ']'
+ GREP=/bin/grep
+ '[' -z '' ']'
+ CAT=/bin/cat
++ /usr/bin/id
++ /bin/awk '-F(' '{print $2}'
++ /bin/awk '-F)' '{print $1}'
+ RUID=root
+ '[' root '!=' root ']'
+ case $PLATFORM in
+ LD_LIBRARY_PATH=/u01/app/oracle/product/10.2.0/crs_1/lib
+ OCRCONFIG=/etc/oracle/ocr.loc
+ SU='/bin/su -l'
+ export LD_LIBRARY_PATH
+ CLSCFG=/u01/app/oracle/product/10.2.0/crs_1/bin/clscfg
+ /u01/app/oracle/product/10.2.0/crs_1/bin/clscfg -add -nn rac2,2 -pn rac2-priv,2 -hn rac2,2
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Attempting to add 1 new nodes to the configuration
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 2: rac2 rac2-priv rac2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
++ /bin/awk '{
nElems = split($1, nodeList, ",");
for (i = 1; i < nElems;)
{
print nodeList[i];
i += 2;
}
}'
++ /bin/echo rac2,2
+ NODES_LIST=rac2
++ /u01/app/oracle/product/10.2.0/crs_1/bin/olsnodes -l
+ LOCALNODE=rac1
++ /u01/app/oracle/product/10.2.0/crs_1/bin/srvctl config nodeapps -a -n rac1
+ VIP_STRING='VIP exists.: /rac1-vip/192.168.100.47/255.255.255.0/eth0'
++ /bin/echo VIP exists.: /rac1-vip/192.168.100.47/255.255.255.0/eth0
++ /bin/awk -F/ '{ print $(NF-1)}'
+ NETMASK=255.255.255.0
++ /bin/echo VIP exists.: /rac1-vip/192.168.100.47/255.255.255.0/eth0
++ /bin/awk -F/ '{ print $NF}'
+ NETIFs=eth0
++ /bin/echo eth0
++ /bin/sed 's|:|\||g'
+ NETIFs=eth0
+ Ni=1
++ /bin/echo rac2
+ for i in '`$ECHO $NODES_LIST`'
+ NODE_NAME=rac2
++ /bin/echo rac2-vip
++ /usr/bin/cut -d, -f1
+ NODE_VIP=rac2-vip
+ NODEVIP=rac2-vip/255.255.255.0/eth0
+ /bin/echo /u01/app/oracle/product/10.2.0/crs_1/bin/srvctl add nodeapps -n rac2 -A rac2-vip/255.255.255.0/eth0 -o /u01/app/oracle/product/10.2.0/crs_1
/u01/app/oracle/product/10.2.0/crs_1/bin/srvctl add nodeapps -n rac2 -A rac2-vip/255.255.255.0/eth0 -o /u01/app/oracle/product/10.2.0/crs_1
+ /u01/app/oracle/product/10.2.0/crs_1/bin/srvctl add nodeapps -n rac2 -A rac2-vip/255.255.255.0/eth0 -o /u01/app/oracle/product/10.2.0/crs_1
++ expr 1 + 1
+ Ni=2
+ . /u01/app/oracle/product/10.2.0/crs_1/install/paramfile.crs
++ ORA_CRS_HOME=/u01/app/oracle/product/10.2.0/crs_1
++ CRS_ORACLE_OWNER=oracle
++ CRS_DBA_GROUP=oinstall
++ CRS_VNDR_CLUSTER=false
++ CRS_OCR_LOCATIONS=/dev/raw/raw1,/dev/raw/raw2
++ CRS_CLUSTER_NAME=crs
++ CRS_HOST_NAME_LIST=rac1,1,rac2,2
++ CRS_NODE_NAME_LIST=rac1,1,rac2,2
++ CRS_PRIVATE_NAME_LIST=rac1-priv,1,rac2-priv,2
++ CRS_LANGUAGE_ID=AMERICAN_AMERICA.WE8ISO8859P1
++ CRS_VOTING_DISKS=/dev/raw/raw3,/dev/raw/raw4,/dev/raw/raw5
++ CRS_NODELIST=rac1,rac2
++ CRS_NODEVIPS=rac1/rac1-vip/255.255.255.0/eth0,rac2/rac2-vip/255.255.255.0/eth0
++ eval echo '$CRS_ORACLE_OWNER'
+++ echo oracle
+ PARAM_VALUE=oracle
++ /bin/echo oracle
++ /bin/awk '/^%/ { print "false"; }'
+ valid=
+ '[' '' = false ']'
++ /bin/echo rac2
++ /bin/sed 's| |,|g'
+ COMMASEPARATED_NODE_LIST=rac2
+ /bin/su -l oracle -c '/u01/app/oracle/product/10.2.0/crs_1/bin/cluutil -sourcefile /etc/oracle/ocr.loc -destfile /u01/app/oracle/product/10.2.0/crs_1/srvm/admin/ocr.loc -nodelist rac2'
[[email protected] install]# cat /u01/app/oracle/product/10.2.0/crs_1/opmn/conf/ons.config
localport=6113
remoteport=6200
loglevel=3
useocr=on[[email protected] ~]# cd /u01/app/oracle/product/10.2.0/crs_1/
[[email protected] crs_1]# sh -x root.sh
+ /u01/app/oracle/product/10.2.0/crs_1/install/rootinstall
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
+ /u01/app/oracle/product/10.2.0/crs_1/install/rootconfig
Checking to see if Oracle CRS stack is already configured
/etc/oracle does not exist. Creating it now.
OCR LOCATIONS = /dev/raw/raw1,/dev/raw/raw2
OCR backup directory '/u01/app/oracle/product/10.2.0/crs_1/cdata/crs' does not exist. Creating now
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/u01/app/oracle/product/10.2.0' is not owned by root
WARNING: directory '/u01/app/oracle/product' is not owned by root
WARNING: directory '/u01/app/oracle' is not owned by root
WARNING: directory '/u01/app' is not owned by root
WARNING: directory '/u01' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node <nodenumber>: <nodename> <private interconnect name> <hostname>
node 1: rac1 rac1-priv rac1
node 2: rac2 rac2-priv rac2
clscfg: Arguments check out successfully.
NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac1
rac2
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
Creating VIP application resource on (0) nodes.
Creating GSD application resource on (0) nodes.
Creating ONS application resource on (0) nodes.
Starting VIP application resource on (2) nodes.1:CRS-0233: Resource or relatives are currently involved with another operation.
Check the log file "/u01/app/oracle/product/10.2.0/crs_1/log/rac2/racg/ora.rac2.vip.log" for more details
..
Starting GSD application resource on (2) nodes...
Starting ONS application resource on (2) nodes.1:CRS-0233: Resource or relatives are currently involved with another operation.
Check the log file "/u01/app/oracle/product/10.2.0/crs_1/log/rac2/racg/ora.rac2.ons.log" for more details
..
Done.[[email protected] ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.orcl.db application ONLINE ONLINE rac1
ora....l1.inst application ONLINE ONLINE rac1
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2 7. 在rac1執行遠端節點的crs新增任務
[[email protected] bin]# pwd
/u01/app/oracle/product/10.2.0/crs_1/bin
[[email protected] bin]# ./racgons add_config rac2:6200
[[email protected] crs_1]# cat /u01/app/oracle/product/10.2.0/crs_1/opmn/conf/ons.config
localport=6113
remoteport=6200
loglevel=3
useocr=on8. 在rac1執行節點新增任務嚮導 (圖形化)
[[email protected] ~]$ $ORACLE_HOME/oui/bin/addNode.sh
[[email protected] crs_1]# cd /u01/app/oracle/product/10.2.0/db_1/
[[email protected] db_1]# sh root.sh
Running Oracle10 root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/10.2.0/db_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.9. 新增例項，有兩種方法，一是圖形化新增，簡單方便，二是手動新增
9.1 使用圖形化新增例項
dbca instance management add an instance
至此，全部結束
9.2 以下是使用手動新增例項，如果9.1能新增成功，以下不需要做
在rac2手工建立asm所需的目錄
[[email protected] oracle]$ mkdir -p /u01/app/oracle/admin/+ASM
[[email protected] oracle]$ cd admin/+ASM/
[[email protected] +ASM]$ mkdir bdump cdump hdump pfile udump將rac1的init+ASM1和orapw+ASM1檔案改為2結尾的，拷貝到rac2的$ORACLE_HOME/dbs
[[email protected] ~]$ scp $ORACLE_HOME/dbs/init+ASM1.ora rac2:$ORACLE_HOME/dbs/init+ASM2.ora
scp $ORACLE_HOME/dbs/orapw+ASM1 rac2:$ORACLE_HOME/dbs/orapw+ASM2
scp $ORACLE_HOME/dbs/initorcl1.ora rac2:$ORACLE_HOME/dbs/initorcl2.ora
scp $ORACLE_HOME/dbs/orapworcl1 rac2:$ORACLE_HOME/dbs/orapworcl2
[[email protected] ~]$ srvctl add asm -n rac2 -i +ASM2 -o $ORACLE_HOME
[[email protected] ~]$ srvctl start asm -n rac2在rac2查詢asm磁碟組是否正確
[[email protected] ~]$ export ORACLE_SID=+ASM2
[[email protected] ~]$
[[email protected] ~]$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.4.0 - Production on Thu Jun 4 12:11:39 2015
Copyright (c) 1982, 2007, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
SQL> select name,state from v$asm_diskgroup;
NAME STATE
------------------------------ -----------
DATA MOUNTED
FRA MOUNTED
SQL> select name, value from v$spparameter where sid = 'orcl1';
no rows selected在rac2手工建立所需的目錄，注意許可權
[[email protected] admin]$ mkdir orcl
[[email protected] orcl]$ mkdir adump bdump cdump dpdump hdump pfile udump在rac1通過sqlplus建立rac2的相關引數 --使用手動新增的原rac2的好處，如果是改為新增rac4 ，那資料庫裡面就要相應的建立以下資訊 （如果使用dbca新增例項則會自動新增，但是原來rac2的資訊還是要在資料庫裡手動刪除如undotbs2）
alter system set instance_number=1 scope=spfile sid='wql1';
create undo tablespace undotbs1 datafile '+WQL' size 200M; 可選
alter system set undo_tablespace='undotbs1' scope=spfile sid='wql1';
alter database add logfile thread 1 group 1 '+WQL' size 50M;
alter database add logfile thread 1 group 2 '+WQL' size 50M;
alter system set thread=1 sid='wql1';
alter database enable thread 1 ;重啟例項1 啟動例項2
在RAC1上使用 netca （圖形化） 配置監聽
在RAC1上使用 dbca （圖形化） 新增RAC2例項
[[email protected] orcl]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.orcl.db application ONLINE ONLINE rac1
ora....l1.inst application ONLINE ONLINE rac1
ora....l2.inst application ONLINE ONLINE rac2
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2 至此，全部結束
參考文件：
https://blogs.oracle.com/Database4CN/entry/%E8%8A%82%E7%82%B9os%E9%87%8D%E8%A3%85%E5%90%8E%E5%8A%A0%E5%9B%9E%E9%9B%86%E7%BE%A4%E7%9A%84%E6%AD%A5%E9%AA%A4_10g_rac
Removing a Node from a 10gR1 RAC Cluster (Doc ID 269320.1)
Adding New Nodes to Your Oracle RAC 10g Cluster on Linux