RedHat 7.4下Oracle 19c RAC部署

• 一、主機環境以及配置
• 1.1 Oracle 軟硬體要求
• 1.2 ASM磁碟組規劃
• 1.3 主機網路規劃
• 1.4 作業系統配置部分
• 二、Grid叢集軟體安裝部分
• 三、Oracle DataBase軟體安裝
• 四、DBCA建立例項

一、主機環境以及配置

虛擬主機環境
RAC主機名：rac1、rac2
CPU：Intel® Xeon® CPU E5-2670 v2 @ 2.50GHz 4core
記憶體：32G
操作環境： RHEL 7.4 + Oracle 19c + UDEV
SSD:INTEL P3700 1.6T NVME
儲存：Openfiler
 

1.1 Oracle 軟硬體要求

官方要求作業系統要在RHEL 7.4及以上
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/supported-red-hat-enterprise-linux-7-distributions-for-x86-64.html#GUID-2E11B561-6587-4789-A583-2E33D705E498

禁用透明大頁

rac需要的最少rpm包
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/supported-red-hat-enterprise-linux-7-distributions-for-x86-64.html#GUID-2E11B561-6587-4789-A583-2E33D705E498

asm磁碟大小規劃最小要求
EXTERNAL最少要30G
NORMAL冗餘最少要60G

https://docs.oracle.com/en/database/oracle/oracle-database/19/cwlin/oracle-clusterware-storage-space-requirements.html#GUID-97FD5D40-A65B-4575-AD12-06C491AF3F41

1.2 ASM磁碟組規劃

1.3 主機網路規劃

1.4 作業系統配置部分

兩個節點主機名設定

hostnamectl set-hostname racnnode1

1、關閉防火牆
在兩個節點關閉防火牆

[root@rac1 ~]# systemctl stop firewalld

[root@rac1 ~]# systemctl disabled firewalld

2、關閉selinux
在兩個節點關閉selinux

[root@rac1 ~]# vi /etc/selinux/config
SELINUX=disabled
[root@rac1 ~]#setenforce 0

3、hosts檔案配置
在兩個節點修改hosts檔案

[root@rac1 ~]# vi /etc/hosts
#public ip ent0
192.168.189.211    rac1
192.168.189.212    rac2
192.168.189.213    rac1-vip
192.168.189.214    rac2-vip
10.10.10.211       rac1-prv
10.10.10.212       rac2-prv
192.168.189.215    rac-scan

4、依賴包配置
在兩個節點安裝rac需要的依賴包

[root@rac1 ~]# 
yum install -y binutils
yum install -y compat-libcap1
yum install -y compat-libstdc++-33
yum install -y compat-libstdc++-33.i686
yum install -y gcc
yum install -y gcc-c++
yum install -y glibc
yum install -y glibc.i686
yum install -y glibc-devel
yum install -y glibc-devel.i686
yum install -y ksh
yum install -y libgcc
yum install -y libgcc.i686
yum install -y libstdc++
yum install -y libstdc++.i686
yum install -y libstdc++-devel
yum install -y libstdc++-devel.i686
yum install -y libaio
yum install -y libaio.i686
yum install -y libaio-devel
yum install -y libaio-devel.i686
yum install -y libXext
yum install -y libXext.i686
yum install -y libXtst
yum install -y libXtst.i686
yum install -y libX11
yum install -y libX11.i686
yum install -y libXau
yum install -y libXau.i686
yum install -y libxcb
yum install -y libxcb.i686
yum install -y libXi
yum install -y libXi.i686
yum install -y make
yum install -y sysstat
yum install -y unixODBC
yum install -y unixODBC-devel
yum install -y readline
yum install -y libtermcap-devel
yum install -y bc
yum install -y compat-libstdc++
yum install -y elfutils-libelf
yum install -y elfutils-libelf-devel
yum install -y fontconfig-devel
yum install -y libXi
yum install -y libXtst
yum install -y libXrender
yum install -y libXrender-devel
yum install -y libgcc
yum install -y librdmacm-devel
yum install -y libstdc++
yum install -y libstdc++-devel
yum install -y net-tools
yum install -y nfs-utils
yum install -y python
yum install -y python-configshell
yum install -y python-rtslib
yum install -y python-six
yum install -y targetcli
yum install -y smartmontools

rhel7還需單獨安裝一個獨立包rpm -ivh compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm

5、關閉時間同步ntpd服務

檢查兩節點時間，時區是否相同，並禁止ntp
[root@rac1 ~]#systemctl disable ntpd.service
[root@rac1 ~]#systemctl stop ntpd.service
[root@rac1 ~]#mv /etc/ntp.conf /etc/ntp.conf.orig
[root@rac1 ~]#systemctl status ntpd

時區設定
# timedatectl list-timezones |grep Shanghai    #查詢中國時區的完整名稱
Asia/Shanghai
# timedatectl set-timezone Asia/Shanghai 

6、建立使用者及使用者組
在兩個節點建立oracle，grid使用者組

[root@rac1 ~]
groupadd -g 54321 oinstall  
groupadd -g 54322 dba  
groupadd -g 54323 oper  
groupadd -g 54324 backupdba  
groupadd -g 54325 dgdba  
groupadd -g 54326 kmdba  
groupadd -g 54327 asmdba  
groupadd -g 54328 asmoper  
groupadd -g 54329 asmadmin  
groupadd -g 54330 racdba  
  
useradd -u 54321 -g oinstall -G dba,asmdba,backupdba,dgdba,kmdba,racdba,oper oracle  
useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper,dba grid 

設定grid、oracle密碼

[root@rac1 ~]
echo "oracle" | passwd --stdin oracle

echo "oracle" | passwd --stdin grid

7、建立檔案目錄
在兩個節點建立grid、oracle檔案目錄

[root@rac1 ~]
mkdir -p /u01/app/19.0.0/grid
mkdir -p /u01/app/grid
mkdir -p /u01/app/oracle
mkdir -p /u01/app/oracle/product/19.0.0/db_1
chown -R grid:oinstall /u01
chown -R oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/

8、修改20-nproc.conf
在兩個節點修改/etc/security/limits.d/20-nproc.conf

[root@rac1 ~]vi /etc/security/limits.d/20-nproc.conf
# Change this
#註釋這行*          soft    nproc    1024
# To this增加下面一行
* - nproc 16384

9、修改limits.conf
在兩個節點修改/etc/security/limits.conf

[root@rac1 ~]vi /etc/security/limits.conf
#ORACLE SETTING
grid  soft  nproc 16384
grid  hard  nproc 16384
grid  soft  nofile 1024
grid  hard  nofile 65536
grid  soft  stack 10240
grid  hard  stack 32768
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack  10240
oracle hard stack  32768
oracle hard memlock 3145728
oracle soft memlock 3145728

10、修改login
在兩個節點修改/etc/pam.d/login檔案

[root@rac1 ~]vi /etc/pam.d/login
#ORACLE SETTING
session    required     pam_limits.so

11、修改sysctl.conf
在兩個節點修改/etc/sysctl.conf

[root@rac1 ~]vi /etc/sysctl.conf
#ORACLE SETTING
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmmax = 15461882265
kernel.shmall = 3774873
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586

12、停止avahi-daemon服務
兩個節點分別操作

[root@rac1 ~]# systemctl disable avahi-daemon.socket
[root@rac1 ~]# systemctl disable avahi-daemon.service
ps -ef|grep avahi-daemon
kill -9 pid avahi-daemon

13、新增NOZEROCONF=yes引數

[root@rac1 rpm]# vi /etc/sysconfig/network
NOZEROCONF=yes

14、修改grid使用者環境變數
節點1 grid

[root@rac1 ~]# su - grid

[grid@rac1:/home/grid]$vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022

export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/19.0.0/grid; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
              ulimit -n 65536
  else
   ulimit -u 16384 -n 65536
      fi
    umask 022
fi

節點2 grid

[root@rac2 ~]# su - grid

[grid@rac2:/home/grid]$vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'
export PS1
umask 022

export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_SID=+ASM2; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/19.0.0/grid; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
              ulimit -n 65536
  else
   ulimit -u 16384 -n 65536
      fi
    umask 022
fi

15、修改oracle使用者環境變數
節點1 oracle

[root@rac1 ~]# su - oracle
[oracle@rac1:/home/oracle]$vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'

export PS1
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/19.0.0/db_1; export ORACLE_HOME
ORACLE_SID=orcl1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/19.0.0/db_1/bin:$ORACLE_HOME/bin; export PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
              ulimit -n 65536
  else
   ulimit -u 16384 -n 65536
      fi
    umask 022
fi

節點2 oracle

[root@rac2 ~]# su - oracle
[oracle@rac2:/home/oracle]$vi ~/.bash_profile

PS1="[`whoami`@`hostname`:"'$PWD]$'

export PS1
export TMP=/tmp
export LANG=en_US
export TMPDIR=$TMP
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/19.0.0/db_1; export ORACLE_HOME
ORACLE_SID=orcl2; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG
PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/19.0.0/db_1/bin:$ORACLE_HOME/bin; export PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
THREADS_FLAG=native; export THREADS_FLAG
if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
              ulimit -n 65536
  else
   ulimit -u 16384 -n 65536
      fi
    umask 022
fi

16、修改root環境變數
加入grid使用者$ORACLE_HOME

[root@rac1 opt]# vi ~/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:/u01/app/19.0.0/grid/bin:$HOME/bin

export PATH

17、禁用透明大頁
oracle官方建議關閉透明大頁

1 修改grub檔案
cp /etc/default/grub /etc/default/grub.bak
vi /etc/default/grub

2 增加一行transparent_hugepage=never到尾部
GRUB_CMDLINE_LINUX=
"rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet transparent_hugepage=never

3 執行命令
grub2-mkconfig -o /boot/grub2/grub.cfg


4 不重啟生效
[root@rac1 ~]#echo never > /sys/kernel/mm/transparent_hugepage/enabled  

5 檢視是否禁用透明大頁
[root@rac1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

[root@rac2 ~]# grep AnonHugePages /proc/meminfo
AnonHugePages:         0 kB---------->>>返回值若是零，代表成功禁用THP

18、修改共享儲存、udev規則
在節點1和節點2，固定裝置名稱並且賦予grid屬性，執行如下命令，用來生成udev規則

1 取得DM_UUID

cd /dev/mapper
for i in `ls mpath*`; 
do printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/$i |grep -i dm_uuid)"; 
done

2 生成rules檔案

vi /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c455232776b726c352d4557426d2d43776568",SYMLINK+="asm_crs_1_30g",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4552477a424f6e592d644b49572d56614c4b",SYMLINK+="asm_crs_2_30g",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c45524e5871614c522d74376d482d5a564272",SYMLINK+="asm_crs_3_30g",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-14f504e46494c4552386e674a70452d4959334f2d47745667",SYMLINK+="asm_data_1_20g",OWNER="grid",GROUP="asmadmin",MODE="0660"

3重新載入udev
udevadm control --reload-rules
udevadm trigger

4檢視asm磁碟

[root@rac1 ~]# ll /dev/asm*

19、節點2拷貝節點1的udev規則檔案

scp rac1:/etc/udev/rules.d/99-oracle-asmdevices.rules /etc/udev/rules.d/99-oracle-asmdevices.rules

二、Grid叢集軟體安裝部分

1、上傳叢集軟體包
在節點1上傳grid安裝包

[root@rac1 grid]# ll
-rwxr-xr-x 1 grid oinstall 5.1G Jan 28 15:58 LINUX.X64_180000_grid_home.zip

2、解壓grid安裝包
在19C中需要把grid包解壓放到grid使用者下ORACLE_HOME目錄內
解壓檔案到/u01/app/19.0.0/grid

[grid@rac2 grid]$ cd /u01/app/19.0.0/grid
[grid@rac2 grid]$ unzip LINUX.X64_193000_grid_home.zip

3、進入grid叢集軟體目錄執行安裝

[grid@rac1 grid]$ ./gridSetup.sh 

4、GUI安裝步驟
4.1 建立新的叢集

4.2 配置叢集名稱以及scan名稱

4.3 節點互信

4.4 公網、私網網段選擇

4.5 選擇asm儲存

4.6 選擇配置GIMR

4.7 這裡選擇ocr、voting file與gimr放在一起

4.8 選擇asm磁碟組

4.9 輸入密碼

4.10 保持預設

4.11 保持預設

4.12保持預設

4.13 確認base目錄

4.14保持預設

4.15這裡可以選擇自動root執行指令碼

4.16 預安裝檢查

4.17 解決相關依賴後，忽略如下報錯

4.18 安裝前摘要

4.19 如下警告可以忽略

如上警告是由於沒有使用DNS解析造成可忽略

4.20 grid安裝完畢

三、Oracle DataBase軟體安裝

解壓壓縮包到oracle使用者的ORACLE_HOME目錄，執行安裝

[oracle@rac1 db_1]$ pwd
/u01/app/oracle/product/19.0.0/db_1
[oracle@rac1 db_1]$ unzip /opt/LINUX.X64_193000_db_home.zip
[oracle@rac1 db_1]$ ./runInstaller

• 1
• 2
• 3
• 4
• 5

1 software only

2 選擇rac

3 ssh互信驗證

4 保持預設

5 保持預設

6 保持預設

7 可選擇自動執行root指令碼

8 預安裝前檢查
忽略如下警告

9 安裝摘要

10 開始安裝

11 執行root指令碼

12 oracle軟體安裝部分結束

四、DBCA建立例項

在建立例項前先通過grid的asmca建立磁碟組，隨後再用oracle dbca建立例項
1、通過asmca建立oracle資料檔案磁碟組
[grid@rac1 grid]$ asmca

data磁碟組

asm磁碟組建立完畢

2、dbca圖形化安裝例項
[oracle@rac1 db_1]$ dbca
常規配置DBCA

DBCA完成

檢視叢集狀態

[root@rac1 ~]# crsctl stat res -t           
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS_GIMR.GHCHKPT.advm
               OFFLINE OFFLINE      rac1                     STABLE
               OFFLINE OFFLINE      rac2                     STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1                     STABLE
               ONLINE  ONLINE       rac2                     STABLE
ora.chad
               ONLINE  ONLINE       rac1                     STABLE
               ONLINE  ONLINE       rac2                     STABLE
ora.crs_gimr.ghchkpt.acfs
               OFFLINE OFFLINE      rac1                     STABLE
               OFFLINE OFFLINE      rac2                     STABLE
ora.helper
               OFFLINE OFFLINE      rac1                     IDLE,STABLE
               OFFLINE OFFLINE      rac2                     IDLE,STABLE
ora.net1.network
               ONLINE  ONLINE       rac1                     STABLE
               ONLINE  ONLINE       rac2                     STABLE
ora.ons
               ONLINE  ONLINE       rac1                     STABLE
               ONLINE  ONLINE       rac2                     STABLE
ora.proxy_advm
               OFFLINE OFFLINE      rac1                     STABLE
               OFFLINE OFFLINE      rac2                     STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       rac1                     STABLE
      2        ONLINE  ONLINE       rac2                     STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.CRS_GIMR.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rac1                     STABLE
      2        ONLINE  ONLINE       rac2                     STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.DATA.dg(ora.asmgroup)
      1        ONLINE  ONLINE       rac1                     STABLE
      2        ONLINE  ONLINE       rac2                     STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1                     STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rac1                     169.254.11.15 10.10.
                                                             10.211,STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       rac1                     Started,STABLE
      2        ONLINE  ONLINE       rac2                     Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       rac1                     STABLE
      2        ONLINE  ONLINE       rac2                     STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       rac1                     STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rac1                     Open,STABLE
ora.orcl.db
      1        ONLINE  ONLINE       rac1                     Open,HOME=/u01/app/o
                                                             racle/product/19.0.0
                                                             /db_1,STABLE
      2        ONLINE  ONLINE       rac2                     Open,HOME=/u01/app/o
                                                             racle/product/19.0.0
                                                             /db_1,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       rac1                     STABLE
ora.rac1.vip
      1        ONLINE  ONLINE       rac1                     STABLE
ora.rac2.vip
      1        ONLINE  ONLINE       rac2                     STABLE
ora.rhpserver
      1        OFFLINE OFFLINE                               STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rac1                     STABLE
--------------------------------------------------------------------------------

檢視資料庫版本

[oracle@rac1 db_1]$ sqlplus / as sysdba

SQL> col banner_full for a120
SQL> select BANNER_FULL from v$version;

BANNER_FULL
--------------------------------------------------------------------------------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production