最近給客戶基於SuSe 11 SP3下多路徑部署Oracle 10g RAC。SuSe 11下用10g，也算一朵奇葩，連篇文件都比較難找，誰叫Oracle太貴呢。下面主要是描述了在該環境下如何去配置多路徑。由於10g下的ocr與votingdisk不能直接存放到asm磁碟，所以依舊要使用raw裝置方式來儲存。下文供大家參考。

一、檢視當前scsi裝置及獲取裝置wwid
#使用 cat /proc/partitions或者/sbin/sfdisk命令檢視當前系統中的scsi裝置
suse11a:~ # more /proc/partitions
major minor  #blocks  name

   8        0  877264896 sda
   8        1     200812 sda1
   8        2  104856255 sda2
   8        3  104856255 sda3
   8        4  667340100 sda4
   8       16 2097152000 sdb
   8       17 2097150976 sdb1
               .....
   8      240    1048576 sdp
  65        0    1048576 sdq
               .....
  65       48 2097152000 sdt
  65       49 2097150976 sdt1
 253        0 2097152000 dm-0
 253        1 2097152000 dm-1
 253        2 1751121920 dm-2
 253        3    1048576 dm-3
 253        4    1048576 dm-4
 253        5 1751120896 dm-5
 253        6 2097150976 dm-6
 253        7 2097150976 dm-7
 
#獲取多路徑裝置的wwid，使用下面的命令來獲取當前系統中多路徑裝置的wwid

### sda: 3600605b007030d301a192f2d13c017a0
### sda1: 3600605b007030d301a192f2d13c017a0
     .....................
### sdt: 360022a11000cb6b10a902db200000000
### sdt1: 360022a11000cb6b10a902db200000000
#獲取後，過濾本地磁碟及重複的wwid

二、配置mulitpatch.conf檔案
#配置mulitpatch.conf檔案的主要目的是用於使用有意義，使用者自定義的裝置名稱，持久化多路徑裝置
#這個步驟是與配置單路徑裝置最大的區別
#下面是已經配置過後的multipath.conf檔案
suse11a:~ # cp /etc/multipath.conf /etc/multipath.conf.bak
suse11a:~ # vi /etc/multipath.conf
suse11a:~ # grep -v ^# /etc/multipath.conf
defaults {
        user_friendly_names yes
}
blacklist {
        devnode ^hd[a-z]
}
multipaths {
      multipath {
              wwid                    36200bc710099b9190ba6fc350000000a
              alias                   mpatha
       }

      multipath {
              wwid                    36200bc710099b9190ba7109b0000000b
              alias                   mpathb
       }
      multipath {
              wwid                    36200bc710099b9190ab1155300000009
              alias                   mpathc
       }
      multipath {
              wwid                    360022a11000cb6b10a902db200000000
              alias                   mpathd
       }
       multipath {
              wwid                    360022a11000cb6b10a9116d000000002
              alias                   mpathe
       }
}

三、校驗多路徑裝置
#使用下面的命令用於多路徑配置生效
suse11a:~ # service multipathd stop
suse11a:~ # service multipathd start

#下面校驗多路徑裝置
suse11a:~ # multipath -ll | grep mpath
mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T         
mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T         
mpathc (36200bc710099b9190ab1155300000009) dm-2 HUAWEI  ,S2600T         
mpathb (36200bc710099b9190ba7109b0000000b) dm-4 HUAWEI  ,S2600T         
mpatha (36200bc710099b9190ba6fc350000000a) dm-3 HUAWEI  ,S2600T

#下面是裝置的詳細資訊
suse11a:~ # multipath -ll
mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T         
size=2.0T features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 5:0:0:1 sdc 8:32  active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 5:0:3:1 sdk 8:160 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 6:0:2:1 sds 65:32 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 6:0:3:1 sdu 65:64 active ready running
mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T         
size=2.0T features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 5:0:0:0 sdb 8:16  active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 5:0:3:0 sdj 8:144 active ready running
|-+- policy='service-time 0' prio=1 status=enabled
| `- 6:0:2:0 sdr 65:16 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 6:0:3:0 sdt 65:48 active ready running
            .........

suse11a:~ # dmsetup ls |sort
mpatha  (253:3)
mpathb  (253:4)
mpathc  (253:2)
mpathc_part1    (253:5)
mpathd  (253:0)
mpathd_part1    (253:6)
mpathe  (253:1)
mpathe_part1    (253:7)
   
#以下命令均可以用於校驗多路徑裝置
# ll /dev/disk/by-id/
# ls -l /dev/dm-*
# ll /dev/mapper/

#/dev/dm-*下面的為動態名，即系統重啟後會發生變化，此部分用於os 內部使用。
#/dev/mapper/下面檢視到的內容為持久化名稱，也即是我們在mulitpatch.conf檔案中定義的。

   
四、配置RAW
#在Oracle 10g R2 rac安裝的過程中，OUI不能夠校驗共享的快涉筆，因此需要使用raw。Oracle 11g R1後可以直接使用多路徑裝置。
#使用udev來管理raw，編輯raw檔案，如下
suse11a:~ # cat /etc/raw | grep -v \#
raw1:mapper/mpatha
raw2:mapper/mpathb
raw3:mapper/mpathc
raw4:mapper/mpathd
raw5:mapper/mpathe

#重啟raw以及配置開機啟動raw
suse11a:~ # rcraw start
suse11a:~ # chkconfig raw on
suse11a:~ # raw -qa
/dev/raw/raw1:  bound to major 253, minor 3
/dev/raw/raw2:  bound to major 253, minor 4
/dev/raw/raw3:  bound to major 253, minor 2
/dev/raw/raw4:  bound to major 253, minor 0
/dev/raw/raw5:  bound to major 253, minor 1

#測試raw的可訪問性
# dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=100
100+0 records in
100+0 records out
102400 bytes (102 kB) copied, 0.762352 seconds, 134 kB/s

#配置許可權，該許可權用於oracle使用者訪問裸裝置
# Author : Leshami
# Blog   : http://blog.csdn.net/leshami
suse11a:~ # more /etc/udev/rules.d/99-raw.rules
KERNEL=="raw[1-2]*", GROUP="oinstall", MODE="664"
KERNEL=="raw[3-5]*", OWNER="oracle", GROUP="oinstall", MODE="664"

五、安裝clusterware
#安裝clusterware時，關於ocr和votingdisk，選擇/dev/raw/raw1，/dev/raw/raw2，如果有配置冗餘，此處應該為多個裸裝置。
#校驗安轉好clusterware後裸裝置的使用情況
suse11a:oracle:ora10g1 > ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          2
         Total space (kbytes)     :    1048296
         Used space (kbytes)      :       4592
         Available space (kbytes) :    1043704
         ID                       :  892902908
         Device/File Name         : /dev/raw/raw1
                                    Device/File integrity check succeeded

                                    Device/File not configured

         Cluster registry integrity check succeeded

suse11a:oracle:ora10g1 > crsctl query  css votedisk
 0.     0    /dev/raw/raw2

located 1 votedisk(s). 
#對於ASM磁碟可以通過dbca來建立，直接使用/dev/raw/raw*以及/dev/mapper/*。當然suse 11支援asmlib，也可以通過asmlib方式來訪問asm磁碟。