儲存高可用——DRBD單主模式的搭建
儲存高可用——DRBD
DRDB官網：http://drbd.linbit.com/home/what-is-drbd/
拓撲圖說明：
分別在兩臺機器上實現網路 raid0 功能
試驗環境：workstation 10
作業系統：rhel6.2(64位)
核心版本：2.6.32-220.el6.x86_64
兩臺節點配置
| 節點角色 | 主機名 | IP地址 | DRBD分割槽 |
| Primary節點 | node01 | 192.168.1.12 | /dev/sdb1 |
| Secondary節點 | node02 | 192.168.1.13 | /dev/sdb1 |
軟體下載列表地址： http://oss.linbit.com/drbd/
軟體版本：drbd-8.4.3.tar.gz
------------------------------------以下是node01上的操作------------------------------------
[[email protected]~]#sed-i's/HOSTNAME=localhost.localdomain/HOSTNAME=node01/g'/etc/sysconfig/network#修改主機名
[[email protected]~]#echo-e"192.168.1.12node01\n192.168.1.13node02">>/etc/hosts#設定域名臨時解析
[[email protected]~]#cat>fdisk.sh#建立自動分割槽指令碼 #!/bin/bash fdisk/dev/sdb<<EOF n p 1 1 wq EOF
[[email protected]~]#shfdisk.sh#分割槽
[[email protected]~]#fdisk-l|grep"sd" Disk/dev/mapper/VolGroup-lv_rootdoesn'tcontainavalidpartitiontable Disk/dev/mapper/VolGroup-lv_swapdoesn'tcontainavalidpartitiontable Disk/dev/sda:10.7GB,10737418240bytes /dev/sda1*16451200083Linux /dev/sda264130699727368eLinuxLVM Disk/dev/sdb:16.1GB,16106127360bytes /dev/sdb11195815727603+83Linux
注：/dev/sdb1為我上一步所分的區
[[email protected]~]#mkdir/data#建立分割槽的掛載目錄
[[email protected]~]#reboot#這裡最好重啟下，目的是使剛才的設定生效
[[email protected]~]#uname-r#檢視核心版本 2.6.32-220.el6.x86_64
[[email protected]~]#yuminstall-ykernel-headerskernel-develflex warning:kernel-headers-2.6.32-220.el6.x86_64.rpm:HeaderV3RSA/SHA256Signature,keyIDfd431d51:NOKEY Preparing...###########################################[100%] 1:kernel-headers###########################################[100%]
注：kernel-devel和kernel-headers版本要和uname -r顯示的核心版本一致,建議用本地源安裝
[[email protected]~]#yuminstall-ywget#安裝wget工具
[[email protected]~]#wgethttp://oss.linbit.com/drbd/8.4/drbd-8.4.3.tar.gz#下載drbd軟體
[[email protected]~]#tarxvfdrbd-8.4.3.tar.gz#解壓軟體包 drbd-8.4.3/.gitignore drbd-8.4.3/COPYING drbd-8.4.3/ChangeLog ......
[[email protected]drbd-8.4.3]#yuminstall-y*gcc*#安裝編譯工具
[[email protected]drbd-8.4.3]#./configure--prefix=/usr/local/drbd--with-km
注意：--with-km是啟用核心模組
[[email protected]drbd-8.4.3]#makeKDIR=/usr/src/kernels/2.6.32-220.el6.x86_64/
注：KDIR的路徑 （這個核心原始碼路徑需要根據自己的系統修改）
[[email protected]drbd-8.4.3]#makeinstall
[[email protected]drbd-8.4.3]#mkdir-p/usr/local/drbd/var/run/drbd [[email protected]drbd-8.4.3]#cp/usr/local/drbd/etc/rc.d/init.d/drbd/etc/rc.d/init.d/ [[email protected]drbd-8.4.3]#chkconfig--adddrbd [[email protected]drbd-8.4.3]#chkconfigdrbdon
以下是安裝drbd模組：
[[email protected]drbd-8.4.3]#cddrbd [[email protected]drbd]#makeclean rm-rf.tmp_versionsModule.markersModule.symversmodules.order rm-f*.[oas]*.ko.*.cmd.*.d.*.tmp*.mod.c.*.flags.depend.kernel* rm-fcompat/*.[oas]compat/.*.cmd [[email protected]drbd]#makeKDIR=/usr/src/kernels/2.6.32-220.el6.x86_64/ Callingtoplevelmakefileofkernelsourcetree,whichIbelieveisin KDIR=/usr/src/kernels/2.6.32-220.el6.x86_64/ make-C/usr/src/kernels/2.6.32-220.el6.x86_64/SUBDIRS=/root/drbd-8.4.3/drbdmodules make[1]:Enteringdirectory`/usr/src/kernels/2.6.32-220.el6.x86_64' CC[M]/root/drbd-8.4.3/drbd/drbd_buildtag.o CC[M]/root/drbd-8.4.3/drbd/drbd_bitmap.o CC[M]/root/drbd-8.4.3/drbd/drbd_proc.o CC[M]/root/drbd-8.4.3/drbd/drbd_worker.o ......
[[email protected]drbd]#cpdrbd.ko/lib/modules/`uname-r`/kernel/lib/ [[email protected]drbd]#modprobedrbd#載入drbd模組 [[email protected]drbd]#lsmod|grepdrbd#驗證模組載入是否成功 drbd3286260 libcrc32c12461drbd
以下是drbd配置部分：
[[email protected]etc]#pwd /usr/local/drbd/etc [[email protected]etc]#moredrbd.conf#主配置檔案裡面已經包含了全域性配置檔案和drbd目錄下以.res結尾檔案，所需要修改全域性配置檔案和建立.res檔案即可。 #Youcanfindanexamplein/usr/share/doc/drbd.../drbd.conf.example include"drbd.d/global_common.conf"; include"drbd.d/*.res";
[[email protected]drbd.d]#viglobal_common.conf#修改drbd全域性配置檔案
global {
usage-count no;
# minor-count dialog-refresh disable-ip-verification
}
common {
protocol C;#使用drbd的同步協議
handlers {
# These are EXAMPLE handlers only.
# They may have severe implications,
# like hard resetting the node under certain circumstances.
# Be careful when chosing your poison.
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; #去掉註釋
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; #去掉註釋
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; #去掉註釋
# fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
# split-brain "/usr/lib/drbd/notify-split-brain.sh root";
# out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
# before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k";
# after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
}
startup {
# wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb
}
options {
# cpu-mask on-no-data-accessible
}
disk {
on-io-error detach;#配置I/O錯誤處理策略為分離
rate 50M;#設定主備節點同步時的網路速率
}
net {
# protocol timeout max-epoch-size max-buffers unplug-watermark
# connect-int ping-int sndbuf-size rcvbuf-size ko-count
# allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri
# after-sb-1pri after-sb-2pri always-asbp rr-conflict
# ping-timeout data-integrity-alg tcp-cork on-congestion
# congestion-fill congestion-extents csums-alg verify-alg
# use-rle}
注：加黑字部分為修改或新增
[[email protected]drbd.d]#touch/usr/local/drbd/etc/drbd.d/drbd.res#建立資源配置檔案
[[email protected] drbd.d]# more drbd.res
resourcer1{
onnode01{
address192.168.1.12:7801;#設定DRBD的監聽埠,用於與另一臺主機通訊。
volume0{
device/dev/drbd1;
disk/dev/sdb1;
meta-diskinternal;
}
}
onnode02{
address192.168.1.13:7801;#設定DRBD的監聽埠,用於與另一臺主機通訊。
volume0{
device/dev/drbd1;
disk/dev/sdb1;
meta-diskinternal;
}
}
}[[email protected]drbd.d]#drbdadmcreate-mdall#初始化資源 Writingmetadata... initializingactivitylog NOTinitializingbitmap Newdrbdmetadatablocksuccessfullycreated.
--------------------------------------以上是node01上的操作----------------------------------------
node02上的操作和node01上的操作全部相同,在此就不重複了
[[email protected]drbd.d]#/etc/init.d/drbdstart#node01啟動drbd
Starting DRBD resources: [
create res: r1
prepare disk: r1
adjust disk: r1
adjust net: r1
]
..........
***************************************************************
DRBD's startup script waits for the peer node(s) to appear.
- In case this node was already a degraded cluster before the
reboot the timeout is 0 seconds. [degr-wfc-timeout]
- If the peer was available before the reboot the timeout will
expire after 0 seconds. [wfc-timeout]
(These values are for resource 'r1'; 0 sec -> wait forever)
To abort waiting enter 'yes' [ -- ]: [ 10]:[ 11]:[ 12]:
.
[[email protected]~]#/etc/init.d/drbdstart#node01啟動drbd StartingDRBDresources:[ createres:r1 preparedisk:r1 adjustdisk:r1 adjustnet:r1 ]
[
[[email protected]~]#netstat-lanput|grep7801#確認監聽埠起來
tcp 0 0 192.168.1.12:7801 192.168.1.13:41242 ESTABLISHED -
tcp 0 0 192.168.1.12:50401 192.168.1.13:7801 ESTABLISHED -
[[email protected]~]#netstat-lanput|grep7801#確認監聽埠起來
tcp 0 0 192.168.1.13:7801 192.168.1.12:50401 ESTABLISHED -
tcp 0 0 192.168.1.13:41242 192.168.1.12:7801 ESTABLISHED -
[email protected]~]#cat/proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2015-01-25 12:08:01
1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:15727084
[[email protected]~]#drbdadm----overwrite-data-of-peerprimaryall#設定node01為主節點 [[email protected]~]#cat/proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2015-01-25 12:08:01
1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---n-
ns:197192 nr:0 dw:0 dr:201368 al:0 bm:11 lo:1 pe:2 ua:5 ap:0 ep:1 wo:f oos:15531500
[>....................] sync'ed: 1.3% (15164/15356)M
finish: 0:01:19 speed: 195,584 (195,584) K/sec
[[email protected]~]#mkfs.ext4/dev/drbd1#格式化檔案系統
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
983040 inodes, 3931771 blocks
......
[[email protected]~]#mount/dev/drbd1/data/#掛載檔案系統
[[email protected] ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
7.5G 1.3G 5.9G 18% /
tmpfs 495M 0 495M 0% /dev/shm
/dev/sda1 485M 31M 429M 7% /boot
/dev/sr0 3.4G 3.4G 0 100% /iso
/dev/drbd1 15G 166M 14G 2% /data
[[email protected]~]#cp/boot/./data/-rvf#模擬資料的寫入
`/boot/./.vmlinuz-2.6.32-220.el6.x86_64.hmac' -> `/data/./.vmlinuz-2.6.32-220.el6.x86_64.hmac'
`/boot/./System.map-2.6.32-220.el6.x86_64' -> `/data/./System.map-2.6.32-220.el6.x86_64'
`/boot/./symvers-2.6.32-220.el6.x86_64.gz' -> `/data/./symvers-2.6.32-220.el6.x86_64.gz'
`/boot/./initramfs-2.6.32-220.el6.x86_64.img' -> `/data/./initramfs-2.6.32-220.el6.x86_64.img'
......
[[email protected]~]#ls/data/
config-2.6.32-220.el6.x86_64 grub System.map-2.6.32-220.el6.x86_64
efi initramfs-2.6.32-220.el6.x86_64.img symvers-2.6.32-220.el6.x86_64.gz
vmlinuz-2.6.32-220.el6.x86_64
主從切換
以下是主從切換:因為上面設定的是node01為primary,node02為secondary,現在目標是將node01設定為secondary，將node02設定為primary:
首先在node01上執行以下命令：
[[email protected]~]#ls/data/
config-2.6.32-220.el6.x86_64 grub lost+found System.map-2.6.32-220.el6.x86_64
efi initramfs-2.6.32-220.el6.x86_64.img symvers-2.6.32-220.el6.x86_64.gz vmlinuz-2.6.32-220.el6.x86_64
[[email protected]~]#cat/proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2015-01-25 12:08:01
1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:12 nr:8 dw:20 dr:2066 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected]~]#umount/data/ [[email protected]~]#drbdadmsecondaryall
然後在node02上執行以下命令： [[email protected]~]#cat/proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2015-01-25 13:16:26
1: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r-----
ns:8 nr:16 dw:24 dr:1033 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected]~]#drbdadmprimaryall
[[email protected]~]#cat/proc/drbd
version: 8.4.3 (api:1/proto:86-101)
GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by [email protected], 2015-01-25 13:16:26
1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:8 nr:16 dw:24 dr:1697 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[[email protected]~]#mount/dev/drbd1/data/#並不需要在node02上在對磁碟mkfs.ext4
[[email protected]~]#ls/data/#biao#切換後看到拷貝的資料就表示成功
config-2.6.32-220.el6.x86_64 grub System.map-2.6.32-220.el6.x86_64
efi initramfs-2.6.32-220.el6.x86_64.img symvers-2.6.32-220.el6.x86_64.gz
vmlinuz-2.6.32-220.el6.x86_64
drbd腦裂的模擬借鑑：
http://zhangmaojun023.blog.163.com/blog/static/10919584620128270550813/
截圖是對drbd的一些說明
轉載於:https://blog.51cto.com/watchman110/1608052