分散式安裝部署MinIO
阿新 • • 發佈：2022-03-10
官方文件地址：http://docs.minio.org.cn/docs/master/distributed-minio-quickstart-guide
前提條件：分散式Minio至少需要4個硬碟，使用分散式Minio自動引入了糾刪碼功能。
這裡演示使用一臺主機上的4塊磁碟，每個磁碟1G
官方步驟：
export MINIO_ACCESS_KEY=<ACCESS_KEY> export MINIO_SECRET_KEY=<SECRET_KEY> minio server http://192.168.1.11/export1 http://192.168.1.12/export2 \ http://192.168.1.13/export3 http://192.168.1.14/export4 \ http://192.168.1.15/export5 http://192.168.1.16/export6 \ http://192.168.1.17/export7 http://192.168.1.18/export8
實際操作步驟：
1.在一臺主機上建立4個不同的路徑模擬4個不同的磁碟來進行
從執行結果來看，這種方式行不通
export MINIO_ACCESS_KEY=admin export MINIO_SECRET_KEY=12345678 mkdir -p /mnt/{data1,data2,data3,data4} /usr/local/minio/bin/minio server http://192.168.20.102/mnt/data1 http://192.168.20.102/mnt/data2 http://192.168.20.102/mnt/data3 http://192.168.20.102/mnt/data4 --console-address ":9001" # 操作完畢後提示資訊如下 WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD Formatting 1st pool, 1 set(s), 4 drives per set. WARNING: Host 192.168.20.102 has more than 2 drives of set. A host failure will result in data becoming unavailable. API: SYSTEM() Time: 09:24:00 UTC 03/10/2022 Error: Disk `/mnt/data1` is part of root disk, will not be used (*errors.errorString) 7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown() 6: cmd/format-erasure.go:792:cmd.initFormatErasure() 5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats() 4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure() 3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools() 2: cmd/server-main.go:640:cmd.newObjectLayer() 1: cmd/server-main.go:491:cmd.serverMain() API: SYSTEM() Time: 09:24:00 UTC 03/10/2022 Error: Disk `/mnt/data2` is part of root disk, will not be used (*errors.errorString) 7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown() 6: cmd/format-erasure.go:792:cmd.initFormatErasure() 5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats() 4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure() 3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools() 2: cmd/server-main.go:640:cmd.newObjectLayer() 1: cmd/server-main.go:491:cmd.serverMain() API: SYSTEM() Time: 09:24:00 UTC 03/10/2022 Error: Disk `/mnt/data3` is part of root disk, will not be used (*errors.errorString) 7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown() 6: cmd/format-erasure.go:792:cmd.initFormatErasure() 5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats() 4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure() 3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools() 2: cmd/server-main.go:640:cmd.newObjectLayer() 1: cmd/server-main.go:491:cmd.serverMain() API: SYSTEM() Time: 09:24:00 UTC 03/10/2022 Error: Disk `/mnt/data4` is part of root disk, will not be used (*errors.errorString) 7: cmd/erasure-sets.go:1257:cmd.markRootDisksAsDown() 6: cmd/format-erasure.go:792:cmd.initFormatErasure() 5: cmd/prepare-storage.go:222:cmd.connectLoadInitFormats() 4: cmd/prepare-storage.go:306:cmd.waitForFormatErasure() 3: cmd/erasure-server-pool.go:91:cmd.newErasureServerPools() 2: cmd/server-main.go:640:cmd.newObjectLayer() 1: cmd/server-main.go:491:cmd.serverMain() Formatting 1st pool, 1 set(s), 4 drives per set. WARNING: Host 192.168.20.102 has more than 2 drives of set. A host failure will result in data becoming unavailable.
改進步驟：
# 根據上一步的執行提示，更新設定的環境變數名 export MINIO_ROOT_USER=admin export MINIO_ROOT_PASSWORD=12345678 # 新加四塊磁碟，每個磁碟空間1G，不分割槽，直接格式化後掛載 # 不重啟，直接重新整理磁碟資料匯流排，獲取新加的磁碟 for host in $(ls /sys/class/scsi_host) ; do echo "- - -" > /sys/class/scsi_host/$host/scan; done # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 40G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 39G 0 part ├─centos-root 253:0 0 35.1G 0 lvm / └─centos-swap 253:1 0 3.9G 0 lvm [SWAP] sdb 8:16 0 1G 0 disk sdc 8:32 0 1G 0 disk sdd 8:48 0 1G 0 disk sde 8:64 0 1G 0 disk # mkfs.ext4 /dev/sdb mke2fs 1.42.9 (28-Dec-2013) /dev/sdb is entire device, not just one partition! 無論如何也要繼續? (y,n) y 檔案系統標籤= OS type: Linux 塊大小=4096 (log=2) 分塊大小=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 65536 inodes, 262144 blocks 13107 blocks (5.00%) reserved for the super user 第一個資料塊=0 Maximum filesystem blocks=268435456 8 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: 完成 正在寫入inode表: 完成 Creating journal (8192 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 [root@myrabbit3 ~]# mkfs.ext4 /dev/sdc mke2fs 1.42.9 (28-Dec-2013) /dev/sdc is entire device, not just one partition! 無論如何也要繼續? (y,n) y 檔案系統標籤= OS type: Linux 塊大小=4096 (log=2) 分塊大小=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 65536 inodes, 262144 blocks 13107 blocks (5.00%) reserved for the super user 第一個資料塊=0 Maximum filesystem blocks=268435456 8 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: 完成 正在寫入inode表: 完成 Creating journal (8192 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 [root@myrabbit3 ~]# mkfs.ext4 /dev/sdd mke2fs 1.42.9 (28-Dec-2013) /dev/sdd is entire device, not just one partition! 無論如何也要繼續? (y,n) y 檔案系統標籤= OS type: Linux 塊大小=4096 (log=2) 分塊大小=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 65536 inodes, 262144 blocks 13107 blocks (5.00%) reserved for the super user 第一個資料塊=0 Maximum filesystem blocks=268435456 8 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: 完成 正在寫入inode表: 完成 Creating journal (8192 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 [root@myrabbit3 ~]# mkfs.ext4 /dev/sde mke2fs 1.42.9 (28-Dec-2013) /dev/sde is entire device, not just one partition! 無論如何也要繼續? (y,n) y 檔案系統標籤= OS type: Linux 塊大小=4096 (log=2) 分塊大小=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 65536 inodes, 262144 blocks 13107 blocks (5.00%) reserved for the super user 第一個資料塊=0 Maximum filesystem blocks=268435456 8 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: 完成 正在寫入inode表: 完成 Creating journal (8192 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 [root@myrabbit3 ~]# mount /dev/sdb /mnt/data1/ [root@myrabbit3 ~]# mount /dev/sdc /mnt/data2/ [root@myrabbit3 ~]# mount /dev/sdd /mnt/data3/ [root@myrabbit3 ~]# mount /dev/sde /mnt/data4/ # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 40G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 39G 0 part ├─centos-root 253:0 0 35.1G 0 lvm / └─centos-swap 253:1 0 3.9G 0 lvm [SWAP] sdb 8:16 0 1G 0 disk /mnt/data1 sdc 8:32 0 1G 0 disk /mnt/data2 sdd 8:48 0 1G 0 disk /mnt/data3 sde 8:64 0 1G 0 disk /mnt/data4 # /usr/local/minio/bin/minio server http://192.168.20.102/mnt/data1 http://192.168.20.102/mnt/data2 http://192.168.20.102/mnt/data3 http://192.168.20.102/mnt/data4 --console-address ":9001" WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD Automatically configured API requests per node based on available memory on the system: 55 Status: 4 Online, 0 Offline. API: http://192.168.20.102:9000 http://127.0.0.1:9000 RootUser: admin RootPass: 12345678 Console: http://192.168.20.102:9001 http://127.0.0.1:9001 RootUser: admin RootPass: 12345678 Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.20.102:9000 admin 12345678 Documentation: https://docs.min.io