ceph運維-物件儲存
阿新 • • 發佈：2020-12-02
一 摘要
二 環境資訊
三 物件儲存運維
（一）物件儲存部署
物件儲存部署到三臺server 節點上。
3.1.1 三臺叢集節點部署物件儲存
3.1.1.1 下載離線安裝包
[cephadmin@ceph001 cephcluster]$ sudo yum -y install --downloadonly --downloaddir=/home/cephadmin/software/cephradosgw/ ceph-radosgw
3.1.1.2 安裝（三臺sever 節點都需要安裝）
經檢查我這邊三臺節點都安裝過了
檢查是否安裝
[root@ceph003 ~]# rpm -qa | grep ceph-ra ceph-radosgw-14.2.15-0.el7.x86_64 [root@ceph003 ~]#
安裝命令
[cephadmin@ceph001 cephcluster]$ sudo yum -y install ceph-radosgw
3.1.1.2 部署（ceph-deploy 部署）
在部署節點執行
[cephadmin@ceph001 cephcluster]$ pwd /home/cephadmin/cephcluster [cephadmin@ceph001 cephcluster]$ ceph-deploy rgw create ceph001 ceph002 ceph003 [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephadmin/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /bin/ceph-deploy rgw create ceph001 ceph002 ceph003 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] rgw : [('ceph001', 'rgw.ceph001'), ('ceph002', 'rgw.ceph002'), ('ceph003', 'rgw.ceph003')] [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] subcommand : create [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7f21975ec290> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] func : <function rgw at 0x7f2197ea6f50> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.rgw][DEBUG ] Deploying rgw, cluster ceph hosts ceph001:rgw.ceph001 ceph002:rgw.ceph002 ceph003:rgw.ceph003
預設埠是7480
我這裡啟動失敗了，而且/etc/ceph/ceph.conf 檔案許可權被改為root 了。
[root@ceph002 ceph]# ll total 12 -rw------- 1 cephadmin cephadmin 151 Nov 30 17:25 ceph.client.admin.keyring -rw-r--r-- 1 root root 313 Dec 2 10:23 ceph.conf -rw-r--r-- 1 cephadmin cephadmin 92 Nov 24 03:33 rbdmap -rw------- 1 cephadmin cephadmin 0 Nov 30 17:16 tmp4_tYYV [root@ceph002 ceph]# pwd /etc/ceph [root@ceph002 ceph]#
檢視/var/log/message 日誌
Dec 2 10:53:19 ceph001 radosgw: failed to fetch mon config (--no-mon-config to skip)
Dec 2 10:53:19 ceph001 systemd: [email protected]: main process exited, code=exited, status=1/FAILURE
Dec 2 10:53:19 ceph001 systemd: Unit [email protected] entered failed state.
Dec 2 10:53:19 ceph001 systemd: [email protected] failed.
Dec 2 10:53:19 ceph001 systemd: [email protected] holdoff time over, scheduling restart.
Dec 2 10:53:19 ceph001 systemd: Stopped Ceph rados gateway.
Dec 2 10:53:19 ceph001 systemd: Started Ceph rados gateway.
Dec 2 10:53:19 ceph001 radosgw: 2020-12-02 10:53:19.753 7f77a49fd900 -1 auth: unable to find a keyring on /var/lib/ceph/radosgw/ceph-rgw/keyring: (2) No such file or directory
Dec 2 10:53:19 ceph001 radosgw: 2020-12-02 10:53:19.753 7f77a49fd900 -1 AuthRegistry(0x558033c28138) no keyring found at /var/lib/ceph/radosgw/ceph-rgw/keyring, disabling cephx
Dec 2 10:53:19 ceph001 radosgw: 2020-12-02 10:53:19.754 7f77a49fd900 -1 auth: unable to find a keyring on /var/lib/ceph/radosgw/ceph-rgw/keyring: (2) No such file or directory
Dec 2 10:53:19 ceph001 radosgw: 2020-12-02 10:53:19.754 7f77a49fd900 -1 AuthRegistry(0x7fffa3d30168) no keyring found at /var/lib/ceph/radosgw/ceph-rgw/keyring, disabling cephx
Dec 2 10:53:19 ceph001 radosgw: failed to fetch mon config (--no-mon-config to skip)
發現我三個節點：
/var/lib/ceph/radosgw/ 該目錄許可權都是root.
[root@ceph001 lib]# cd ceph/
[root@ceph001 ceph]# ll
total 0
drwxr-x--- 2 ceph ceph 26 Dec 1 14:16 bootstrap-mds
drwxr-x--- 2 ceph ceph 26 Nov 30 17:46 bootstrap-mgr
drwxr-x--- 2 ceph ceph 26 Nov 30 17:38 bootstrap-osd
drwxr-x--- 2 ceph ceph 6 Nov 24 03:33 bootstrap-rbd
drwxr-x--- 2 ceph ceph 6 Nov 24 03:33 bootstrap-rbd-mirror
drwxr-x--- 2 ceph ceph 26 Dec 2 10:23 bootstrap-rgw
drwxr-x--- 3 ceph ceph 20 Dec 2 10:54 crash
drwxr-x--- 3 ceph ceph 26 Dec 1 14:16 mds
drwxr-x--- 3 ceph ceph 26 Nov 30 17:46 mgr
drwxr-x--- 3 ceph ceph 26 Nov 30 17:16 mon
drwxr-x--- 3 ceph ceph 20 Nov 30 17:38 osd
drwxr-xr-x 3 root root 30 Dec 2 10:23 radosgw
drwxr-x--- 2 ceph ceph 6 Nov 30 17:16 tmp
[root@ceph001 ceph]# cd radosgw/
[root@ceph001 radosgw]# ll
total 0
drwxr-xr-x 2 root root 48 Dec 2 10:23 ceph-rgw.ceph001
[root@ceph001 radosgw]# cd ceph-rgw.ceph001/
[root@ceph001 ceph-rgw.ceph001]# ll
total 4
-rw-r--r-- 1 root root 0 Dec 2 10:47 done
-rw-r--r-- 1 root root 69 Dec 2 10:47 keyring
-rw-r--r-- 1 root root 0 Dec 2 10:47 systemd
[root@ceph001 ceph-rgw.ceph001]# pwd
/var/lib/ceph/radosgw/ceph-rgw.ceph001
修改許可權三臺節點上都修改許可權：
[root@ceph003 ~]# chown cephadmin:cephadmin /etc/ceph/ceph.conf
[root@ceph003 ~]# chown -R ceph:ceph /var/lib/ceph/radosgw/
[root@ceph003 ~]#
再次手工啟動
[cephadmin@ceph001 cephcluster]$ sudo systemctl restart [email protected]
[cephadmin@ceph001 cephcluster]$ sudo systemctl status [email protected]
報另外一個錯
Dec 2 11:08:15 ceph001 radosgw: 12: (__libc_start_main()+0xf5) [0x7ff5f40a03d5]
Dec 2 11:08:15 ceph001 radosgw: 13: (()+0x2bed71) [0x557668cadd71]
Dec 2 11:08:15 ceph001 radosgw: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Dec 2 11:08:15 ceph001 radosgw: -3> 2020-12-02 11:08:15.913 7ff601a8c900 -1 ERROR: failed to initialize watch: (34) Numerical result out of range
Dec 2 11:08:15 ceph001 radosgw: 0> 2020-12-02 11:08:15.917 7ff601a8c900 -1 *** Caught signal (Segmentation fault) **
Dec 2 11:08:15 ceph001 radosgw: in thread 7ff601a8c900 thread_name:radosgw
Dec 2 11:08:15 ceph001 radosgw: ceph version 14.2.15 (afdd217ae5fb1ed3f60e16bd62357ca58cc650e5) nautilus (stable)