ceph-node1 192.168.0.232 這個節點同時用作ceph-deploy
ceph-node2 192.168.0.250
ceph-node3 192.168.0.220
操作系統： centos7.4
內核：3.10.0-693.21.1.el7.x86_64
關閉防火墻，selinux

安裝ceph-deploy：

cat >/etc/yum.repos.d/ceph.repo
[ceph-noarch]
name=Ceph noarch packages
baseurl=https://download.ceph.com/rpm/el7/noarch
enabled=1
gpgcheck=1
type=rpm-md
gpgkey=https://download.ceph.com/keys/release.asc
 

更新操作系統並安裝ceph-deploy，其實兩臺也需要update，不需要安裝ceph-deploy

sudo yum update
sudo yum install ceph-deploy

在所有節點創建ceph用戶

sudo useradd -d /home/ceph-node -m ceph-node
sudo passwd ceph-node
echo "ceph-node ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-node
sudo chmod 0440 /etc/sudoers.d/ceph-node

在ceph-deploy節點創建密鑰並分發到各節點

ssh-keygen
ssh-copy-id ceph-node@ceph-node2
ssh-copy-id ceph-node@ceph-node1

修改ssh配置文件默認ssh連接是用ceph-node用戶

Host ceph-node1
   Hostname ceph-node1
   User ceph-node
Host ceph-node2
   Hostname ceph-node2
   User ceph-node
Host ceph-node3
   Hostname ceph-node3
   User ceph-node

每個節點設置時間同步

yum -y install ntpdate
crontab -e
*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com 
 

2.安裝ceph

切換到ceph-deploy節點

mkdir /root/my-cluster
cd /root/my-cluster

如果安裝過程有錯誤或者失敗可以用以下命令從頭再來

ceph-deploy purge {ceph-node} [{ceph-node}]
ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys
rm ceph.*

創建集群

ceph-deploy new ceph-node1 ceph-node2 ceph-node3
vim ceph.conf #加入公共網絡網段
public network = 192.168.0.0/24

安裝ceph

ceph-deploy install --release luminous ceph-node1 ceph-node2 ceph-node3

部署mon生成密鑰文件

ceph-deploy mon create-initial

發送配置文件

ceph-deploy admin ceph-node1 ceph-node2 ceph-node3

創建mgr節點，mgr需要高可用所以創建三個節點

ceph-deploy mgr create ceph-node1
ceph-deploy mgr create ceph-node2
ceph-deploy mgr create ceph-node3

創建osd

ceph-deploy osd create --data /dev/sdb ceph-node1
ceph-deploy osd create --data /dev/sdc ceph-node1
ceph-deploy osd create --data /dev/sdd ceph-node1
ceph-deploy osd create --data /dev/sdb ceph-node2
ceph-deploy osd create --data /dev/sdc ceph-node2
ceph-deploy osd create --data /dev/sdd ceph-node2
ceph-deploy osd create --data /dev/sdb ceph-node3
ceph-deploy osd create --data /dev/sdc ceph-node3
ceph-deploy osd create --data /dev/sdd ceph-node3

使用ceph -s查看集群是否健康，osd是否加入節點

添加額外mon節點，mon節點也需要高可用

ceph-deploy mon add ceph-node2 ceph-node3

至此ceph集群安裝完成

3.安裝常見錯誤

• 錯誤：

  [ceph_deploy][ERROR ] RuntimeError: connecting to host: n1 resulted in errors: IOError cannot send (already closed?)
  解決方法：
  編輯visudo註釋下面的
  Defaults    requiretty

• 錯誤：

  [n2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
  解決方法：
  在ceph.conf添加
  public network = 192.168.122.0/24

• 錯誤：

  ceph-deploy安裝的時候到yum安裝那報錯
  解決：可能是源有問題執行yum -y remove centos-release-ceph

centos7 安裝ceph12.2.5