基於kolla部署多節點OpenStack(Ocata版)
阿新 • • 發佈：2019-01-06
在這篇文章(http://blog.csdn.net/dylloveyou/article/details/77148560)中，我們已經部署了單節點的OpenStack環境，這次我們嘗試部署一個多節點的環境(包括1個controller節點，1個compute節點，1個storage節點)，之前的單節點因為已經安裝了kolla-ansible和docker registry，我們把它當做部署節點。
1. 環境準備
新建立3臺虛擬機器，分別作為controller節點，compute節點，storage節點。其中controller節點3張網絡卡，compute、storage節點2張網絡卡。作業系統為centos7.4
1）關閉Selinux
vi /etc/sysconfig/selinux
SELINUX=disabled2）關閉firewalld
systemctl stop firewalld
systemctl disable firewalld3）設定主機名，hosts檔案
vi /etc/hosts
192.168.128.77 control01
192.168.128.85 controller
192.168.128.86 compute
192.168.128.87 storage
4）storage節點加盤
storage節點新加一塊磁碟，作為cinder的lvm後端
加完後重啟系統，可以檢視到新加的磁碟
[[email protected] ~]# fdisk -l
Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sda: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 建立pv、vg
[root@storage ~]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created.
[root@storage ~]# vgcreate cinder-volumes /dev/sdb
Volume group "cinder-volumes" successfully created修改配置
vi /etc/lvm/lvm.conf
修改 devices 下面的
filter = [ "a/sda/", "a/sdb/", "r/.*/"] 重啟lvm服務
systemctl restart lvm2-lvmetad.service5）確認節點配置
controller 3 張網絡卡
compute 2 張網絡卡
storage 2 張網絡卡(其實一張即可，因為不需要tunnel網路)
storage 2 塊硬碟
每個主機的eth0網絡卡作為 API&Management 網路，使用的是“NAT模式”
每個主機的eth1網絡卡作為 Tenant(VM) 網路，是承載VxLAN的底層網路，使用的是“僅主機模式 ”
controller節點也作為網路節點，需要模擬OpenStack的外網，增加網絡卡eth2，這裡也選擇“僅主機模式”
2.安裝docker
1）加入Docker的repo源
# tee /etc/yum.repos.d/docker.repo << 'EOF'
[dockerrepo]
name=Docker Repository
baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF 2）安裝Docker 1.12.6
yum install docker-engine-1.12.6 docker-engine-selinux-1.12.6 -y3）設定Docker
mkdir /etc/systemd/system/docker.service.d
tee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF'
[Service]
MountFlags=shared
EOF4）重啟相關服務
systemctl daemon-reload
systemctl enable docker
systemctl restart docker5）編輯/usr/lib/systemd/system/docker.service檔案
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.128.77:40006）重啟Docker服務
systemctl daemon-reload
systemctl restart docker3.配置免密登入
部署節點和其他節點免密登陸
[[email protected] ~]# ssh-keygen
[[email protected] ~]# ssh-copy-id controller
[[email protected] ~]# ssh-copy-id compute
[[email protected] ~]# ssh-copy-id storage
4.部署OpenStack
1）停止部署節點容器
因為部署節點之前已經部署了All in One的OpenStack，為了降低虛擬機器的資源佔用(我是用筆記本上面的虛擬機器做的實驗)，先停止本節的的OpenStack相關容器。
[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/
[root@control01 tools]# . kolla-ansible stop檢視只剩registry容器在執行
[root@control01 tools]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b18c5accabc2 registry "/bin/registry serve " 3 months ago Up 22 minutes 0.0.0.0:4000->5000/tcp registry2）修改multinode檔案
[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/
[root@control01 kolla-ansible-4.0.3.dev36]# cp ansible/inventory/* /home/
[root@control01 ~]# cd /home
[root@control01 home]# ls
all-in-one multinode
[root@control01 home]# cp multinode mymultinode[[email protected] home]# vi mymultinode
[control]
controller
[network]
controller
[compute]
compute
[monitoring]
controller
[storage]
storage
......裡面的內容很好理解，表示一個控制節點 controller，網路節點也安裝到控制節點，一個計算節點 compute ，一個儲存節點 storage，後面的部分不用修改
3）修改global和password配置檔案
[root@control01 ~]# cd /etc/kolla/
[root@control01 kolla]# vi globals.ymlkolla_internal_vip_address: "192.168.128.85" 外部訪問地址，如果是非HA環境，是控制節點IP
docker_registry: "192.168.128.77:4000" 映象庫，用部署節點的映象庫
docker_namespace: "99cloud"
network_interface: "eth0" 網路介面，預設就這一個介面，這裡我們以eth0作為API網路
api_interface: "{{ network_interface }}" 用的是eth0
tunnel_interface: "eth1" eth1作為vxlan的承載網路
neutron_external_interface: "eth2" eth2作為外部網路
enable_cinder: "yes" 啟用cinder
enable_cinder_backend_lvm: "yes" cinder後端用lvm
cinder_volume_group: "cinder-volumes"
enable_haproxy: "no" 不啟用haproxy[root@control01 kolla]# vi passwords.yml
keystone_admin_password: admin admin使用者登入密碼4）部署前檢查
[root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/
[root@control01 ~]# . kolla-ansible prechecks -i /home/mymultinode報錯 ImportError: No module named docker
TASK [prechecks : Checking docker-py version] **********************************
fatal: [compute]: FAILED! => {"changed": false, "cmd": ["/usr/bin/python", "-c", "import docker; print docker.__version__"], "delta": "0:00:00.012605", "end": "2017-12-02 20:57:39.611343", "failed": true, "failed_when_result": true, "rc": 1, "start": "2017-12-02 20:57:39.598738", "stderr": "Traceback (most recent call last):\n File \"<string>\", line 1, in <module>\nImportError: No module named docker", "stdout": "", "stdout_lines": [], "warnings": []}
......解決：在各個節點安裝 python-docker-py
yum install python-docker-py再次執行檢查，沒有問題，開始部署
5）部署
[root@control01 tools]# . kolla-ansible deploy -i /home/mymultinode開始部署，等待大約20分鐘，部署完成，一般不會有什麼問題，如果有問題，則需要根據報錯及容器日誌查詢原因
登入系統，檢視各個服務正常，表明部署成功。
可以通過docker ps命令檢視各個節點執行的容器
[[email protected] ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a344f712eb4f 192.168.128.77:4000/99cloud/centos-source-horizon:4.0.2.1 "kolla_start" 43 minutes ago Up 24 minutes horizon
7089aea91586 192.168.128.77:4000/99cloud/centos-source-heat-engine:4.0.2.1 "kolla_start" 45 minutes ago Up 45 minutes heat_engine
071f259e23dc 192.168.128.77:4000/99cloud/centos-source-heat-api-cfn:4.0.2.1 "kolla_start" 45 minutes ago Up 45 minutes heat_api_cfn
d75bb72de87a 192.168.128.77:4000/99cloud/centos-source-heat-api:4.0.2.1 "kolla_start" 45 minutes ago Up 45 minutes heat_api
824215d900c8 192.168.128.77:4000/99cloud/centos-source-neutron-metadata-agent:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes neutron_metadata_agent
c5f9262efdc5 192.168.128.77:4000/99cloud/centos-source-neutron-l3-agent:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes neutron_l3_agent
b2b47375d344 192.168.128.77:4000/99cloud/centos-source-neutron-dhcp-agent:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes neutron_dhcp_agent
9a36954ca062 192.168.128.77:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes neutron_openvswitch_agent
0b8bc000ccb3 192.168.128.77:4000/99cloud/centos-source-neutron-server:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes neutron_server
f822f81226d1 192.168.128.77:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 47 minutes ago Up 47 minutes openvswitch_vswitchd
63af3bdb8e2d 192.168.128.77:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 48 minutes ago Up 48 minutes openvswitch_db
b0b1a42a1a17 192.168.128.77:4000/99cloud/centos-source-nova-novncproxy:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes nova_novncproxy
97277a9a1d14 192.168.128.77:4000/99cloud/centos-source-nova-consoleauth:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes nova_consoleauth
1be7b07ba70a 192.168.128.77:4000/99cloud/centos-source-nova-conductor:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes nova_conductor
ae94b6f70ebc 192.168.128.77:4000/99cloud/centos-source-nova-scheduler:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes nova_scheduler
5975962a3c33 192.168.128.77:4000/99cloud/centos-source-nova-api:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes nova_api
0de1f8e33774 192.168.128.77:4000/99cloud/centos-source-nova-placement-api:4.0.2.1 "kolla_start" 50 minutes ago Up 50 minutes placement_api
0efb76129caa 192.168.128.77:4000/99cloud/centos-source-cinder-scheduler:4.0.2.1 "kolla_start" 57 minutes ago Up 57 minutes cinder_scheduler
2d50185586a8 192.168.128.77:4000/99cloud/centos-source-cinder-api:4.0.2.1 "kolla_start" 57 minutes ago Up 57 minutes cinder_api
5c4e87b1db04 192.168.128.77:4000/99cloud/centos-source-glance-registry:4.0.2.1 "kolla_start" 58 minutes ago Up 58 minutes glance_registry
635bc1f305a9 192.168.128.77:4000/99cloud/centos-source-glance-api:4.0.2.1 "kolla_start" 58 minutes ago Up 58 minutes glance_api
de6ec58f6d6e 192.168.128.77:4000/99cloud/centos-source-keystone:4.0.2.1 "kolla_start" About an hour ago Up About an hour keystone
f881fe857541 192.168.128.77:4000/99cloud/centos-source-rabbitmq:4.0.2.1 "kolla_start" About an hour ago Up About an hour rabbitmq
ef6941e59581 192.168.128.77:4000/99cloud/centos-source-mariadb:4.0.2.1 "kolla_start" About an hour ago Up About an hour mariadb
022b6935fc16 192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" About an hour ago Up About an hour cron
acdc83df5d52 192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" About an hour ago Up About an hour kolla_toolbox
6f0a2b4fc44e 192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" About an hour ago Up About an hour fluentd
2b2f15895c0a 192.168.128.77:4000/99cloud/centos-source-memcached:4.0.2.1 "kolla_start" About an hour ago Up About an hour memcached[[email protected] ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e99bdf6a6513 192.168.128.77:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 48 minutes ago Up 48 minutes neutron_openvswitch_agent
90621685744e 192.168.128.77:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 48 minutes ago Up 48 minutes openvswitch_vswitchd
537a8f2dbf06 192.168.128.77:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 49 minutes ago Up 49 minutes openvswitch_db
63761ee44b3c 192.168.128.77:4000/99cloud/centos-source-nova-compute:4.0.2.1 "kolla_start" 51 minutes ago Up 51 minutes nova_compute
53b1784996c7 192.168.128.77:4000/99cloud/centos-source-nova-libvirt:4.0.2.1 "kolla_start" 51 minutes ago Up 51 minutes nova_libvirt
292c7bcbd170 192.168.128.77:4000/99cloud/centos-source-nova-ssh:4.0.2.1 "kolla_start" 52 minutes ago Up 52 minutes nova_ssh
64d2031a345d 192.168.128.77:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" About an hour ago Up About an hour iscsid
04b2c9bd71e6 192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" About an hour ago Up About an hour cron
c22eede98b72 192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" About an hour ago Up About an hour kolla_toolbox
2478ceaa712a 192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" About an hour ago Up About an hour fluentd[[email protected] ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
91607ec544b1 192.168.128.77:4000/99cloud/centos-source-cinder-backup:4.0.2.1 "kolla_start" 55 minutes ago Up 55 minutes cinder_backup
b40ad28f6f8b 192.168.128.77:4000/99cloud/centos-source-cinder-volume:4.0.2.1 "kolla_start" 55 minutes ago Up 55 minutes cinder_volume
61ca356cbab1 192.168.128.77:4000/99cloud/centos-source-tgtd:4.0.2.1 "kolla_start" About an hour ago Up About an hour tgtd
40e86a921d22 192.168.128.77:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" About an hour ago Up About an hour iscsid
0133da5c2ba9 192.168.128.77:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" About an hour ago Up About an hour cron
3bd6802e44ed 192.168.128.77:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" About an hour ago Up About an hour kolla_toolbox
e0187c061936 192.168.128.77:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" About an hour ago Up About an hour fluentd