官方安裝文件從前幾個版本開始在 配置網路的時候就由ovs改成了Linuxbridge，原因有很多，這裡不討論。我是按照官方文件進行部署的，自然使用的是Linuxbridge。DVR中文名叫分散式路由，可以在計算節點實現東西向流量的轉發，不用再繞道控制節點（從M版開始網路節點併入了控制節點），這樣就能夠減輕控制節點的負載。有資料顯示DVR只能部署在ovs上，而且我在網路上看到的所有關於DVR部署的文件都是基於ovs的，所有我必須將Linuxbridge換成ovs。

先說一下我的openstack環境狀況：一個控制節點，兩個計算節點，控制節點的第一塊網絡卡IP地址為192.168.1.51，用作管理網，第二塊網絡卡無IP地址，作為provider網路，負責與外網通訊。計算節點1的第一塊網絡卡IP地址為192.168.1.71，連線管理網。計算節點2的第一塊網絡卡IP地址為192.168.1.72，連線管理網。我使用的是self-service網路。系統是Ubuntu16.04，所有安裝步驟都是按照N版官方文件進行的。

如果你已經安裝了Linuxbridge，那麼需要先解除安裝Linuxbridge

# apt purge neutron-Linuxbridge-agent

然後在資料庫中刪除neutron資料庫

mysql> DROP DATABASE neutron;

一、下邊是N版安裝ovs的步驟：

1、在網路節點上

首先進入資料庫

# mysql -u root -p

然後建立neutron資料庫

mysql> CREATE DATABASE neutron;

給資料庫賦權

GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'NEUTRON_DBPASS';
你自己選一個合適的密碼替換NEUTRON_DBPASS
 

注意，為了簡單起見，我的所有的密碼都是123456

使用admin credentials

# . admin-openrc

建立service credentials

# openstack user create --domain default --password-prompt neutron
# openstack role add --project service --user neutron admin
# openstack service create --name neutron --description "OpenStack Networking" network
 

建立Networking service API endpoints

# openstack endpoint create --region RegionOne network public http://controller:9696
# openstack endpoint create --region RegionOne network internal http://controller:9696
# openstack endpoint create --region RegionOne network admin http://controller:9696

安裝元件

# apt install neutron-server neutron-plugin-ml2 neutron-l3-agent neutron-dhcp-agent neutron-metadata-agent neutron-openvswitch-agent

/etc/neutron/neutron.conf配置如下

[DEFAULT]
rpc_backend = rabbit
core_plugin = ml2
service_plugins = router
allow_overlapping_ips = True
transport_url = rabbit://openstack:[email protected]
auth_strategy = keystone
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True
[database]
connection = mysql+pymysql://neutron:[email protected]/neutron
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = 123456
[nova]
auth_url = http://controller:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = nova
password = 123456
[oslo_messaging_rabbit]
rabbit_host = 192.168.1.51
rabbit_userid = openstack
rabbit_password = 123456

/etc/neutron/plugins/ml2/ml2_conf.ini配置如下

[ml2]
type_drivers = flat,vlan,gre,vxlan
tenant_network_types = vxlan
mechanism_drivers = openvswitch,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider
[ml2_type_vxlan]
vni_ranges = 1:1000
[securitygroup]
enable_ipset = True

/etc/nova/nova.conf配置如下

[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
state_path=/var/lib/nova
force_dhcp_release=True
verbose=True
ec2_private_dns_show_ip=True
enabled_apis=osapi_compute,metadata
transport_url = rabbit://openstack:[email protected]
auth_strategy = keystone
my_ip = 192.168.1.51
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
[database]
connection = mysql+pymysql://nova:[email protected]/nova
[api_database]
connection = mysql+pymysql://nova:[email protected]/nova_api
[oslo_concurrency]
lock_path=/var/lock/nova
[libvirt]
use_virtio_for_bridges=True
[wsgi]
api_paste_config=/etc/nova/api-paste.ini
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = 123456
[vnc]
vncserver_listen = $my_ip
vncserver_proxyclient_address = $my_ip
[glance]
api_servers = http://controller:9292
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[neutron]
url = http://controller:9696
auth_url = http://controller:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = 123456
service_metadata_proxy = True
metadata_proxy_shared_secret = 123456

/etc/neutron/plugins/ml2/openvswitch_agent.ini配置如下

[agent]
tunnel_types = vxlan
l2_population = True
prevent_arp_spoofing = True
[ovs]
local_ip = 192.168.1.51
bridge_mappings = provider:br-ex
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

/etc/neutron/l3_agent.ini配置如下

[DEFAULT]
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
external_network_bridge =

/etc/neutron/dhcp_agent.ini配置如下

[DEFAULT]
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = True

/etc/neutron/metadata_agent.ini配置如下

[DEFAULT]
nova_metadata_ip = 192.168.1.51
metadata_proxy_shared_secret = 123456

同步資料庫

# su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron

重啟服務

# service nova-api restart
# service neutron-server restart
# service openvswitch-switch restart
# service neutron-openvswitch-agent restart
# service neutron-dhcp-agent restart
# service neutron-metadata-agent restart
# service neutron-l3-agent restart

新建一個外部網路橋接

# ovs-vsctl add-br br-ex

將外部網路橋接對映到網絡卡，這裡我的第二塊網絡卡是連線外網的

ovs-vsctl add-port br-ex eno2

再次重啟服務

# service nova-api restart
# service neutron-server restart
# service openvswitch-switch restart
# service neutron-openvswitch-agent restart
# service neutron-dhcp-agent restart
# service neutron-metadata-agent restart
# service neutron-l3-agent restart

2、在計算節點上（以計算節點1為例）

在/etc/sysctl.conf中新增

net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1

然後使之生效

# sysctl -p

安裝元件

# apt-get install -y neutron-openvswitch-agent

/etc/neutron/neutron.conf配置如下

[DEFAULT]
rpc_backend = rabbit
transport_url = rabbit://openstack:[email protected]
auth_strategy = keystone
core_plugin = ml2
service_plugins = router
allow_overlapping_ips = True
[keystone_authtoken]
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = neutron
password = 123456
[oslo_messaging_rabbit]
rabbit_host = 10.0.0.11
rabbit_userid = openstack
rabbit_password = 123456

/etc/neutron/plugins/ml2/openvswitch_agent.ini配置如下

[agent]
tunnel_types = vxlan
l2_population = True
prevent_arp_spoofing = True
[ovs]
local_ip = 192.168.1.71
[securitygroup]
enable_security_group = True
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

/etc/nova/nova.conf配置如下

[neutron]
url = http://controller:9696
auth_url = http://controller:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = 123456

完成上述配置之後，重啟計算服務

# service nova-compute restart

然後重啟ovs

# service openvswitch-switch restart
# service neutron-openvswitch-agent restart

至此全部配置完成，在控制節點上檢視網路服務是否正常

# openstack network agent list

如果配置正常，那麼控制節點上的neutron-metadata-agent、neutron-openvswitch-agent、neutron-l3-agent、neutron-dhcp-agent，計算節點上的neutron-openvswitch-agent應該是正常工作的

二、配置DVR

在完成ovs的配置之後，可以進行DVR的配置

1、在控制節點上

在/etc/neutron/neutron.conf中相應模組加入下列內容

[DEFAULT]
l3_ha = True
router_distributed = True

在/etc/neutron/plugins/ml2/ml2_conf.ini中相應模組加入下列內容

[agent]
l2_population = True
enable_distributed_routing = True
arp_responder = True

在/etc/neutron/l3_agent.ini中相應模組加入下列內容

[DEFAULT]
ha_vrrp_auth_password = password
agent_mode = dvr_snat

配置完成後重啟服務

# service nova-api restart
# service openvswitch-switch restart
# service neutron-openvswitch-agent restart
# service neutron-l3-agent restart
# service neutron-dhcp-agent restart
# service neutron-metadata-agent restart

2、在計算節點上（以計算節點1為例）

編輯/etc/sysctl.conf，修改為

net.ipv4.ip_forward=1
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1

然後載入配置

# sysctl -p

在計算節點上安裝相關服務

# apt-get install neutron-l3-agent  neutron-metadata-agent neutron-plugin-ml2

在/etc/neutron/plugins/ml2/ml2_conf.ini中相應模組加入下列內容

[ml2]
mechanism_drivers = openvswitch,l2population
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
[ovs]
local_ip = 192.168.1.71
bridge_mappings = external:br-ex
[agent]
l2_population = True
tunnel_types = vxlan
enable_distributed_routing = True
arp_responder = True

在/etc/neutron/l3_agent.ini中相應模組加入下列內容

[DEFAULT]
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
external_network_bridge =
agent_mode = dvr

在/etc/neutron/metadata_agent.ini中相應模組加入下列內容

[DEFAULT]
auth_uri = http://192.168.1.51:5000
auth_url = http://192.168.1.51:35357
auth_region = RegionOne
auth_plugin = password
project_domain_id = default
user_domain_id = default
project_name = service
username = neutron
password = 123456
nova_metadata_ip = 192.168.1.51
metadata_proxy_shared_secret = 123456

重啟服務

# service nova-compute restart
# service openvswitch-switch restart
# service neutron-openvswitch-agent restart
# service neutron-l3-agent restart
# service neutron-metadata-agent restart

至此DVR配置完畢，大家可以在同一個計算節點的不同網路起兩個例項，通過分散式路由連線它們，用tcpdump監聽分散式路由的埠和計算節點的網絡卡，當兩個例項通訊的時候你會發現分散式路由的埠上監測到了流量，而計算節點的網絡卡上沒有流量產生。

openstack包含的內容非常多，我可能連皮毛都沒摸到，需要學習的還有非常多，希望今後能與各位一同進步。文章中可能存在不足，歡迎大家指正。

參考

https://docs.openstack.org/newton/install-guide-ubuntu/neutron-controller-install.html

https://docs.openstack.org/newton/install-guide-ubuntu/neutron-compute-install.html

https://kairen.gitbooks.io/openstack-ubuntu-newton/content/ubuntu-binary/neutron/

OpenStack