kingbaseES R3叢集防火牆配置案例

案例環境：

作業系統：
[root@node1 ~]# cat /etc/centos-release
CentOS Linux release 7.2.1511 (Core) 

資料庫：
test=# select version();
                                                         version                                                         
-----------------------------------------------------------------------------------------
 Kingbase V008R003C002B0270 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46), 64-bit
(1 row)

 

案例說明：

   1）對於叢集節點之間需要在kingbasecluster、watchdog、kingbase資料庫服務進行通訊。
   2）對於防火牆在可以關閉的情況下，可以在部署叢集之前關閉防火牆。
   3）不能關閉防火牆的注意在防火牆規則中配置叢集服務通訊埠。

在部署叢集是需要防火牆配置的通訊埠如下：

1、系統防火牆啟動

[root@node1 ~]# firewall-cmd --list-all
FirewallD is not running

[root@node1 ~]# systemctl start firewalld
You have mail in /var/spool/mail/root

[root@node1 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: active (running) since Mon 2021-03-01 12:04:30 CST; 8s ago
 Main PID: 2899 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─2899 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Mar 01 12:04:29 node1 systemd[1]: Starting firewalld - dynamic firewall daemon...
Mar 01 12:04:30 node1 systemd[1]: Started firewalld - dynamic firewall daemon.

 

2、檢視防火牆規則

[root@node1 ~]# firewall-cmd --list-all
public (default, active)
  interfaces: enp0s3 enp0s8
  sources: 
  services: dhcpv6-client ssh
  ports: 
  masquerade: no
  forward-ports: 
  icmp-blocks: 
  rich rules: 

3、配置埠轉發規則

[root@node1 ~]# firewall-cmd --permanent --zone=public --add-port=9999/tcp --add-port=9000/tcp --add-port=54321/tcp --add-port=9898/tcp --add-port=9694/udp
success

[root@node1 ~]# firewall-cmd --list-all
public (default, active)
  interfaces: enp0s3 enp0s8
  sources: 
  services: dhcpv6-client ssh
  ports: 
  masquerade: no
  forward-ports: 
  icmp-blocks: 
  rich rules: 


重新整理防火牆規則：

  [root@node1 ~]# firewall-cmd --reload
success


檢視防火牆規則：
[root@node1 ~]# firewall-cmd --list-all
public (default, active)
  interfaces: enp0s3 enp0s8
  sources: 
  services: dhcpv6-client ssh
  ports: 9999/tcp 9000/tcp 54321/tcp 9898/tcp 9694/udp
  masquerade: no
  forward-ports: 
  icmp-blocks: 
  rich rules: 

 

4、啟動叢集服務

[kingbase@node1 bin]$ ./kingbase_monitor.sh start
-----------------------------------------------------------------------
2021-03-01 12:14:14 KingbaseES automation beging...
ping trust ip 192.168.7.1 success ping times :[3], success times:[2]
ping trust ip 192.168.7.1 success ping times :[3], success times:[2]
start crontab kingbase position : [1]
Redirecting to /bin/systemctl restart  crond.service
start crontab kingbase position : [2]
Redirecting to /bin/systemctl restart  crond.service
ADD VIP NOW AT 2021-03-01 12:13:45 ON enp0s3
execute: [/sbin/ip addr add 192.168.7.245/24 dev enp0s3 label enp0s3:2]
execute: /home/kingbase/cluster/kha/db/bin/arping -U 192.168.7.245 -I enp0s3 -w 1
ARPING 192.168.7.245 from 192.168.7.245 enp0s3
Sent 1 probes (1 broadcast(s))
Received 0 response(s)
ping vip 192.168.7.245 success ping times :[3], success times:[2]
ping vip 192.168.7.245 success ping times :[3], success times:[3]
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
ksql: could not connect to server: No route to host
        Is the server running on host "192.168.7.249" and accepting
        TCP/IP connections on port 54322?
There are no 1 standbys in sys_stat_replication, please check all the standby servers replica from primary

如上所示，叢集啟動故障，因為資料庫服務監聽埠使用了非預設的54321，而是54322，在防火牆規則中沒有新增54322/tcp的埠轉發，導致無法和另外節點的kingbaseES資料庫服務通訊，啟動失敗。

5、新增新的埠轉發規則

[root@node1 ~]# firewall-cmd --permanent --zone=public --add-port=54322/tcp 
success
[root@node1 ~]# firewall-cmd --reload
success
[root@node1 ~]# firewall-cmd --list-all
public (default, active)
  interfaces: enp0s3 enp0s8
  sources: 
  services: dhcpv6-client ssh
  ports: 54322/tcp 9694/udp 54321/tcp 9000/tcp 9898/tcp 9999/tcp
  masquerade: no
  forward-ports: 
  icmp-blocks: 
  rich rules:

重新啟動叢集：

[kingbase@node1 bin]$ ./kingbase_monitor.sh restart
-----------------------------------------------------------------------
2021-03-01 12:17:27 KingbaseES automation beging...
2021-03-01 12:17:27 stop kingbasecluster [192.168.7.248] ...
DEL VIP NOW AT 2021-03-01 12:17:28 ON enp0s3
No VIP on my dev, nothing to do.
2021-03-01 12:17:28 Done...
2021-03-01 12:17:28 stop kingbasecluster [192.168.7.249] ...
DEL VIP NOW AT 2021-03-01 12:16:41 ON enp0s3
No VIP on my dev, nothing to do.
2021-03-01 12:17:29 Done...
2021-03-01 12:17:29 stop kingbase [192.168.7.248] ...
set /home/kingbase/cluster/kha/db/data down now...
2021-03-01 12:17:32 Done...
2021-03-01 12:17:33 Del kingbase VIP [192.168.7.245/24] ...
DEL VIP NOW AT 2021-03-01 12:17:34 ON enp0s3
No VIP on my dev, nothing to do.
2021-03-01 12:17:34 Done...
2021-03-01 12:17:34 stop kingbase [192.168.7.249] ...
set /home/kingbase/cluster/kha/db/data down now...
2021-03-01 12:17:39 Done...
2021-03-01 12:17:40 Del kingbase VIP [192.168.7.245/24] ...
DEL VIP NOW AT 2021-03-01 12:16:53 ON enp0s3
execute: [/sbin/ip addr del 192.168.7.245/24 dev enp0s3]
Oprate del ip cmd end.
2021-03-01 12:17:40 Done...
......................
all stop..
ping trust ip 192.168.7.1 success ping times :[3], success times:[2]
ping trust ip 192.168.7.1 success ping times :[3], success times:[2]
start crontab kingbase position : [1]
Redirecting to /bin/systemctl restart  crond.service
start crontab kingbase position : [2]
Redirecting to /bin/systemctl restart  crond.service
ADD VIP NOW AT 2021-03-01 12:17:08 ON enp0s3
execute: [/sbin/ip addr add 192.168.7.245/24 dev enp0s3 label enp0s3:2]
execute: /home/kingbase/cluster/kha/db/bin/arping -U 192.168.7.245 -I enp0s3 -w 1
ARPING 192.168.7.245 from 192.168.7.245 enp0s3
Sent 1 probes (1 broadcast(s))
Received 0 response(s)
ping vip 192.168.7.245 success ping times :[3], success times:[2]
ping vip 192.168.7.245 success ping times :[3], success times:[2]
now,there is a synchronous standby.
wait kingbase recovery 5 sec...
start crontab kingbasecluster line number: [2]
Redirecting to /bin/systemctl restart  crond.service
start crontab kingbasecluster line number: [3]
Redirecting to /bin/systemctl restart  crond.service
......................
all started..
...
now we check again
=======================================================================
|             ip |                       program|              [status] 
[  192.168.7.248]|             [kingbasecluster]|              [active]
[  192.168.7.249]|             [kingbasecluster]|              [active]
[  192.168.7.248]|                    [kingbase]|              [active]
[  192.168.7.249]|                    [kingbase]|              [active]
=======================================================================
You have mail in /var/spool/mail/kingbase

如上所示，叢集啟動成功！！！

6、總結

  對於叢集部署，一定要和系統管理員做好溝通，提前配置好防火牆的規則，否則，在部署和叢集執行期間會出現各種故障。