clickhouse資料庫1分配1副本搭建
【需求】
根據需求需要搭建clickhouse資料庫副本操作
【環境介紹】
系統環境：CentOS Linux release 7.6 +ClickHouse Server version 20.10.1 +apache-zookeeper-3.6.3 +java version "1.8.0_311"
【hosts環境】
兩個節點新增hosts資訊,這裡生產建議統一主機名稱識別，例如：cknode1,cknode2等
10.0.0.xx1 productionbackup
10.0.0.xx2 mysql-dev
【java環境】
可以從Oracle官網或者GitHub下載JAVA
https://www.oracle.com/java/technologies/downloads
https://github.com/AdoptOpenJDK/openjdk8-binaries/releases
解壓及軟連線
tar -zxvf jdk-8u311-linux-x64.tar.gz -C /usr/local/
ln -s /usr/local/jdk1.8.0_311 /usr/local/java
設定環境變數
vim /etc/profile
export JAVA_HOME=/usr/local/java
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
source /etc/profile
檢視版本
java -version
java version "1.8.0_311"
Java(TM) SE Runtime Environment (build 1.8.0_311-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.311-b11, mixed mode
【zookeeper環境】
兩個節點安裝，節點一為主節點（一般建議3個節點安裝）
兩節點叢集安裝
選擇安裝的版本，建議使用穩定版
http://archive.apache.org/dist/zookeeper/
兩個節點下載安裝包做軟鏈
wget http://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz -C /usr/local/
ln -s /usr/local/apache-zookeeper-3.6.3-bin /usr/local/zookeeper
兩個節點配置環境變數
vim /etc/profile
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin
source /etc/profile
兩個節點新增配置檔案
cd /usr/local/zookeeper/conf
參考官方
https://clickhouse.tech/docs/en/operations/tips/#zookeeper
兩個節點部署zoo.cfg配置檔案
vim zoo.cfg
tickTime=2000
initLimit=30000
syncLimit=10
maxClientCnxns=2000
maxSessionTimeout=60000000
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
autopurge.snapRetainCount=10
autopurge.purgeInterval=1
preAllocSize=131072
snapCount=3000000
leaderServes=yes
clientPort=2181
quorumListenOnAllIPs=true
這裡需要在zoo.cfg新增叢集資訊,兩個節點分別為
# 主節點為
server.1=10.0.0.xx1:2889:3889
server.2=10.0.0.xx2:2888:3888
# 從節點為
server.1=10.0.0.xx1:2889:3889
server.2=10.0.0.xx2:2888:3888
建立目錄
mkdir -p /data/zookeeper/data
mkdir -p /usr/local/zookeeper/logs
myid設定
# 主節點為
echo "1">/data/zookeeper/data/myid
# 從節點為
echo "2">/data/zookeeper/data/myid
配置zk日誌
預設zk日誌輸出到一個檔案,且不會自動清理,所以,一段時間後zk日誌會非常大
1.zookeeper-env.sh conf目錄下新建zookeeper-env.sh檔案
vim zookeeper-env.sh
#!/usr/bin/env bash
#tip:custom configurationfile，do not amend the zkEnv.sh file
#chang the log dir and output of rolling file
ZOO_LOG_DIR="/usr/local/zookeeper/logs"
ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
新增許可權
chmod 755 zookeeper-env.sh
2.log4j.properties 修改日誌的輸入形式
zookeeper.root.logger=INFO, ROLLINGFILE
#zookeeper.root.logger=INFO, CONSOLE
# Max log file size of 10MB
log4j.appender.ROLLINGFILE.MaxFileSize=128MB
# uncomment the next line to limit number of backup files
log4j.appender.ROLLINGFILE.MaxBackupIndex=10
3.配置執行zk的JVM
conf目錄下新建java.env檔案,修改到sudo chmod 755 java.env許可權,主要用於GC log,RAM等的配置.
vim java.env
#!/usr/bin/env bash
#config the jvm parameter in a reasonable
#note that the shell is source in so that do not need to use export
#set java classpath
#CLASSPATH=""
#set jvm start parameter , also can set JVMFLAGS variable
SERVER_JVMFLAGS="-Xms1024m -Xmx2048m $JVMFLAGS"
新增許可權
chmod 755 java.env
啟動zookeeper服務(所有節點)
zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
狀態檢視
zkServer.sh status
節點一狀態：Mode: leader
節點二狀態：Mode: follower
部署注意事項：
zoo.cfg配置檔案中server這裡使用不同埠，檢視網上說埠一一致，但是卻啟動不了；
需要新增quorumListenOnAllIPs=true參宿，否則兩節點互相訪問不了；
如果設定了iptables，需要開通對應埠，2181/2889:3889/2888:3888；
如果是低版本除錯可以使用zkServer.sh start-foreground日誌輸出觀察資訊；
myid設定建議對應server.x,x對應值配置；
生產建議使用三個節點，故障選舉；
測試叢集資訊
兩節點登陸：zkCli.sh -server 127.0.0.1:2181
登陸後兩節點確認資訊：ls /
節點一建立物件：create /zk_test mydata
節點二查詢物件：get /zk_test ===》存在zk_test的mydata資訊
節點一修改物件：set /zk_test junk
節點二查詢物件：get /zk_test ===》存在zk_test的junk資訊
節點一刪除物件：delete /zk_test
節點二查詢物件：ls / ===》不存在zk_test資訊
說明：節點二能看到對應操作步驟結果即可，follower只能查詢不能設定、修改物件
【clickhouse安裝】
按照官網安裝clickhouse
準備操作
需要驗證當前伺服器的 CPU 是否支援 SSE 4.2 指令集，因為向量化執行需要用到這項特性:
grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
連網安裝
依次執行如下命令:
yum install yum-utils -y
rpm --import https://repo.clickhouse.tech/CLICKHOUSE-KEY.GPG
yum-config-manager --add-repo https://repo.clickhouse.tech/rpm/clickhouse.repo yum install clickhouse-server clickhouse-client -y
如果您沒法連結網際網路，則也可以使用 rpm 的方式來進行離線安裝:需要下載的安裝包有:
clickhouse-server-20.5.4.40-2.noarch.rpm
clickhouse-common-static-20.5.4.40-2.x86_64.rpm
clickhouse-client-20.5.4.40-2.noarch.rpm
下載地址在:
https://repo.yandex.ru/clickhouse/rpm/stable/x86_64/ https://packagecloud.io/Altinity/clickhouse
安裝完成後，修改配置檔案/etc/clickhouse-server/config.xml：
去掉註釋，能所有連線：<listen_host>::</listen_host>
新增配置，關聯叢集配置檔案：<include_from>/etc/clickhouse-server/metrika.xml</include_from>
新增default使用者密碼，一般建議加密方式
/etc/clickhouse-server/users.xml
<password>default</password>
新增metrika配置檔案
/etc/clickhouse-server/metrika.xml
<?xml version="1.0"?>
<yandex>
<!--ck叢集節點-->
<clickhouse_remote_servers>
<ch_cluster_all>
<!--分片1-->
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>productionbackup</host>
<port>9000</port>
<user>default</user>
<password>default</password>
</replica>
<!--複製集1-->
<replica>
<host>mysql-dev</host>
<port>9000</port>
<user>default</user>
<password>default</password>
</replica>
</shard>
</ch_cluster_all>
</clickhouse_remote_servers>
<!--zookeeper相關配置-->
<zookeeper-servers>
<node index="1">
<host>productionbackup</host>
<port>2181</port>
</node>
<node index="2">
<host>mysql-dev</host>
<port>2181</port>
</node>
</zookeeper-servers>
<macros>
<layer>01</layer>
<shard>01</shard>
<!--分片號-->
<replica>10.0.0.xx1</replica>
<!--當前節點IP-->
</macros>
<networks>
<ip>::/0</ip>
</networks>
<!--壓縮相關配置-->
<clickhouse_compression>
<case>
<min_part_size>10000000000</min_part_size>
<min_part_size_ratio>0.01</min_part_size_ratio>
<method>lz4</method>
<!--壓縮演算法lz4壓縮比zstd快, 更佔磁碟-->
</case>
</clickhouse_compression>
</yandex>
節點二
<?xml version="1.0"?>
<yandex>
<!--ck叢集節點-->
<clickhouse_remote_servers>
<ch_cluster_all>
<!--分片1-->
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>productionbackup</host>
<port>9000</port>
<user>default</user>
<password>default</password>
</replica>
<!--複製集1-->
<replica>
<host>mysql-dev</host>
<port>9000</port>
<user>default</user>
<password>default</password>
</replica>
</shard>
</ch_cluster_all>
</clickhouse_remote_servers>
<!--zookeeper相關配置-->
<zookeeper-servers>
<node index="1">
<host>productionbackup</host>
<port>2181</port>
</node>
<node index="2">
<host>mysql-dev</host>
<port>2181</port>
</node>
</zookeeper-servers>
<macros>
<layer>01</layer>
<shard>01</shard>
<!--分片號-->
<replica>10.0.0.xx2</replica>
<!--當前節點IP-->
</macros>
<networks>
<ip>::/0</ip>
</networks>
<!--壓縮相關配置-->
<clickhouse_compression>
<case>
<min_part_size>10000000000</min_part_size>
<min_part_size_ratio>0.01</min_part_size_ratio>
<method>lz4</method>
<!--壓縮演算法lz4壓縮比zstd快, 更佔磁碟-->
</case>
</clickhouse_compression>
</yandex>
兩節點啟動clickhouse
systemctl start clickhouse-server
systemctl status clickhouse-server
檢視日誌沒有錯誤後，測試登入
clickhouse-client -udefault --password default --port 9000
檢視叢集，檢視到部署的叢集資訊，其他為預設資訊，忽略或者刪除中config.xml的remote_servers裡面內容即可
select * from system.clusters;
【測試叢集操作】
建立資料庫：
create database testdb on cluster ch_cluster_all;
建立表，注意ck-node001/2資訊
節點一建立
CREATE TABLE testdb.sbtest_local(EventDate DateTime,CounterID UInt32,UserID UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/tables/01-01/sbtest','ck-node001')PARTITION BY toYYYYMM(EventDate)ORDER BY (CounterID, EventDate, intHash32(UserID))SETTINGS index_granularity = 8192;
節點二建立
CREATE TABLE testdb.sbtest_local(EventDate DateTime,CounterID UInt32,UserID UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/tables/01-01/sbtest','ck-node002')PARTITION BY toYYYYMM(EventDate)ORDER BY (CounterID, EventDate, intHash32(UserID))SETTINGS index_granularity = 8192;
寫入分片1的寫入節點的local表
insert into testdb.sbtest_local VALUES (now(), 10000, 10000)
在分片1的local表可見一條記錄
select * from testdb.sbtest_local
在分片1副本節點local讀取可見一條記錄
select * from testdb.sbtest_local