安裝部署kafka和zookeeper叢集(三節點)
阿新 • • 發佈：2020-12-17
目錄
安裝zookeeper和kafka
軟體及版本
| 軟體 | 版本 |
|---|---|
| zookeeper | 3.5.7 |
| kafka | 2.12-2.2.2 |
| jdk | 1.8 |
安裝zookeeper
tar xf apache-zookeeper-3.5.7-bin.tar.gz -C /usr/local/
cd /usr/local/
ln -s apache-zookeeper-3.5.7-bin zookeeper
cd zookeeper/conf/
cp zoo_sample.cfg zoo.cfg
修改配置檔案
dataDir=/data1/zk/data
dataLogDir=/data1/zk/log
server.1=192.168.68.7:2888:3888
server.2=192.168.69.7:2888:3888
server.3=192.168.70.7:2888:3888
[root@kafka_70 conf]# grep -v '^#' zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 clientPort=2181 dataDir=/data1/zk/data dataLogDir=/data1/zk/log server.1=192.168.68.7:2888:3888 server.2=192.168.69.7:2888:3888 server.3=192.168.70.7:2888:3888
建立資料和日誌目錄
mkdir /data1/zk/data
mkdir /data1/zk/log
建立myid檔案, 不同的節點設定不同的數字, 比如測試環境的三節點myid為1, 2, 3
echo 1 > /data1/zk/data/myid
安裝kafka
tar xf kafka_2.12-2.2.2.tgz -C /usr/local/
cd /usr/local/
ln -s kafka_2.12-2.2.2 kafka
mkdir -p /data1/kafka/log
修改配置檔案
broker.id=0 #不同的節點配置不同的brokerID, 以三節點為例, 分別為0, 1, 2 listeners=PLAINTEXT://kafka3:9092 #修改為主機名, 需要在資料傳送端配置好kafka主機名的host解析 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/data1/kafka/log num.partitions=1 num.recovery.threads.per.data.dir=1 offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=192.168.68.7:2181,192.168.69.7:2181,192.168.70.7:2181 #配置zookeeper叢集 zookeeper.connection.timeout.ms=6000 group.initial.rebalance.delay.ms=0
修改/etc/profile檔案, 新增環境變數
export ZK_HOME=/usr/local/zookeeper/
export PATH=$ZK_HOME/bin:$PATH
export KAFKA_HOME=/usr/local/kafka
export PATH=${KAFKA_HOME}/bin:$PATH
啟動服務
zkServer.sh start
zkServer.sh status
kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties