kafka + zookeeper 叢集

一、服務規劃

系統：CentOS Linux release 7.8.2003 
版本：redis-5.0.10
地址清單：
176.16.2.11
176.16.2.12
176.16.2.13

zookeeper 資訊

配置資訊：
port:2181
server.1=176.16.2.11:2888:3888
server.2=176.16.2.12:2888:3888
server.3=176.16.2.13:2888:3888

配置資訊 （注意 zk 叢集每個節點的myid標識分別為 1，2，3）

zookeeper 叢集標識：
# vim zookeeper
 
/data/myid 
1

zookeeper 配置資訊：
# vim zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/services/zookeeper/data
clientPort=2181
server.1=176.16.2.11:2888:3888
server.2=176.16.2.12:2888:3888
server.3=176.16.2.13:2888:3888

啟動：
cd zookeeper/bin/ && ./zkServer.sh restart

kafka 資訊

配置目錄：kafka/config
資料目錄：
 
/tmp/zookeeper
埠：9092

配置檔案：（server.properties 中 broker.id 標識分別為1，2，3）

# vim config/producer.properties
bootstrap.servers=176.16.2.11:9092,176.16.2.12:9092,176.16.2.13:9092

# vim config/consumer.properties
bootstrap.servers=176.16.2.11:9092,176.16.2.12:9092,176.16.2.13:9092
group.id=test-consumer-group

# cat config
 
/zookeeper.properties
dataDir=/tmp/zookeeper
clientPort=2181
maxClientCnxns=0
admin.enableServer=false

kakfa節點1：server.properties

# cat config/server.properties
broker.id=1
listeners=PLAINTEXT://176.16.2.11:9092
advertised.listeners=PLAINTEXT://176.16.2.11:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/opt/logs/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=176.16.2.11:2181,176.16.2.12:2181,176.16.2.13:2181
zookeeper.connection.timeout.ms=18000

kafka節點2：server.properties

# cat config/server.properties
broker.id=2
listeners=PLAINTEXT://176.16.2.12:9092
advertised.listeners=PLAINTEXT://176.16.2.12:9092
log.dirs=/opt/logs/kafka-logs
zookeeper.connect=176.16.2.11:2181,176.16.2.12:2181,176.16.2.13:2181
zookeeper.connection.timeout.ms=18000

kafka節點3：server.properties

# cat config/server.properties
broker.id=3
listeners=PLAINTEXT://176.16.2.13:9092
advertised.listeners=PLAINTEXT://176.16.2.13:9092
log.dirs=/opt/logs/kafka-logs
zookeeper.connect=176.16.2.11:2181,176.16.2.12:2181,176.16.2.13:2181
zookeeper.connection.timeout.ms=18000

啟動：

nohup kafka/bin/kafka-server-start.sh kafka/config/server.properties &