1.搭建zookeeper

• 2.解壓，linux命令：
  sudo tar -zxvf zookeeper-3.4.3.tar.gz
• 3.配置環境變數 ，linux命令：vi ~/.bashrc ,新增ZOOKEEPER_HOME

export JAVA_HOME=/usr/java/jdk1.8.0_60

export ZOOKEEPER_HOME=/opt/software/zookeeper-3.4.6

export STORM_HOMW=/opt/software/apache-storm-0.9.5
export PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME
 
/bin:$PATH

• 4.conf設定，dataDir、clientPort、最下面的server注意下.
  記得手動建立Dir的目錄，不然會啟動失敗。

[[email protected] conf]# cat zoo.cfg 
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
 

# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/var/zookeeper/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
 

# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

server.1=192.168.3.160:2888:3888
server.2=192.168.3.161:2888:3888
server.3=192.168.3.162:2888:3888

• 5.常用命令
  zookeeper-3.4.6/bin/zkServer.sh {start|start-foreground|stop|restart|status|upgrade|print-cmd}
  先start之後，status檢視狀態

[[email protected] zookeeper-3.4.6]# bin/zkServer.sh start
JMX enabled by default
Using config: /opt/software/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[[email protected] software]# zookeeper-3.4.6/bin/zkServer.sh status
JMX enabled by default
Using config: /opt/software/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: standalone

[[email protected] zookeeper-3.4.6]# bin/zkServer.sh stop
JMX enabled by default
Using config: /opt/software/zookeeper-3.4.6/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED

2.Kafka搭建

• 2.解壓，linux命令：
  sudo tar -zxvf kafka_2.11-0.8.2.2.tgz
• 3.conf設定，注意host.name=192.168.3.160，zookeeper.connect=192.168.3.163:2181

...
############################# Server Basics #############################

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0

############################# Socket Server Settings #############################

# The port the socket server listens on
port=9092

# Hostname the broker will bind to. If not set, the server will bind to all interfaces
host.name=192.168.3.160

...
############################# Zookeeper #############################

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=192.168.3.163:2181

# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000

• 4.啟動kafka server: nohup bin/kafka-server-start.sh config/server.properties >/dev/null 2>&1 &
  此命令忽略hangup可以在遠端連線關閉後繼續執行。

kafka叢集的多個broke連線到同一個zookeeper，生產者往一個broke傳送訊息，消費者從zookeeper獲得訂閱。

3.Storm搭建

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

########### These MUST be filled in for a storm configuration
# storm.zookeeper.servers:
#     - "server1"
#     - "server2"
storm.zookeeper.servers:
  - "192.168.3.161"

storm.zookeeper.port: 2181

#
# nimbus.host: "nimbus"
#
nimbus.host: "192.168.3.160"
nimbus.childopts: -Xmx1024m -Djava.net.preferIPv4Stack=true

ui.childopts: -Xmx768m -Djava.net.preferIPv4Stack=true

ui.host: 0.0.0.0
ui.port: 8080

supervisor.childopts: -Djava.net.preferIPv4Stack=true
worker.childopts: -Xmx768m -Dfile.encoding=utf-8 -Djava.net.preferIPv4Stack=true

supervisor.slots.ports:
  - 6700
  - 6701
  - 6702
  - 6703

storm.local.dir: /data/cluster/storm

storm.log.dir: /data/cluster/storm/logs

logviewer.port: 8000
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"
drpc.servers:
  - "192.168.3.160"

## Metrics Consumers
# topology.metrics.consumer.register:
#   - class: "backtype.storm.metric.LoggingMetricsConsumer"
#     parallelism.hint: 1
#   - class: "org.mycompany.MyMetricsConsumer"
#     parallelism.hint: 1
#     argument:
#       - endpoint: "metrics-collector.mycompany.org"

supervisor.slots.ports: 對於每個Supervisor工作節點，需要配置該工作節點可以執行的worker數量。每個worker佔用一個單獨的埠用於接收訊息，該配置選項即用於定義哪些埠是可被worker使用的。預設情況下，每個節點上可執行4個workers，分別在6700、6701、6702和6703埠.

• 4.常用命令

# bin/storm nimbus < /dev/null 2<&1 &
# bin/storm supervisor < /dev/null 2<&1 &
# bin/storm ui < /dev/null 2<&1 &
#
#bin/storm jar storm-demo-1.0.jar io.sterm.demo.topology.WordCountTopology
#bin/storm kill word-count

• 啟動nimbus後臺執行：bin/storm nimbus < /dev/null 2<&1 &
• 啟動supervisor後臺執行：bin/storm supervisor < /dev/null 2<&1 &
• 啟動ui後臺執行：bin/storm ui < /dev/null 2<&1 &