1. 程式人生 > >Apache Eagle安裝配置指南

Apache Eagle安裝配置指南

Apache Eagle開源沒多久,網上關於安裝它的文章幾乎沒有,在此我獻上自己的安裝步驟供大家參考。

官網安裝文件地址:http://goeagle.io/docs/deployment-in-production.html

官網的安裝大體步驟都沒問題,只是在配置的時候需要注意幾點。

1.環境依賴 HBase: 0.98 or later Storm: 0.9.3 or later Kafka: 0.8.x or later Zookeeper: 3.4.5 Java: 1.7.x Spark,用於使用者模型訓練 Logstash-1.5.5 2.Kafka配置 啟動kafka前需要先配置kafka中的zookeeper地址
./kafka-server-start.sh ../config/server.properties & 建立eagle需要的topic ./kafka-topics.sh --create --zookeeper slave02.mesos:2181,slave03.mesos:2181,slave04.mesos:2181 --replication-factor 1 --partitions 1 --topic Demo_hdfs_audit_log 3.配置eagle 修改bin/eagle-env.sh 修改conf/eagle-service.conf
# eagle configuration
eagle{
        # eagle web service configuration
        service{
                # storage type: ["hbase","jdbc"]
                # default is "hbase"
                storage-type="hbase"

                # hbase configuration: hbase.zookeeper.quorum
                # default is "localhost"
                hbase-zookeeper-quorum="slave02.mesos,slave03.mesos,slave04.mesos"

                # hbase configuration: hbase.zookeeper.property.clientPort
                # default is 2181
                hbase-zookeeper-property-clientPort=2181

                # hbase configuration: zookeeper.znode.parent
                # default is "/hbase"
                zookeeper-znode-parent="/hbase"
        }
}
新建一個conf/Demo-hdfsAuditLog-application.conf
[email protected] conf]$ cat Demo-hdfsAuditLog-application.conf 
{
  "envContextConfig" : {
    "env" : "storm",
    "mode" : "cluster",
    "topologyName" : "Demo-hdfsAuditLog-topology",
    "stormConfigFile" : "security-auditlog-storm.yaml",
    "parallelismConfig" : {
      "kafkaMsgConsumer" : 1,
      "hdfsAuditLogAlertExecutor*" : 1
    }
  },
  "dataSourceConfig": {
    "topic" : "Demo_hdfs_audit_log",
    "zkConnection" : "slave02.mesos:2181,slave03.mesos:2181,slave04.mesos:2181",
    "zkConnectionTimeoutMS" : 15000,
    "fetchSize" : 1048586,
    "deserializerClass" : "eagle.security.auditlog.HdfsAuditLogKafkaDeserializer",
    "transactionZKServers" : "slave02.mesos,slave03.mesos,slave04.mesos",
    "transactionZKPort" : 2181,
    "transactionZKRoot" : "/consumers",
    "consumerGroupId" : "eagle.hdfsaudit.consumer",
    "transactionStateUpdateMS" : 2000
  },
  "alertExecutorConfigs" : {
     "hdfsAuditLogAlertExecutor" : {
       "parallelism" : 1,
       "partitioner" : "eagle.alert.policy.DefaultPolicyPartitioner",
       "needValidation" : "true"
     }
  },
  "eagleProps" : {
    "site" : "Demo",
    "dataSource": "hdfsAuditLog",
  	"dataJoinPollIntervalSec" : 30,
    "mailHost" : "tendcloud.com",
    "mailSmtpPort":"25",
    "mailDebug" : "true",
    "eagleService": {
      "host": "master.mesos",
      "port": 9099
      "username": "admin",
      "password": "secret"
    }
  },
  "dynamicConfigSource" : {
  	"enabled" : true,
  	"initDelayMillis" : 0,
  	"delayMillis" : 30000
  }
}
注意配置這裡面的eagleService的host

4.啟動 建立hbase表 bin/eagle-service-init.sh 啟動eagle web service bin/eagle-service.sh start 引入metadata bin/eagle-topology-init.sh 5.登陸eagle web http://ip:9099/eagle-service 使用者名稱admin,密碼secret  Storm中的topology
至於新增policy和site等過程請參見官網。 最後的web頁面如下: