一：實驗環境

三臺CentOS 6.7主機，IP地址

• 172.16.10.50 master
• 172.16.10.51 slave1
• 172.16.10.52 slave2

JDK版本：1.8.0
CDH版本：5.8.3
Kylin版本：1.5.4.1

cloudera-manager、parcels下載地址：

http://archive.cloudera.com/cm5/cm/5/
http://archive.cloudera.com/cdh5/parcels/

二：安裝CDH

系統環境配置：

• 修改主機名
• 配置hosts
• master ssh免密碼登入到slave
• 安裝jdk
• 安裝mysql

安裝Cloudera Manager Server(master)

下載cloudera-manager-el6-cm5.3.2_x86_64.tar.gz到/opt目錄

[[email protected] opt]# wget http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.8.3_x86_64.tar.gz
[[email protected] opt]# tar zxvf cloudera-manager-el6-cm5.8.3_x86_64.tar.gz
 

下載parcel檔案到parcel-repo中

[[email protected] parcel-repo]# wget http://archive.cloudera.com/cdh5/parcels/5.8.3/CDH-5.8.3-1.cdh5.8.3.p0.2-el6.parcel
[[email protected] parcel-repo]# wget http://archive.cloudera.com/cdh5/parcels/5.8.3/CDH-5.8.3-1.cdh5.8.3.p0.2-el6.parcel.sha1
[[email protected] parcel-repo]# wget http://archive.cloudera.com/cdh5/parcels/5.8.3/manifest.json
[
 
[email protected] parcel-repo]# mv CDH-5.8.3-1.cdh5.8.3.p0.2-el6.parcel.sha1 CDH-5.8.3-1.cdh5.8.3.p0.2-el6.parcel.sha

建立cloudera-scm系統使用者

[[email protected] opt]# useradd --system --home=/opt/cm-5.8.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

配置Cloudera Manager Server資料庫

配置前先將mysql的jdbc庫：mysql-connector-java-5.1.35-bin.jar拷貝到/opt/cm-5.8.3/share/cmf/lib/，再授權mysql賬號,mysql-connector-java.jar下載地址

http://dev.mysql.com/downloads/connector/j/

mysql> grant all privileges on *.* to 'scm'@'%' IDENTIFIED BY 'scm';
mysql> grant all privileges on *.* to [email protected] identified by 'root';
mysql> flush privileges;

[[email protected] opt]# /opt/cm-5.8.3/share/cmf/schema/scm_prepare_database.sh mysql -h 172.16.11.50 -P 3306 -uroot -proot --scm-host master scm scm scm

安裝Cloudera Manager Agent(slave)

1. 在master上修改/opt/ cm-5.8.3/etc/cloudera-scm-agent/config.ini中的server host 為主節點主機名：mater
2. 用scp將目錄同步到agent上

[[email protected] opt]# scp -r /opt/cm-5.8.3 slave1:/opt
[[email protected] opt]# scp -r /opt/cm-5.8.3 slave2:/opt

1. 在slave中建立cloudera-scm系統使用者

[[email protected] opt]# useradd --system --home=/opt/cm-5.8.3/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

在master啟動Cloudera server

[[email protected] opt]# /opt/cm-5.8.3/etc/init.d/cloudera-scm-server start

在slave啟動Cloudera agent

[[email protected] opt]# /opt/cm-5.8.3/etc/init.d/cloudera-scm-agent start

登入web管理介面

http://masterip:7180 預設賬號密碼:admin/admin

遇到的問題：

1. CDH5.8中hive metastore使用mysql作為元資料儲存，但是CDH自動生成配置檔案中沒有生成mysql連線相關引數，手動新增配置：

[[email protected] ~]# vim /etc/hive/conf/hive-site.xml 
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://master/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>root</value>
</property>
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>root</value>
</property>

1. CDH5.8在web管理介面無法啟動hive metastore，需命令列手動啟動

[[email protected] ~]# hive --service metastore &

三：安裝kylin

下載kylin：

下載地址：http://kylin.apache.org/cn/download/

本次實驗只需下載for CDH版即可

[[email protected] opt]# wget http://www.apache.org/dyn/closer.cgi/kylin/apache-kylin-1.5.4.1/apache-kylin-1.5.4.1-cdh5.7-bin.tar.gz

安裝

解壓到/opt目錄下

[[email protected] opt]# ll
drwxr-xr-x  10 hdfs hdfs 4096 Nov 18 10:11 apache-kylin-1.5.4.1-cdh5.7-bin
drwxr-xr-x   6 1106 4001 4096 Nov 16 15:13 cloudera
drwxr-xr-x   9 1106 4001 4096 Oct 20 02:11 cm-5.8.3

新增環境變數

[[email protected] opt]# vim /etc/profile
export JAVA_HOME=/usr/local/java
export KYLIN_HOME=/opt/apache-kylin-1.5.4.1-cdh5.7-bin
export HBASE_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hbase
export HADOOP_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hive
export HADOOP_CMD=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hadoop/bin/hadoop
export HCAT_HOME=/opt/cloudera/parcels/CDH/lib/hive-hcatalog
export PATH=$PATH:$JAVA_HOME/bin:/usr/local/python2.7/bin

編輯kylin配置檔案：

[[email protected] ~]# vim /opt/apache-kylin-1.5.4.1-cdh5.7-bin/conf/kylin.properties
kylin.job.jar=/opt/apache-kylin-1.5.4-cdh5.7-bin/lib/kylin-job-1.5.4.jar
kylin.coprocessor.local.jar= /opt/apache-kylin-1.5.4-cdh5.7-bin/lib/kylin-coprocessor-1.5.4.jar
kylin.job.yarn.app.rest.check.status.url=http://slave2:8088/ws/v1/cluster/apps/${job_id}?anonymous=true

匯入kylin測試資料：

[[email protected] ~]# cd /opt/apache-kylin-1.5.4.1-cdh5.7-bin/bin/
[[email protected] bin]# sh sample.sh

啟動kylin：

[[email protected] bin]# ./kylin.sh start

登入後臺：http://xxxxx:7070/kylin
賬號密碼：admin/KYLIN