hue安裝與部署
阿新 • • 發佈:2019-01-04
執行環境
- centOS 6.6
- hadoop 2.4.0
- hive 1.2.0
- spark 1.4.1
- HUE 3.9
介紹:
Hue是一個開源的Apache Hadoop UI系統,最早是由Cloudera Desktop演化而來,由Cloudera貢獻給開源社群,它是基於Python Web框架Django實現的。通過使用Hue我們可以在瀏覽器端的Web控制檯上與Hadoop叢集進行互動來分析處理資料,例如操作HDFS上的資料,執行MapReduce Job等等。
HUE功能:
- 資料庫查詢編輯器,支援 Hive, Impala, MySql, PostGres, Sqlite and Oracle
- 動態查詢儀表盤,支援 Solr
- 支援 Spark 編輯器和儀表盤
- 瀏覽器檢視狀態,支援 YARN, HDFS, Hive table Metastore, HBase, ZooKeeper
- 支援 Pig Editor, Sqoop2, Oozie workflows 編輯器和儀表盤
- 將資料匯入hdfs
在centOS編譯需要的依賴
- ant
- asciidoc
- cyrus-sasl-devel
- cyrus-sasl-gssapi
- gcc
- gcc-c++
- krb5-devel
- libtidy (for unit tests only)
- libxml2-devel
- libxslt-devel
- make
- mvn (from maven package or maven3 tarball)
- mysql
- mysql-devel
- openldap-devel
- python-devel
- sqlite-devel
- openssl-devel (for version 7+)
- gmp-devel
新增maven源
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
安裝依賴
sudo yum install apache-maven ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel
原始碼下載
下載,編譯原始碼(hue 3.9),編譯時間較長
git clone https://github.com/cloudera/hue.git branch-3.9
cd branch-3.9
make apps
編譯完後也可以選擇安裝
make install
在desktop/conf/pseudo-distributed.ini
中修改配置
假設所有服務都執行在testHost下
[desktop]
# hue webServer 地址和埠號
http_host=192.168.47.213
http_port=8001
# Webserver runs as this user
server_user=hue
server_group=hue
# This should be the Hue admin and proxy user
default_user=hue
# This should be the hadoop cluster admin
default_hdfs_superuser=admin
[hadoop]
fs_defaultfs=tdhdfs
webhdfs_url=http://192.168.47.216:50070/webhdfs/v1/
hadoop_conf_dir=/usr/install/hadoop/etc/hadoop/conf
[[yarn_clusters]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=192.168.47.216
# The port where the ResourceManager IPC listens on
resourcemanager_port=23140
# Whether to submit jobs to this cluster
submit_to=True
# URL of the ResourceManager API
resourcemanager_api_url=http://192.168.47.216:23188
# URL of the ProxyServer API
proxy_api_url=http://192.168.47.216:8888
# URL of the HistoryServer API
history_server_api_url=http://192.168.47.216:19888
[beeswax]
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=192.168.47.213
# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000
# Hive configuration directory, where hive-site.xml is located
hive_conf_dir=/usr/install/apache-hive-1.2.0-bin/conf
[zookeeper]
host_ports=zk1:2181,zk2:2181,zk3:2181
啟動
build/env/bin/supervisor
執行介面
一些問題
使用hive 0.13.0時會報
can't fetch result之類的錯誤
是版本原因,升級hive版本為1.2.0問題消失
連線hive error
Could not install table: Error creating table sample_07: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found (code THRIFTTRANSPORT): TTransportException('Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found',).
解決方案:
yum -y install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi