hue(2):Hue與Hadoop整合
阿新 • • 發佈:2018-12-15
一、實現功能
hue介面可以瀏覽、更改、新增hdfs上的檔案,從而,便於更多人員操作大資料平臺。
二、實現步驟
1.hadoop配置
(1)配置etc/Hadoop/hdfs-site.xml
<property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> <!--下面這個自己本來就有!--> <property> <name>dfs.permissions.enabled</name> <value>false</value> </property>
(2)配置etc/Hadoop/core-site.xml:
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
(3)重啟Hadoop服務
2.修改desktop/conf/hue.ini
[[hdfs_clusters]] # HA support by using HttpFs [[[default]]] # Enter the filesystem uri fs_defaultfs=hdfs://hadoop:8020 # NameNode logical name. ## logical_name= # Use WebHdfs/HttpFs as the communication mechanism. # Domain should be the NameNode or HttpFs host. # Default port is 14000 for HttpFs. webhdfs_url=http://hadoop:50070/webhdfs/v1 # Change this if your HDFS cluster is Kerberos-secured ## security_enabled=false # Default umask for file and directory creation, specified in an octal value. ## umask=022 # Directory of the Hadoop configuration hadoop_conf_dir=/opt/modules/hadoop-2.6.0-cdh5.7.0/etc/hadoop hadoop_hdfs_home=/opt/modules/hadoop-2.6.0-cdh5.7.0/ hadoop_bin =/opt/modules/hadoop-2.6.0-cdh5.7.0/bin/ [[yarn_clusters]] [[[default]]] # Enter the host on which you are running the ResourceManager resourcemanager_host=hadoop # The port where the ResourceManager IPC listens on resourcemanager_port=8032 # Whether to submit jobs to this cluster submit_to=True # Resource Manager logical name (required for HA) ## logical_name= # Change this if your YARN cluster is Kerberos-secured ## security_enabled=false # URL of the ResourceManager API resourcemanager_api_url=http://hadoop:9090 #這個自己由於yarn收到攻擊所以更改8088到9090 # URL of the ProxyServer API proxy_api_url=http://hadoop:9090 #這個自己由於yarn收到攻擊所以更改8088到9090 # URL of the HistoryServer API history_server_api_url=http://hadoop:19888 [filebrowser] # Location on local filesystem where the uploaded archives are temporary stored. archive_upload_tempdir=/tmp
三、測試
1.重啟hdfs
2.重啟yarn
3.開啟historyserver
4.啟動hue
build/env/bin/supervisor