1. 程式人生 > >hue(2):Hue與Hadoop整合

hue(2):Hue與Hadoop整合

一、實現功能

hue介面可以瀏覽、更改、新增hdfs上的檔案,從而,便於更多人員操作大資料平臺。

二、實現步驟

1.hadoop配置

(1)配置etc/Hadoop/hdfs-site.xml

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>
<!--下面這個自己本來就有!-->
<property>
  <name>dfs.permissions.enabled</name>
  <value>false</value>
</property>

(2)配置etc/Hadoop/core-site.xml:

<property>
	<name>hadoop.proxyuser.hue.hosts</name>
	<value>*</value>
</property>
<property>
	<name>hadoop.proxyuser.hue.groups</name>
	<value>*</value>
</property>

(3)重啟Hadoop服務

2.修改desktop/conf/hue.ini

 [[hdfs_clusters]]
    # HA support by using HttpFs

    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://hadoop:8020

      # NameNode logical name.
      ## logical_name=

      # Use WebHdfs/HttpFs as the communication mechanism.
      # Domain should be the NameNode or HttpFs host.
      # Default port is 14000 for HttpFs.
      webhdfs_url=http://hadoop:50070/webhdfs/v1

      # Change this if your HDFS cluster is Kerberos-secured
      ## security_enabled=false

      # Default umask for file and directory creation, specified in an octal value.
      ## umask=022

      # Directory of the Hadoop configuration
	  hadoop_conf_dir=/opt/modules/hadoop-2.6.0-cdh5.7.0/etc/hadoop
	  hadoop_hdfs_home=/opt/modules/hadoop-2.6.0-cdh5.7.0/
	  hadoop_bin =/opt/modules/hadoop-2.6.0-cdh5.7.0/bin/
	  
  [[yarn_clusters]]
    [[[default]]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=hadoop

      # The port where the ResourceManager IPC listens on
      resourcemanager_port=8032

      # Whether to submit jobs to this cluster
      submit_to=True

      # Resource Manager logical name (required for HA)
      ## logical_name=

      # Change this if your YARN cluster is Kerberos-secured
      ## security_enabled=false

      # URL of the ResourceManager API
      resourcemanager_api_url=http://hadoop:9090 #這個自己由於yarn收到攻擊所以更改8088到9090

      # URL of the ProxyServer API
      proxy_api_url=http://hadoop:9090	#這個自己由於yarn收到攻擊所以更改8088到9090

      # URL of the HistoryServer API
      history_server_api_url=http://hadoop:19888	

[filebrowser]
  # Location on local filesystem where the uploaded archives are temporary stored.
  archive_upload_tempdir=/tmp

三、測試

1.重啟hdfs
2.重啟yarn
3.開啟historyserver
4.啟動hue

build/env/bin/supervisor