CDH大資料環境安裝
一、安裝虛擬機器vmware
二、下載安裝作業系統
檔名稱 CentOS-6.5-x86_64-bin-DVD1.iso
下載地址 http://www.xitongzhijia.net/linux/201603/69281.html
安裝教程 https://jingyan.baidu.com/article/eae0782787b4c01fec548535.html
- 關閉centOS防火牆
sudo service iptables stop
sudo chkconfig iptables off
sudo chkconfig iptables --list
2、關閉SELinux
vi /etc/selinux/config
修改為SELinux=disabled
檢視SELinux狀態 getenforce
3、設定靜態ip
http://www.cnblogs.com/magialmoon/archive/2013/08/10/3250393.html
http://www.xitongzhijia.net/xtjc/20150115/34968.html
http://www.linuxidc.com/Linux/2013-01/77709.htm
ip addr
首先修改
NETWORKING=yes
HOSTNAME=localhost.localdomain
GATEWAY=192.168.129.2
指定閘道器地址。
然後修改/etc/sysconfig/network-scripts/ifcfg-eth0.
修改 /etc/udev/rules.d/70-persistent-net.rules 檔案
刪除掉 關於 eth0 的資訊。修改 第二條 eth1 的網絡卡的名字為 eth0.
設定完之後,要重啟虛擬機器。
4.修改hosts檔案
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.175.130 master
192.168.175.131 slave1
192.168.175.132 slave2
5、安裝、設定好一個CentOS虛擬機器後,再克隆出兩個CentOS虛擬機器
6、修改主機名
vi /etc/sysconfig/network
7、重啟虛擬機器
centos 6.5切換root
su 使用者名稱
CentOS 6.5 ping不通
將作業系統介面右上角的網路圖示編輯,將system eth0編輯設定為自動連線。
檢視linux軟體安裝資訊
rpm -qa|grep java
三、下載安裝CDH5
CDH5 下載安裝指南 http://www.jianshu.com/p/57179e03795f
安裝報錯
cloudera-manager-agent-5.2.0-1.cm520.p0.60.el6.x86_64 (/cloudera-manager-agent-5.2.0-1.cm520.p0.60.el6.x86_64)
需要:libpython2.6.so.1.0()(64bit)
解決辦法:
http://www.cnblogs.com/lclq/archive/2016/06/27/5620196.html
下載Python-2.6.6.tar.bz2
.tar.bz2
解壓:tar jxvf FileName.tar.bz2
http://blog.csdn.net/gcangle/article/details/50098151
CentOS上安裝軟體錯誤提示:configure: error: no acceptable C compiler found in $PATH
執行 yum -y install gcc
./configure //配置python原始碼
make
make install
遇到問題
Cenos版本不對,導致python版本不對。Cdh 5.2需要python2.6.而我第一次裝的centos7自帶的python是2.7版本的
管理介面
使用者名稱密碼:admin/admin
檢視任務佇列介面(host1)
http://192.168.175.133:8088/cluster
正在安裝 cloudera-manager-agent 包...
BEGIN yum list installed cloudera-manager-agent
Loaded plugins: fastestmirror, refresh-packagekit, security
Installed Packages
cloudera-manager-agent.x86_64 5.2.0-1.cm520.p0.60.el6 @/cloudera-manager-agent-5.2.0-1.cm520.p0.60.el6.x86_64
END (0)
BEGIN echo jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons | grep cloudera-manager-agent
END (0)
BEGIN yum info cloudera-manager-agent
jdk oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons
Loaded plugins: fastestmirror, refresh-packagekit, security
Determining fastest mirrors
Error: Cannot find a valid baseurl for repo: base
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os error was
14: PYCURL ERROR 7 - "Failed to connect to 2607:f8f8:700:12::10: 網路不可達"
END (1)
cloudera-manager-agent must have Version=5.2.0 and Build=60, exiting
closing logging file descriptor
注意:hosts檔案配置和hostname要一一對應
資料庫
service cloudera-scm-agent status
service cloudera-scm-server status
CM管理介面等不了
問題 service httpd沒啟動
service httpd status
service httpd start
問題:
[[email protected] ~]# yum list|grep cloudera
Error: Cannot find a valid baseurl for repo: base
解決辦法:
/etc/sysconfig /network-scripts/ifcfg-eth0
新增 DNS2=114.114.114.114
[[email protected] ~]# cat /etc/resolv.conf
nameserver 192.168.175.2
nameserver 114.114.114.114
could not contact scm server at promote.cache-dns.local:7182, giving up
解決辦法:mv
/usr/bin
/host /usr
/bin/host.bak
安裝telnet
yum -y install telnet
無法檢測到 Agent 發出的檢測訊號
http://www.aboutyun.com/thread-9293-1-1.html
service cloudera-scm-agent restart 然後再重新安裝
問題:主機時鐘不同步。host1 (May 14, 2017 7:46:09 PM CST) 和 host0 (May 14, 2017 7:08:35 PM CST) 之間的最大差異。
解決辦法:安裝ntpq服務
http://www.cnblogs.com/kerrycode/archive/2015/08/20/4744804.html
問題:[[email protected] ~]# service httpd start
正在啟動 httpd:httpd: Could not reliably determine the server's fully qualified domain name
解決辦法:
vi /etc/httpd/conf/httpd.conf
找到#ServerName(注意大小寫和#號)
再下面新增ServerName localhost:80
然後重啟 service httpd restart
問題:如何開機自啟動httpd服務
檢視所有的開機自啟動的項
chkconfig --list
新增服務到自啟動列表
chkconfig –add
開機自啟動服務
chkconfig httpd on
Cloudera Management Service 設定服務
啟動 Service Monitor,Host Monitor,Event Server,Alert Publisher
永久修改系統時間
CentOS要修改硬體時間
[[email protected] ~]# hwclock --set --date '2017-5-15 14:47:57'
[[email protected] ~]# hwclock --hctosys
[[email protected] ~]# clock -w