在centos7上使用本地庫方式安裝cdh
阿新 • • 發佈:2018-12-22
一、環境配置
1、前提準備
- 已配置靜態ip
- 準備一個完整的跟當前系統一樣的,系統映象iso檔案:centos下載
- 下載需要的cloudera manager的rpm包:下載連結
- 下載cdh的parcel包:下載連結
- 下載mysql的安裝包:下載連結
2、配置hosts
將所有需要的機器安裝如下格式配置到hosts檔案中:
1.1.1.1 foo-1.example.com foo-1 2.2.2.2 foo-2.example.com foo-2 3.3.3.3 foo-3.example.com foo-3 4.4.4.4 foo-4.example.com foo-4
3、配置免密登入
使用如下命令生成公鑰,並且copy給所有節點,包括自身
ssh-keygen (一直回車即可)
ssh-copy-id <hostname>
注:也可以做成指令碼,請自行網上查詢。
4、關閉防火牆
systemctl status firewalld.service
systemctl stop firewalld.service
systemctl disable firewalld.service
5、設定交換分割槽
sysctl vm.swappiness=0
echo "vm.swappiness=0" > /etc/sysctl.conf
6、關閉透明大頁
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
chmod +x /etc/rc.d/rc.local
echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" > /etc/rc.d/rc.local
echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
二、安裝本地yum庫
1、掛載系統映象
- 注意磁碟大小
mkdir /mnt/cdrom
mount -o loop os/CentOS-7-x86_64-Everything-1804.iso /mnt/cdrom/
2、配置repo
- 將本地的yum配置檔案備份
cd /etc/yum.repo.d/
mkdir repo_bak
mv Centos-* repo_bak/
- 新建os.repo、cm.repo、mysql.repo檔案,新增一下內容
[os]
name=os
baseurl=file:///mnt/cdrom
gpgcheck=0
[cm]
name=cm
baseurl=http://<hostname>/cm/cdh
gpgcheck=0
[mysql]
name=mysql
baseurl=http://<hostname>/mysql
gpgcheck=0
3、安裝httpd
yum install httpd
4、安裝createrepo
yum install createrepo
5、建立目錄
cd /var/www/html
mkdir -p cm/cdh
mkdir -p cm/parcel
mkdir mysql
mkdir os
6、拷貝系統依賴和安裝包
cp -rf /mnt/cdrom/* os/
cp /root/cdh/* cm/cdh/
cp /root/parcel/* cm/parcel
cp /root/mysql/* mysql/
7、初始化目錄
cd /var/www/html/
createrepo cm
createrepo mysql
createrepo os
8、修改os.repo如下
[os]
name=os
baseurl=http://<hostname>/os
gpgcheck=0
9、分發repo檔案到所有節點
ssh <hostname> "mkdir /etc/yum.repo.d/repo_bak"
ssh <hostname> "mv /etc/yum.repo.d/Centos-* /etc/yum.repo.d/repo_bak"
scp /etc/yum.repo.d/*.repo <hostname>:/etc/yum.repo.d/
三、安裝CDH
1、安裝jdk
yum install oracle-j2sdk1.7-*
2、安裝mysql
rpm -qa | grep mariadb
rpm -e --nodeps <上述命令顯示的rpm包> ----移除已有的mariadb
yum install MySQL-*
service mysql start
mysql -uroot -p
set password for 'root'@'localhost'=password('root'); ----修改預設的root密碼
3、建立資料庫
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';
4、拷貝mysql連線包
mkdir /usr/share/java
cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
5、安裝cloudera server
yum install cloudera-manager-daemons cloudera-manager-server
6、初始化資料庫
/usr/share/cmf/schema/scm_prepare_database.sh --scm-host <安裝cm的節點> -h <資料庫的安裝節點> mysql cm cm cm
7、啟動cloudera server
systemctl start cloudera-scm-server
檢視啟動資訊:
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
啟動成功標誌:
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
8、開始安裝叢集
瀏覽器訪問以下地址:
http://<hostname>:7180