cdh5.3.10_hadoop環境遷移
主機遷移後hadoop叢集調整：
1、修改/etc/hosts（三臺主機均需要修改）
修改前：
[[email protected] ~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost.localdomain localhost4 localhost4.localdomain4 localhost
::1 localhost.localdomain localhost.localdomain localhost6 localhost6.localdomain6 localhost

10.117.130.207 hadoop-master
10.117.130.208 hadoop-slave1
10.117.130.209 hadoop-slave2
[

[email protected] ~]# 
修改後：
[[email protected] ~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost.localdomain localhost4 localhost4.localdomain4 localhost
::1 localhost.localdomain localhost.localdomain localhost6 localhost6.localdomain6 localhost

10.117.130.110 hadoop-master
10.117.130.111 hadoop-slave1
10.117.130.112 hadoop-slave2
[

[email protected] ~]#

2、修改ntp時鐘同步（三臺主機均需要修改）
修改前：
[[email protected] ~]# cat /etc/ntp.conf 
server 10.117.130.207 prefer
修改後：
[[email protected] ~]# cat /etc/ntp.conf 
server 10.117.130.110 prefer

3、修改mysql中hadoop相關配置（僅在master節點修改）
修改前：
[[email protected] data]# mysql -u root -p 
Enter password: 
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 58
Server version: 5.6.29 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| am                 |
| hive               |
| mysql              |
| oozie              |
| performance_schema |
| rm                 |
| scm                |
| test               |
+--------------------+
9 rows in set (0.00 sec)


mysql> use scm;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A


Database changed
mysql> show tables;
+--------------------------------+
| Tables_in_scm                  |
+--------------------------------+
| AUDITS                         |
| CLIENT_CONFIGS                 |
| CLIENT_CONFIGS_TO_HOSTS        |
| CLUSTERS                       |
| CLUSTERS_AUD                   |
| CLUSTER_ACTIVATED_RELEASES     |
| CLUSTER_ACTIVATED_RELEASES_AUD |
| CLUSTER_MANAGED_RELEASES       |
| CLUSTER_UNDISTRIBUTED_RELEASES |
| CM_PEERS                       |
| CM_VERSION                     |
| COMMANDS                       |
| COMMAND_SCHEDULES              |
| CONFIGS                        |
| CONFIGS_AUD                    |
| CONFIG_CONTAINERS              |
| CREDENTIALS                    |
| GLOBAL_SETTINGS                |
| HOSTS                          |
| HOSTS_AUD                      |
| HOST_TEMPLATES                 |
| HOST_TEMPLATE_TO_ROLE_CONF_GRP |
| METRICS                        |
| PARCELS                        |
| PARCEL_COMPONENTS              |
| PROCESSES                      |
| PROCESS_ACTIVE_RELEASES        |
| RELEASES                       |
| RELEASES_AUD                   |
| REVISIONS                      |
| ROLES                          |
| ROLES_AUD                      |
| ROLE_CONFIG_GROUPS             |
| ROLE_CONFIG_GROUPS_AUD         |
| ROLE_STALENESS_STATUS          |
| SCHEMA_VERSION                 |
| SERVICES                       |
| SERVICES_AUD                   |
| SNAPSHOT_POLICIES              |
| USERS                          |
| USER_ROLES                     |
| USER_SETTINGS                  |
+--------------------------------+
42 rows in set (0.00 sec)

mysql> mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER                      | NAME          | IP_ADDRESS     | RACK_ID  | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
|       1 |                      14 | 135caa69-b742-467b-bf08-6742781a1b2c | hadoop-master | 10.117.130.207 | /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
|       2 |                      13 | de481baa-e125-4c35-8538-aa3bf7229f33 | hadoop-slave1 | 10.117.130.208| /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
|       3 |                      11 | 6af8fa7f-1ce1-47cd-a1a6-6d8ef91c77f2 | hadoop-slave2 | 10.117.130.209 | /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.110' where host_id=1;
Query OK, 1 row affected (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.111' where host_id=2;
Query OK, 1 row affected (0.00 sec)
mysql> update HOSTS set ip_address= '10.117.130.112' where host_id=3;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0

mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER                      | NAME          | IP_ADDRESS     | RACK_ID  | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
|       1 |                      14 | 135caa69-b742-467b-bf08-6742781a1b2c | hadoop-master | 10.117.130.110 | /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
|       2 |                      13 | de481baa-e125-4c35-8538-aa3bf7229f33 | hadoop-slave1 | 10.117.130.111 | /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
|       3 |                      11 | 6af8fa7f-1ce1-47cd-a1a6-6d8ef91c77f2 | hadoop-slave2 | 10.117.130.112 | /default | NA     |                   1 |                 0 |                  0 |          1 |         4 |          12626055168 | NULL        | NULL              | NULL           |
+---------+-------------------------+--------------------------------------+---------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)


mysql>

mysql> commit;
Query OK, 0 rows affected (0.00 sec)
mysql>

4、重啟cdh-hadoop的服務
[[email protected] data]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-server restart
鍋滄 cloudera-scm-server錛 [紜畾]
Starting cloudera-scm-server: [紜畾]
[[email protected] data]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent restart
Stopping cloudera-scm-agent: [紜畾]
姝ｅ湪鍚姩 cloudera-scm-agent錛 [紜畾]
[[email protected] data]
[[email protected] ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent錛 [紜畾]
[[email protected] ~]# 
[[email protected] ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent錛 [紜畾]
[[email protected] ~]#

5、登入cdh叢集web管理頁面
http://10.117.130.110:7180/cmf/home
首先保證CDH第一個服務即主機一項是正常的
看到部分服務提示配置資訊過舊：需要重新部署客戶端配置 此時點選連結按照頁面提示重新部署客戶端配置即可

6、重新部署cdh部分服務的客戶端配置後重啟cdh服務，cdh恢復正常

注意：由於cdh三臺虛擬主機是映象過來的，cdh相關服務檔案或程序有狀態保留，需要查殺相關程序
本次處理中，碰到節點2重啟agent客戶端程序一直在執行，問題出在9000埠相應的程序佔用，導致cdh識別節點2出現問題
[[email protected] ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
cloudera-scm-agent is already running
[[email protected] ~]# netstat -nap | grep 9000
tcp 0 0 10.117.130.111:9000 0.0.0.0:* LISTEN 12510/python 
tcp 0 0 10.117.130.111:44223 10.117.130.111:9000 TIME_WAIT - 
[[email protected] ~]# kill -9 0
Connection to hadoop-slave1 closed.
[[email protected] ~]# netstat -nap | grep 9000
[[email protected] ~]# /opt/cloudera-manager/cm-5.3.10/etc/init.d/cloudera-scm-agent start
姝ｅ湪鍚姩 cloudera-scm-agent錛 [紜畾]
[[email protected] ~]# netstat -nap | grep 9000
tcp 0 0 10.117.130.112:9000 0.0.0.0:* LISTEN 3689/python 
tcp 0 0 10.117.130.112:50759 10.117.130.112:9000 TIME_WAIT - 
[[email protected] ~]#