mysql ndbcluster 安裝

1.資料庫規劃：
作業系統 centos 6.3
ndbcluster 版本：MySQL-Cluster-gpl-7.4.8-1.el6.x86_64
2個管理節點、2個sql節點、2個數據節點（管理節點和sql節點放在同一臺伺服器上）
如下：
管理節點1 192.168.1.17   
管理節點2 192.168.1.18

sql節點1 192.168.1.17
sql節點2 192.168.1.18
ndbd 節點1 192.168.1.19  
ndbd 節點2 192.168.1.20  

2.作業系統設定：
設定服務主機名稱
關閉selinux和防火牆
設定核心引數和最大程序數

下載mysql叢集
MySQL-Cluster-gpl-7.4.8-1.el6.x86_64.rpm-bundle.tar
上傳到伺服器的相應目錄下如 /home/mysqlinstall

3.安裝管理節點和sql節點：
To check if your system has any RPM version of any MySQL package currently installed, run:
shell> rpm -qa | grep -i mysql
檢查
[

[email protected] ~]#  rpm -qa | grep -i mysql
mysql-libs-5.1.61-4.el6.x86_64
如果有刪除
shell> rpm -e mysql-libs-5.1.61-4.el6.x86_64
刪除報錯：
[[email protected] ~]# rpm -e mysql-libs-5.1.61-4.el6.x86_64
error: Failed dependencies:
        libmysqlclient.so.16()(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
        libmysqlclient.so.16(libmysqlclient_16)(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
        mysql-libs is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64
[

[email protected] ~]# 
解決：
[[email protected] ~]# rpm -e --nodeps mysql-libs-5.1.61-4.el6.x86_64
[[email protected] ~]# 

[[email protected] mysqlinstall]# rpm -ivh MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm 
[[email protected] mysqlinstall]# rpm -ivh MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm 

4.安裝資料節點：
執行上面步驟
[

[email protected] mysqlinstall]# rpm -ivh MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm 
[[email protected] mysqlinstall]# rpm -ivh MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm 

5.配置叢集
管理節點建立目錄
mkdir /var/lib/mysql-cluster
mkdir -p /data/mysql/ndbdata

管理節點配置檔案config.ini：
# TCP PARAMETERS
[tcp default]
SendBufferMemory=20M
ReceiveBufferMemory=20M
# Increasing the sizes of these 2 buffers beyond the default values
# helps prevent bottlenecks due to slow disk I/O.
# MANAGEMENT NODE PARAMETERS
[ndb_mgmd default]
DataDir=/data/mysql-cluster
# It is possible to use a different data directory for each management
# server, but for ease of administration it is preferable to be
# consistent.
[ndb_mgmd]
NodeId=1
HostName=192.168.1.17
# NodeId=management-server-A-nodeid
[ndb_mgmd]
NodeId=2
HostName=192.168.1.18
[ndbd default]
NoOfReplicas=2
# Using 2 replicas is recommended to guarantee availability of data;
# using only 1 replica does not provide any redundancy, which means
# that the failure of a single data node causes the entire cluster to
# shut down. We do not recommend using more than 2 replicas, since 2 is
# sufficient to provide high availability, and we do not currently test
# with greater values for this parameter.
LockPagesInMainMemory=1
# On Linux and Solaris systems, setting this parameter locks data node
# processes into memory. Doing so prevents them from swapping to disk,
# which can severely degrade cluster performance.
DataMemory=2000M
IndexMemory=800M
# The values provided for DataMemory and IndexMemory assume 4 GB RAM
# per data node. However, for best results, you should first calculate
# the memory that would be used based on the data you actually plan to
# store (you may find the ndb_size.pl utility helpful in estimating
# this), then allow an extra 20% over the calculated values. Naturally,
# you should ensure that each data node host has at least as much
# physical memory as the sum of these two values.
# ODirect=1
# Enabling this parameter causes NDBCLUSTER to try using O_DIRECT
# writes for local checkpoints and redo logs; this can reduce load on
# CPUs. We recommend doing so when using MySQL Cluster on systems running
# Linux kernel 2.6 or later.
NoOfFragmentLogFiles=600
MaxNoOfConcurrentOperations=100000
MaxNoOfLocalOperations=110000
SchedulerSpinTimer=400
SchedulerExecutionTimer=100
RealTimeScheduler=1
# Setting these parameters allows you to take advantage of real-time scheduling
# of NDB threads to achieve increased throughput when using ndbd. They
# are not needed when using ndbmtd; in particular, you should not set
# RealTimeScheduler for ndbmtd data nodes.
TimeBetweenGlobalCheckpoints=1000
TimeBetweenEpochs=200
#DiskCheckpointSpeed=20M
#DiskCheckpointSpeedInRestart=100M
MaxNoOfConcurrentScans=500
RedoBuffer=32M
# CompressedLCP=1
# CompressedBackup=1
# Enabling CompressedLCP and CompressedBackup causes, respectively, local
#checkpoint files and backup files to be compressed, which can result in a space
#savings of up to 50% over noncompressed LCPs and backups.
MaxNoOfLocalScans=1000
MaxNoOfTables=4096
MaxNoOfOrderedIndexes=2048
MaxNoOfExecutionThreads=4
TransactionDeadLockDetectionTimeOut=30000
HeartbeatIntervalDbDb=5000
HeartbeatIntervalDbApi=5000
MaxNoOfConcurrentTransactions=4096
MaxNoOfConcurrentIndexOperations=30000
MaxNoOfFiredTriggers=10000
MaxNoOfAttributes=600000
MaxNoOfUniqueHashIndexes=512
[ndbd]
Nodeid=3
HostName=192.168.1.19
DataDir=/data/mysql/ndbdata
# NodeId=data-node-A-nodeid
#LockExecuteThreadToCPU=1
#LockMaintThreadsToCPU=0
# On systems with multiple CPUs, these parameters can be used to lock NDBCLUSTER
# threads to specific CPUs
[ndbd]
Nodeid=4
HostName=192.168.1.20
DataDir=/data/mysql/ndbdata
# NodeId=data-node-B-nodeid
#LockExecuteThreadToCPU=1
#LockMaintThreadsToCPU=0
# You must have an [ndbd] section for every data node in the cluster;
# each of these sections must include a HostName. Each section may
# optionally include a NodeId for convenience, but in most cases, it is
# sufficient to allow the cluster to allocate node IDs dynamically. If
# you do specify the node ID for a data node, it must be in the range 1
# to 48 inclusive and must be unique among all IDs specified for
# cluster nodes.
# SQL NODE / API NODE PARAMETERS
[mysqld]
Nodeid=5
HostName=192.168.1.17
[mysqld]
Nodeid=6
HostName=192.168.1.18
[mysqld]
Nodeid=7
[mysqld]
Nodeid=8

sql節點和資料節點配置檔案/etc/my.cnf
[mysqld]
# Options for mysqld process:
ndbcluster                      #run NDB storage engine
#skip-grant-tables
character_set_server=utf8
lower_case_table_names=1
max_connections=1000
ndb-connectstring=192.168.1.17,192.168.1.18  # location of management server
default-storage-engine=ndbcluster
datadir=/data/mysql
# provide connection string for management server host (default port: 1186)
[ndbd]
connect-string=192.168.1.17,192.168.1.18  # location of management server
#provide connection string for management server host (default port: 1186)
[ndb_mgm]
connect-string=192.168.1.17,192.168.1.18  # location of management server
#provide location of cluster configuration file
[ndb_mgmd]
config-file=/var/lib/mysql-cluster/config.ini

5.啟動關閉叢集：
啟動管理節點
ndb_mgmd -f /var/lib/mysql-cluster/config.ini --initial (第一次啟動加--initial修改了配置檔案後的啟動)

啟動資料節點
ndbmtd --initial

啟動sql節點
service mysql start

檢視叢集狀態
shell> ndb_mgm -e "SHOW"

關閉管理節點和資料節點
ndb_mgm -e shutdown
關閉sql節點
service mysql stop

安裝遇到的問題：
1.許可權問題啟動mysql報錯
[[email protected] mysql]# service mysql start
Starting MySQL..The server quit without updating PID file ([FAILED]sql/sqlmgm2.pid).
[[email protected] mysql]# 

[[email protected] mysql]# ll
total 110644
-rw-rw---- 1 mysql mysql       56 Dec 14 17:13 auto.cnf
-rw-rw---- 1 mysql mysql 12582912 Dec 14 17:39 ibdata1
-rw-rw---- 1 mysql mysql 50331648 Dec 14 17:40 ib_logfile0
-rw-rw---- 1 mysql mysql 50331648 Dec 14 17:13 ib_logfile1
drwx------ 2 mysql mysql     4096 Dec 14 17:39 mysql
drwx------ 2 root  root      4096 Dec 14 17:39 ndbinfo
drwx------ 2 root  root      4096 Dec 14 17:39 performance_schema
-rw-r----- 1 mysql mysql    30068 Dec 14 17:40 sqlmgm2.err
drwx------ 2 root  root      4096 Dec 14 17:33 test
[[email protected] mysql]# chown mysql:mysql -R mysql
[[email protected] mysql]# service mysql start
Starting MySQL..                                           [  OK  ]
[[email protected] mysql]# 

2.安裝rpm包時，報包依賴關係如下：
[[email protected] mysqlinstall]# rpm -ivh MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm 
warning: MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
error: Failed dependencies:
        libnuma.so.1()(64bit) is needed by MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64
        libnuma.so.1(libnuma_1.1)(64bit) is needed by MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64
        libnuma.so.1(libnuma_1.2)(64bit) is needed by MySQL-Cluster-server-gpl-7.4.8-1.el6.x86_64       
解決方法是把安裝的依賴的包
[[email protected] mysqlinstall]# yum install numactl
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: mirrors.163.com
 * extras: mirrors.opencas.cn
 * updates: mirrors.opencas.cn
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package numactl.x86_64 0:2.0.9-2.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================================================
 Package                      Arch                        Version                             Repository                 Size
==============================================================================================================================
Installing:
 numactl                      x86_64                      2.0.9-2.el6                         base                       74 k
Transaction Summary
==============================================================================================================================
Install       1 Package(s)
Total download size: 74 k
Installed size: 171 k
Is this ok [y/N]: y
Downloading Packages:
numactl-2.0.9-2.el6.x86_64.rpm                                                                         |  74 kB     00:00     
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Importing GPG key 0xC105B9DE:
 Userid : CentOS-6 Key (CentOS 6 Official Signing Key) <[email protected]>
 Package: centos-release-6-3.el6.centos.9.x86_64 (@anaconda-CentOS-201207061011.x86_64/6.3)
 From   : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : numactl-2.0.9-2.el6.x86_64                                                                                 1/1 
  Verifying  : numactl-2.0.9-2.el6.x86_64                                                                                 1/1 
Installed:
  numactl.x86_64 0:2.0.9-2.el6                                                                                                
Complete!

3.啟動資料庫節點無法和管理節點相連
原因由於防火牆和selinux沒有關閉
解決關閉防火牆和selinux
一、關閉防火牆
1、重啟後永久性生效：
開啟：chkconfig iptables on
關閉：chkconfig iptables off
2、即時生效，重啟後失效：
開啟：service iptables start
關閉：service iptables stop
二、關閉SELinux
vim /etc/selinux/config  # 改為 SELINUX=disabled
# 儲存退出，重啟伺服器
init 6
禁用SeLinux
#永久禁用，需要重啟生效。
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config  
# 臨時禁用，不需要重啟
setenforce 0  

4.root使用者不能登入
[[email protected] ~]# mysql -uroot -p
Enter password: 
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
[[email protected] ~]# mysql -uroot -p
Enter password: 
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

設定--skip-grant-tables跳過授權表認證
service mysqld stop
在配置檔案中新增--skip-grant-tables（/etc/my.cnf）
service mysqld start
另外開個SSH連線
[[email protected] ~]# mysql
mysql>use mysql
mysql>update user set password=password("123456") where user="root";
mysql>flush privileges;
mysql>exit
去掉--skip-grant-tables重啟mysql,root使用者可以用設定的密碼登入

設定root使用者遠端登入
GRANT ALL PRIVILEGES ON *.* TO [email protected]'%' IDENTIFIED BY '123456' WITH GRANT OPTION; 
執行報如下錯誤，ERROR 1820 (HY000): You must SET PASSWORD before executing this statement
[[email protected] ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.27-ndb-7.4.8-cluster-gpl
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> 
mysql> 
mysql> GRANT ALL PRIVILEGES ON *.* TO [email protected]"%" IDENTIFIED BY 'root' WITH GRANT OPTION;
ERROR 1820 (HY000): You must SET PASSWORD before executing this statement
mysql> GRANT ALL PRIVILEGES ON *.* TO [email protected]"%" IDENTIFIED BY 'root001' WITH GRANT OPTION;
ERROR 1820 (HY000): You must SET PASSWORD before executing this statement

執行SET PASSWORD修改成功
mysql> SET PASSWORD = PASSWORD('root001');
Query OK, 0 rows affected (0.02 sec)
mysql>  GRANT ALL PRIVILEGES ON *.* TO [email protected]"%" IDENTIFIED BY 'root' WITH GRANT OPTION;
Query OK, 0 rows affected (0.02 sec)
mysql> 
FLUSH PRIVILEGES;

執行完成
遠端連線測試，mysql叢集節點，發現一個節點可以連線，一個節點不能連線
這個問題糾結很久
原因上面的操作，沒有在另外一個節點上執行導致的，mysql叢集的使用者認證是分開管理的，要兩個節點都要執行
索引以後建立使用者要兩個節點都執行

5.配置兩個管理節點時show報錯Could not get configuration
[[email protected] mysql-cluster]# ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: 192.168.1.17:1186
ERROR Message: The cluster configuration is not yet confirmed by all defined management servers. This management server is still waiting for node 6 to connect.
Could not get configuration
*  4012: Failed to get configuration
*        The cluster configuration is not yet confirmed by all defined management servers. This management server is still waiting for node 6 to connect.
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2 (not connected, accepting connect from 192.168.1.18)
id=3 (not connected, accepting connect from 192.168.1.17)
[ndb_mgmd(MGM)] 2 node(s)
id=1    @192.168.1.17  (mysql-5.6.27 ndb-7.4.8)
id=6    @192.168.1.18  (mysql-5.6.27 ndb-7.4.8)
[mysqld(API)]   2 node(s)
id=4 (not connected, accepting connect from 192.168.1.17)
id=5 (not connected, accepting connect from 192.168.1.18)
兩個管理節點同時都啟動起來，才能看到狀態
ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @192.168.1.18  (mysql-5.6.27 ndb-7.4.8, Nodegroup: 0, *)
id=3    @192.168.1.17  (mysql-5.6.27 ndb-7.4.8, Nodegroup: 0)
[ndb_mgmd(MGM)] 2 node(s)
id=1    @192.168.1.17  (mysql-5.6.27 ndb-7.4.8)
id=6    @192.168.1.18  (mysql-5.6.27 ndb-7.4.8)
[mysqld(API)]   2 node(s)
id=4    @192.168.1.17  (mysql-5.6.27 ndb-7.4.8)
id=5    @192.168.1.18  (mysql-5.6.27 ndb-7.4.8)