1. 程式人生 > >CentOS和Ubuntu下安裝配置Greenplum資料庫叢集(包括安裝包和原始碼編譯安裝)

CentOS和Ubuntu下安裝配置Greenplum資料庫叢集(包括安裝包和原始碼編譯安裝)

首先說一下,無論是CentOS/RedHat還是Ubuntu都可以按原始碼方式、安裝包方式編譯安裝。

1.   規劃

192.168.4.93(h93)       1個主master  2個主segment、2個映象segment

192.168.4.94(h94)       1個備master  2個主segment、2個映象segment

都安裝在/home/wslu/gp/gpsql目錄。

2.   安裝依賴

在h93和h94安裝以下依賴:

2.1.  Ubuntu

apt-get install -y git-core
apt-get install -y gcc g++
apt-get install -y ccache
apt-get install -y libreadline-dev
apt-get install -y bison flex
apt-get install -y zlib1g-dev
apt-get install -y openssl libssl-dev
apt-get install -y libpam-dev
apt-get install -y libcurl4-dev
apt-get install -y libbz2-dev
apt-get install -y python-dev
apt-get install -y ssh
 
apt-get install -y libcurl4-dev
Package libcurl4-dev is a virtual package provided by:
libcurl4-openssl-dev 7.38.0-4+deb8u2
libcurl4-nss-dev 7.38.0-4+deb8u2
libcurl4-gnutls-dev 7.38.0-4+deb8u2
 
apt-get install -y python-pip
 
pip install lockfile
pip install paramiko
pip install setuptools
pip install epydoc
pip install psi
Note: debian8 required pip install --pre psi

2.2.  Centos

yum源要按照我的總結中的設定。

yum install –y git.x86_64
yum install –y gcc.x86_64 gcc-c++.x86_64
yum install –y ccache.x86_64
yum install readline.x86_64 readline-devel.x86_64
yum install bison.x86_64 bison-devel.x86_64
yum install flex.x86_64 flex-devel.x86_64
yum install zlib.x86_64 zlib-devel.x86_64
yum install -y openssl.x86_64 openssl-devel.x86_64
yum install -y pam.x86_64 pam-devel.x86_64
yum install –y libcurl.x86_64 libcurl-devel.x86_64
yum install bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64
yum install libssh2.x86_64 libssh2-devel.x86_64
yum install python-devel.x86_64
yum install -y python-pip.noarch
 
接著執行:
pip install lockfile
pip install paramiko
pip install setuptools
pip install epydoc
pip install psi
或者執行:
yum install python-lockfile.noarch
yum install python-PSI.x86_64
yum install python-paramiko.noarch
yum install python-setuptools.noarch
yum install epydoc.noarch

3.   原始碼安裝

3.1.  克隆原始碼

在h93和h94執行:

$ mkdir /home/wslu/gp/greenplum
$ cd /home/wslu/gp/greenplum
$ git clone https://github.com/greenplum-db/gpdb.

3.2.  編譯安裝

在h93和h94執行:

$ cd /home/wslu/gp/greenplum
$ CFLAGS+="-O2" ./configure--prefix=/home/wslu/gp/gpsql --enable-debug --enable-depend --enable-cassert
$ make
$ make install

安裝時如果遇到某些python包(lockfile、 paramiko、PSI等)找不到,可以參考HAWQ, (原始碼https://github.com/apache/incubator-hawq)<hawq_src>/tools/bin/pythonSrc/ 下所有的壓縮包拷貝到/home/wslu/gp/greenplum/gpMgmt/bin/pythonSrc/ext/ 中,然後再make install即可。

至此叢集原始碼編譯完成。

4.   安裝包安裝

下載greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip

$ unzip greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip

在h93和h94使用普通使用者安裝:

$./greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.bin 

安裝到 /home/wslu/gp/gpsql

5.   設定作業系統引數

在h93和h94執行下述設定:

5.1.  系統級引數配置

以下配置需要在root下進行配置,所以需要能夠得到root許可權。

首先是防火牆的設定:

    service iptables status    #檢視是否開啟了防火牆
    service iptables stop       #關閉防火牆
    chkconfig iptables off      #設定開機不會啟動防火牆

其次是ssh的速度設定:

  sed -i 's/^GSS/#&/g' /etc/ssh/sshd_config      #用來加速SSH連線的
    service sshd reload

核心處理和記憶體方面的設定:

# 設定核心引數, 並在啟動時生效
sysctl -p - >>/etc/sysctl.conf <<EOF
# configurations
kernel.sysrq=1
kernel.core_pattern=core
kernel.core_uses_pid=1
kernel.msgmnb=65536
kernel.msgmax=65536
kernel.msgmni=2048
kernel.sem=25600 3200000 10000 14200
net.ipv4.tcp_syncookies=1
net.ipv4.ip_forward=0
net.ipv4.conf.default.accept_source_route=0
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_max_syn_backlog=4096
net.ipv4.conf.all.arp_filter=1
net.ipv4.ip_local_port_range=1025 65535
net.core.netdev_max_backlog=10000
net.core.rmem_max=2097152
net.core.wmem_max=2097152
vm.overcommit_memory=1
EOF

如果配置叢集時,因為要設定的共享記憶體大於系統最大共享記憶體而報錯,則可以按照以下官方推薦的設定配置:

# vi /etc/sysctl.conf
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1 
kernel.core_uses_pid = 1 
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1 
net.ipv4.ip_forward = 0 
net.ipv4.conf.default.accept_source_route = 0 
net.ipv4.tcp_tw_recycle = 1 
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1 
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2


檔案讀寫方面的設定:

# 設定limits
cat >>/etc/security/limits.d/99-kingbasedbcloud.conf <<EOF
#KingbaseDBCloud configurations
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
EOF

5.2. 資料庫級的引數設定

GUC引數設定,此設定需要按照實際機器來進行配置,常用的配置有如下幾點:

work_mem=1GB
shared_buffers=2GB
max_connections=500
max_pool_size=2000
enable_mergejoin=off
enable_nestloop=off
max_prepared_transactions=50
autovacuum=off
interconnect_setup_timeout=1200

6.   demo叢集

安裝完成後,就可以使用demo叢集了。

執行下面命令即可在本機建立包含3個segment,3個segment-mirror,1個master的叢集:

$ cd /home/wslu/gp/gpsql
$ source greenplum_path.sh
$ gpssh-exkeys –h localhost
$ cd gpAux/gpdemo
$ make cluster
$ source gpdemo-env.sh

如果不想用demo叢集,可以直接跳過本小節。

7.   設定環境變數

$ source gpsql/greenplum_path.sh
$ exportMASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1

8.   交換key

在h93執行:

$ gpssh-exkeys –h h94

         會提示輸入h94的wslu使用者的密碼。(使用者必須和h93相同)

則h93和h94之間可以無密碼登陸了。

9.初始化叢集

(1)       在h93和h94執行下述指令,以建立資料目錄:

$ mkdir gpsql/data/primary gpsql/data/mirror gpsql/data/master –p

(2)       在h93建立配置檔案configs/gpinitsystem_config,內容如下:

ARRAY_NAME="EMC Greenplum DW"
SEG_PREFIX=gpseg
PORT_BASE=40000
declare -a DATA_DIRECTORY=(/home/wslu/gp/gpsql/data/primary /home/wslu/gp/gpsql/data/primary)
MASTER_HOSTNAME=h93
MASTER_DIRECTORY=/home/wslu/gp/gpsql/data/master
MASTER_PORT=5432
TRUSTED_SHELL=ssh
CHECK_POINT_SEGMENTS=8
ENCODING=UNICODE
MIRROR_PORT_BASE=50000
REPLICATION_PORT_BASE=41000
MIRROR_REPLICATION_PORT_BASE=51000
declare -a MIRROR_DATA_DIRECTORY=(/home/wslu/gp/gpsql/data/mirror /home/wslu/gp/gpsql/data/mirror)

注意:configs目錄是我自己建立的、便於儲存自定義配置檔案的目錄。該步驟的目的是建立一個初始化時要用的配置檔案,並沒有路徑的要求。

(3)       在h93建立配置檔案configs/hostfile_gpinitsystem,內容如下:

h93
h94

注意:configs目錄是我自己建立的、便於儲存自定義配置檔案的目錄。該步驟的目的是建立一個初始化時要用的配置檔案,並沒有路徑的要求。

(4)       在h93執行下述指令初始化叢集:

[[email protected] gpsql]$ gpinitsystem -c configs/gpinitsystem_config -h configs/hostfile_gpinitsystem –a
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Checking configuration parameters, please wait...
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Reading Greenplum configuration file configs/gpinitsystem_config
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Locale has not been set in configs/gpinitsystem_config, will set to default value
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Locale set to en_US.utf8
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-No DATABASE_NAME set, will exit following template1 updates
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-MASTER_MAX_CONNECT not set, will set to default value 250
20160114:14:30:03:005980 gpinitsystem:h93:wslu-[INFO]:-Checking configuration parameters, Completed
20160114:14:30:04:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing multi-home checks, please wait...
..
20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Configuring build for standard array
20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing multi-home checks, Completed
20160114:14:30:05:005980 gpinitsystem:h93:wslu-[INFO]:-Building primary segment instance array, please wait...
....
20160114:14:30:08:005980 gpinitsystem:h93:wslu-[INFO]:-Building group mirror array type , please wait...
....
20160114:14:30:12:005980 gpinitsystem:h93:wslu-[INFO]:-Checking Master host
20160114:14:30:12:005980 gpinitsystem:h93:wslu-[INFO]:-Checking new segment hosts, please wait...
........
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Checking new segment hosts, Completed
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Database Creation Parameters
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master Configuration
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master instance name       = EMC Greenplum DW
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master hostname            = h93
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master port                = 5432
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master instance dir        = /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:30:28:005980 gpinitsystem:h93:wslu-[INFO]:-Master LOCALE              = en_US.utf8
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum segment prefix   = gpseg
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master Database            =
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master connections         = 250
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master buffers             = 128000kB
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Segment connections        = 750
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Segment buffers            = 128000kB
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Checkpoint segments        = 8
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Encoding                   = UNICODE
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Postgres param file        = Off
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Initdb to be used          = /home/wslu/gp/gpsql/bin/initdb
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-GP_LIBRARY_PATH is         = /home/wslu/gp/gpsql/lib
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Ulimit check               = Passed
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Array host connect type    = Single hostname per node
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [1]      = ::1
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [2]      = 192.168.4.93
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Master IP address [3]      = fe80::225:90ff:fe3b:86c2
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Standby Master             = Not Configured
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Primary segment #          = 2
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Total Database segments    = 4
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Trusted shell              = ssh
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Number segment hosts       = 2
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror port base           = 50000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Replicaton port base       = 41000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror replicaton port base= 51000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirror segment #           = 2
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirroring config           = ON
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Mirroring type             = Group
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:----------------------------------------
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Primary Segment Configuration
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:----------------------------------------
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/primary/gpseg0        40000          2          0       41000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/primary/gpseg1        40001          3          1       41001
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/primary/gpseg2        40000          4          2       41000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/primary/gpseg3        40001          5          3       41001
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Mirror Segment Configuration
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:---------------------------------------
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/mirror/gpseg0          50000          6          0       51000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h94      /home/wslu/gp/gpsql/data/mirror/gpseg1          50001          7          1       51001
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/mirror/gpseg2          50000          8          2       51000
20160114:14:30:29:005980 gpinitsystem:h93:wslu-[INFO]:-h93      /home/wslu/gp/gpsql/data/mirror/gpseg3          50001          9          3       51001
Continue with Greenplum creation Yy/Nn>
y
20160114:14:30:32:005980 gpinitsystem:h93:wslu-[INFO]:-Building the Master instance database, please wait...
20160114:14:31:08:005980 gpinitsystem:h93:wslu-[INFO]:-Starting the Master in admin mode
20160114:14:32:01:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing parallel build of primary segment instances
20160114:14:32:01:005980 gpinitsystem:h93:wslu-[INFO]:-Spawning parallel processes    batch [1], please wait...
....
20160114:14:32:02:005980 gpinitsystem:h93:wslu-[INFO]:-Waiting for parallel processes batch [1], please wait...
...........................................................
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Parallel process exit status
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as completed           = 4
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as killed              = 0
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as failed              = 0
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Commencing parallel build of mirror segment instances
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Spawning parallel processes    batch [1], please wait...
....
20160114:14:33:01:005980 gpinitsystem:h93:wslu-[INFO]:-Waiting for parallel processes batch [1], please wait...
.........................................
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Parallel process exit status
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as completed           = 4
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as killed              = 0
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Total processes marked as failed              = 0
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:------------------------------------------------
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Deleting distributed backout files
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Removing back out file
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-No errors generated from parallel processes
20160114:14:33:43:005980 gpinitsystem:h93:wslu-[INFO]:-Restarting the Greenplum instance in production mode
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Starting gpstop with args: -a -i -m -d /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Gathering information and validating the environment...
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Obtaining Greenplum Master catalog information
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Obtaining Segment details from master...
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 4.3.99.00 build dev'
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-There are 0 connections to the database
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Commencing Master instance shutdown with mode='immediate'
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Master host=h93
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Commencing Master instance shutdown with mode=immediate
20160114:14:33:43:001932 gpstop:h93:wslu-[INFO]:-Master segment instance directory=/home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:33:44:001932 gpstop:h93:wslu-[INFO]:-Attempting forceful termination of any leftover master process
20160114:14:33:44:001932 gpstop:h93:wslu-[INFO]:-Terminating processes for segment /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Starting gpstart with args: -a -d /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Gathering information and validating the environment...
20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Greenplum Binary Version: 'postgres (Greenplum Database) 4.3.99.00 build dev'
20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Greenplum Catalog Version: '300701081'
20160114:14:33:45:002019 gpstart:h93:wslu-[INFO]:-Starting Master instance in admin mode
20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Obtaining Greenplum Master catalog information
20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Obtaining Segment details from master...
20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Setting new master era
20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Master Started...
20160114:14:33:46:002019 gpstart:h93:wslu-[INFO]:-Shutting down master
20160114:14:33:47:002019 gpstart:h93:wslu-[INFO]:-Commencing parallel primary and mirror segment instance startup, please wait...
........
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Process results...
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Successful segment starts                                            = 8
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Failed segment starts                                                = 0
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-   Skipped segment starts (segments are marked down in configuration)   = 0
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Successfully started 8 of 8 segment instances
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-----------------------------------------------------
20160114:14:33:55:002019 gpstart:h93:wslu-[INFO]:-Starting Master instance h93 directory /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-Command pg_ctl reports Master h93 instance active
20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-No standby master configured.  skipping...
20160114:14:33:56:002019 gpstart:h93:wslu-[INFO]:-Database successfully started
20160114:14:33:59:005980 gpinitsystem:h93:wslu-[INFO]:-Completed restart of Greenplum instance in production mode
20160114:14:33:59:005980 gpinitsystem:h93:wslu-[INFO]:-Loading gp_toolkit...
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Scanning utility log file for any warning messages
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Log file scan check passed
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Greenplum Database instance successfully created
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To complete the environment configuration, please
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-update wslu .bashrc file with the following
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-1. Ensure that the greenplum_path.sh file is sourced
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-2. Add "export MASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1"
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   to access the Greenplum scripts for this instance:
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   or, use -d /home/wslu/gp/gpsql/data/master/gpseg-1 option for the Greenplum scripts
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-   Example gpstate -d /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Script log file = /home/wslu/gpAdminLogs/gpinitsystem_20160114.log
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To remove instance, run gpdeletesystem utility
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-To initialize a Standby Master Segment for this Greenplum instance
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Review options for gpinitstandby
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-The Master /home/wslu/gp/gpsql/data/master/gpseg-1/pg_hba.conf post gpinitsystem
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-has been configured to allow all hosts within this new
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-array to intercommunicate. Any hosts external to this
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-new array must be explicitly added to this file
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-Refer to the Greenplum Admin support guide which is
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-located in the /home/wslu/gp/gpsql/docs directory
20160114:14:34:02:005980 gpinitsystem:h93:wslu-[INFO]:-------------------------------------------------------
[[email protected] gpsql]$

(5)       檢視目錄結構:

[[email protected] gpsql]$ ls data
master  mirror  primary
[[email protected] gpsql]$ ls data/master/
gpseg-1
[[email protected] gpsql]$ ls data/mirror/
gpseg2  gpseg3
[[email protected] gpsql]$ ls data/primary/
gpseg0  gpseg1
[[email protected] gpsql]$
 
[[email protected] gpsql]$ ls data/
master  mirror  primary
[[email protected] gpsql]$ ls data/master/
[[email protected] gpsql]$ ls data/primary/
gpseg2  gpseg3
[[email protected] gpsql]$ ls data/mirror/
gpseg0  gpseg1
[[email protected] gpsql]$

(6)       在h94初始化備master(主備master必須在不同主機,如果要配置單機多節點,則不能配置備master。這是因為目前主備master必須在相同目錄,所以必然不同主機如果埠不是5432那麼需要指定PGPORT):

[[email protected] gpsql]$ PGPORT=5432 PGDATABASE=postgres gpinitstandby -s h94
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Validating environment and parameters for standby initialization...
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Checking for filespace directory /home/wslu/gp/gpsql/data/master/gpseg-1 on h94
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master initialization parameters
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master hostname               = h93
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master data directory         = /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum master port                   = 5432
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master hostname       = h94
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master port           = 5432
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum standby master data directory = /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-Greenplum update system catalog         = On
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:- Filespace locations
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:------------------------------------------------------
20160114:14:40:47:003933 gpinitstandby:h93:wslu-[INFO]:-pg_system -> /home/wslu/gp/gpsql/data/master/gpseg-1
Do you want to continue with standby master initialization? Yy|Nn (default=N):
> y
20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Syncing Greenplum Database extensions to standby
20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-The packages on h94 are consistent.
20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Adding standby master to catalog...
20160114:14:40:53:003933 gpinitstandby:h93:wslu-[INFO]:-Database catalog updated successfully.
20160114:14:40:54:003933 gpinitstandby:h93:wslu-[INFO]:-Updating pg_hba.conf file...
20160114:14:41:00:003933 gpinitstandby:h93:wslu-[INFO]:-pg_hba.conf files updated successfully.
20160114:14:41:09:003933 gpinitstandby:h93:wslu-[INFO]:-Updating filespace flat files...
20160114:14:41:09:003933 gpinitstandby:h93:wslu-[INFO]:-Filespace flat file updated successfully.
20160114:14:41:10:003933 gpinitstandby:h93:wslu-[INFO]:-Starting standby master
20160114:14:41:10:003933 gpinitstandby:h93:wslu-[INFO]:-Checking if standby master is running on host: h94  in directory: /home/wslu/gp/gpsql/data/master/gpseg-1
20160114:14:41:11:003933 gpinitstandby:h93:wslu-[INFO]:-Cleaning up pg_hba.conf backup files...
20160114:14:41:17:003933 gpinitstandby:h93:wslu-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
20160114:14:41:17:003933 gpinitstandby:h93:wslu-[INFO]:-Successfully created standby master on h94

(7)       此時,h94的data/master目錄就不為空了:

$ [[email protected] gpsql]$ ls data/master/
gpseg-1

10. 測試

[[email protected] gpsql]$ psql -p 5432 postgres
psql (8.3devel)
Type "help" for help.
 
postgres=#
postgres=#
postgres=# \db
        List of tablespaces
    Name    | Owner | Filespae Name
------------+-------+---------------
 pg_default | wslu  | pg_system
 pg_global  | wslu  | pg_system
(2 rows)
至此,叢集就初始化完成了。 叢集初始化完成後,預設會啟動叢集。
---------------------------------------------------------華麗的分割線------------------------------------------------------------- 以下為Greenplum叢集部分常用指令,以及關於配置單機多節點的一點說明。

11. 啟動叢集

以後,想手動啟動叢集,執行:

$ gpstart –a

這裡說一下,每次想使用叢集的任何指令,必須執行:

$ source greenplum-path.sh
$ exportMASTER_DATA_DIRECTORY=/home/wslu/gp/gpsql/data/master/gpseg-1

下文不再贅述。

12.停止叢集

$ gpstop –a

13.重啟叢集

$ gpstop –a –r

14. 檢視叢集狀態

$ gpstate –m | -e

15.reload配置檔案

在不停止叢集情況下,若配置檔案修改,reload配置檔案:

$ gpstop –u

16. 維護模式下啟動master

僅僅啟動master來執行維護管理任務,不會影響segment中的資料。例如,在維護模式下你可以僅連線master例項的資料庫並且編輯系統表設定。

1.執行gpstart使用-m選項:

$ gpstart –m

2.維護模式下連線master來維護系統表。例如:

$ PGOPTIONS='-c gp_session_role=utility'psql template1

3.完成管理任務後,使master關閉工具模式。然後,重啟進入正常模式:

$ gpstop -m

17.訪問資料庫

可以使用psql連線叢集:

 [[email protected] gpsql]$ psql -p 5432 postgres
psql (8.3devel)
Type "help" for help.
 
postgres=#

基本上常見命令與PostgreSQL的相同。可參考PostgreSQL官方手冊和Greenplum官方手冊。

18.GUC引數配置

使用gpconfig設定guc引數

$ gpconfig -c gp_vmem_protect_limit -v4096MB

gpconfig可以設定master和所有segment的guc引數,也可以使用--masteronly引數只設置master的引數。設定完guc引數後需要根據guc引數型別決定重啟叢集或reload配置檔案。

顯示guc引數

$ psql –c ‘showstatement_mem;’ 或 gpconfig –show statement_mem
$ psql –c ‘show all;’ 或 gpconfig –l

19.附錄:關於配置單機多節點的說明

如果要配置單機多節點,比如只想在h93配置2個主segment、2個映象segment、1個master,只需要把hostfile_config中的h94刪掉,然後在h93刪除data/primary,data/mirror,data/master目錄下的內容,重新初始化即可。

相關推薦

CentOSUbuntu安裝配置Greenplum資料庫叢集包括安裝原始碼編譯安裝

首先說一下,無論是CentOS/RedHat還是Ubuntu都可以按原始碼方式、安裝包方式編譯安裝。 1.   規劃 192.168.4.93(h93)       1個主master  2個主segment、2個映象segment 192.168.4.94(h94) 

Linux(ubuntu)Pycharm配置python執行環境編譯器中匯入虛擬環境

Pycharm配置python的執行環境(Linux(ubuntu)、Windows) 前言 前段時間跑python程式碼,一直在編輯器中寫,命令列中呼叫執行。如果寫一些小程式還可以,但如果要搭建神經網路,用到tensorflow或theano這些深度框

Ubuntu搭建深度學習環境教程scikit-neuralnetwork、matplotlib、Theano等

      今天給大家介紹下在ubuntu14.04中安裝scikit-neuralnetwork:(安裝流程為從頂層向下安裝,頂層包需要下層什麼樣的包就再補安裝什麼樣的包去滿足頂層包的需要環境,這樣安裝的成功率會很高),另外需要有耐心去檢視ubuntu的Terminal提示

MacBook Air安裝Win7的經驗總結包括雙系統,單系統WIN7

    用MAC自帶的BOOT CAMP很簡單。說明下工具,MACBOOK AIR,2G記憶體,128G硬碟,無光碟機,無網上說購機送的系統恢復U盤,手頭只有個聯想的win7系統本,一個8GU盤,一個行動硬碟。     實用工具裡找到Boot camp助手,發現根本沒有網上

UbuntuHive配置MySQL資料庫安裝方式

Hive只在一個節點上安裝即可 一、安裝MySQL •Ubuntu 採用apt-get安裝 •sudo apt-get install mysql-server •建立資料庫hive •c

centosubuntupycharm無法輸入中文的解決辦法

添加 div /usr usr 網上 pycha 編輯 ibus code 編輯啟動的腳本文件 vim /usr/bin/pycharm ubuntu下添加 export GTK_IM_MODULE=fcitx export QT_IM_MODULE=fcit

ubuntuQt配置qwtlibmodbus

Qt安裝在ubuntu上出了很多問題,其中一個就是qmake不能用。 解決方法:進入/usr/lib/x86_64-linux-gnu/qt-default/qtchooser目錄,對目錄下的default.conf修改為 /home/mlw/Qt5.7.0/5.7/gcc_64/bin /

解決 CentOSUbuntu virt-manager方格亂碼的問題

只是因為沒有相應的字型而已,下載就好了。 CentOS 下: yum install dejavu-lgc-sans-fonts。 Ubuntu下: apt install font-manager apt install fonts-arphic-ukai apt in

windowsUbuntu安裝mongodb

windows 下載 mongodb官網下載壓縮版安裝包:下載地址:https://www.mongodb.com/download-center/community 注意選擇版本(目前windows只有一個版本,同時適用於32位和64位系統) 安裝 1、解壓: mongodb-win32-x8

Linux CentOS安裝配置MySQL資料庫

安裝mysql資料庫   a)下載mysql源安裝包:wget http://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm   b)安裝mysql源:yum localinstall mysql5

window ubuntu的 autopep8 安裝

第一步: windows:cmd視窗輸入:pip install autopep8 ubuntu:pip install autopep8 許可權不夠的話輸入sudo pip install autopep8 第二步: 將pep8 配置到軟體裡面(我這裡

windows安裝配置MYSQL資料庫mysql5.7版本

一、下載地址:http://dev.MySQL.com/downloads/mysql/ 不用註冊,直接下載 二、下載後直接解壓就行,不需安裝 三、配置 1.我下載的版本mysql-5.7.23-winx64,解壓在d盤根目錄中的。因此在D:\mysql-5.7

Opencv 2.4.9在Ubuntu配置安裝

一、依賴包安裝        1.  build-essential 軟體包,為編譯程式提供必需軟體包的列表資訊,這樣軟體包才知道標頭檔案、庫函式在哪裡。還會下載依賴的軟體包,安裝gcc/g++/gdb/make 等基本程式設計工具,最後才組成一個開發環境。 sudo a

新手學Linux----Linux安裝配置MongoDB資料庫

    最近在學習研究linux,今天就教教大家怎麼在linux上安裝配置MongoDB資料庫 一、安裝步驟     1、下載及解壓     下載完成後用WinSCP複製到/us

linux環境go1.8安裝配置具體步驟二進位制部署原始碼編譯部署

一、直接使用二進位制安裝包部署說明:之前學習過go語言(大概是0.9版本),後來更新太快,也沒怎麼使用,就荒廢掉了,今年有專案需要用go開發,重新撿起。這是我在學習go語言過程中整理的內容,這裡記錄下,也方便我以後查閱。作業系統: CentOS 6.9_x64go語言版本:

解決CentOSUbuntuvirt-manager方格亂碼的問題

virt-manager顯示亂碼的解決方法,安裝相應字型: ubuntu下: sudo apt install font-manager sudo apt install fonts-arphic

CentOS的ip設定/Ubuntu的ip設定:在Oracle VM VirtualBOXCentOS 7/Ubuntu 系統設定網路:接入internet上網 區域網 ip

##################CentOS為手動設定靜態IP步驟############# 注:以下部分ip用"*",替代保密:1:設定ip OK後,檢視這些資訊[配置這些即可]: Netmask 子網掩碼,Gateway 預設閘道器,HostName 主機名稱,Do

windows安裝配置 MariaDB 資料庫

1. http://www.upupw.net/Apache/ 下載5.6版本2. 解壓後,執行upupw.exe. 鍵入s3,啟動資料庫。3. 資料庫使用者名稱和密碼 均為root。 posted on 2018-02-18 09:09 AlanTop 閱讀(

CentOS安裝配置nginx-rtmp-module同時支持IPv4IPv6

最好 ive install module to change str style 一個 arc   因測試需要配置IPv6環境下的rtmp流媒體服務器,想著直接使用docker鏡像,搜索對比後發現這個能滿足要求: https://hub.docker.com

Mac OS X安裝配置MySQL資料庫

最近需要在電腦上自己搭一個數據庫,由於在大學期間使用的是MySQL,也算是對它情有獨鍾吧,這次在Mac上依舊就使用MySQL吧,在這裡記錄一下安裝以及配置的過程(注:博文中使用的OS X版本為:10.11.4) MySQL下載 要想配置,當然需要安