參考文件：
1)德哥部落格：http://blog.163.com/[email protected]/blog/static/163877040201192535630895/
2)/wiki介紹：http://wiki.postgresql.org/wiki/PL/Proxy#Documentation（包含PLProxy下載地址，用法介紹，用法舉例）
3)德哥視訊：http://pan.baidu.com/share/link?shareid=5060&uk=201377085

遺留問題：
1）plproxy 建立Cluster 後可以執行在RUN ALL下，這個時候如果連結pgbouncer期中一個節點失效
那麼，執行在整個節點上的函式全部都會報錯，對於分散式資料庫需要從分庫查詢資料時就會影響
到可以正常工作的庫。
2）從各個文章中發現有介紹怎麼在代理庫上建立代理函式，查詢當前庫的版本和節點配置資訊以及
他們的連結引數有什麼意義。

PostgreSQL分散式設計: 三層結構 :  1. 路由層(幾乎無限擴充套件)    主角: plproxy,postgresql 2. 連線池層(幾乎無限擴充套件)    主角: pgbouncer 3. 資料層(幾乎無限擴充套件)    主角: postgresql 擴充套件方式: 1. 路由層擴充套件: 路由層包含了資料層的連線配置(FDW或函式),plproxy語言寫的函式殼(內建路由演算法),這些基本上是靜態資料,所以擴充套件非常方便. 新增伺服器就行了. 2. 連線池層擴充套件: 連線池層擴充套件加伺服器. 3. 資料層擴充套件: 資料層擴充套件,新增伺服器,通過流複製增加資料節點,結合路由演算法重分佈資料(建議路由演算法2^n取模), 物理分佈: 1. 路由層和連線池層儘量靠近部署.可以考慮部署在同一臺物理機. 2. 資料層儘量每個節點一臺物理機. 環境需求: CentOS

5.7 x64flex-2.5.35PostgreSQL-9.1.1plproxy-2.2pgfincore-v1.1libevent-1.4.14b-stablepgbouncer 1.4.2 測試環境描述: 1. pgbench :172.16.3.1762. pgbouncer on pgbench HOST :172.16.3.176:1998(proxy0 = host=172.16.3.150 dbname=proxy port=1921 pool_size=16proxy1= host=172.16.3.39 dbname=proxy port=1921 pool_size=16

proxy2= host=172.16.3.40 dbname=proxy port=1921 pool_size=16proxy3= host=172.16.3.33 dbname=proxy port=1921 pool_size=16)3.PostgreSQL資料節點:172.16.3.150:1921/digoal,172.16.3.39:1921/digoal,172.16.3.40:1921/digoal,172.16.3.33:1921/digoal4.PostgreSQL plproxy節點:172.16.3.150:1921/proxy,172.16.3.39:1921/proxy,172.16

.3.40:1921/proxy,172.16.3.33:1921/proxy5. pgbouncers on plproxy HOST :172.16.3.150:1999,172.16.3.39:1999,172.16.3.40:1999,172.16.3.33:1999(digoal0 = host=172.16.3.150 dbname=digoal port=1921 pool_size=8digoal1= host=172.16.3.39 dbname=digoal port=1921 pool_size=8digoal2= host=172.16.3.40 dbname=digoal port=1921 pool_size=8digoal3= host=172.16.3.33 dbname=digoal port=1921 pool_size=8) 環境搭建: 1. 編譯安裝flex-2.5.35

./configure && make && make install

2. 編譯安裝PostgreSQL-9.1.1 ./configure --prefix=/opt/pgsql --with-pgport=1921--with-perl --with-python --with-openssl --with-pam --without-ldap --with-libxml --with-libxslt --enable-thread-safety --with-wal-segsize=64gmake worldgmake install-world 3. 編譯安裝plproxy-2.2 mv plproxy-2.2 postgresql-9.1.1/contrib/plproxy-2.2make PG_CONFIG=/path/to/pg_configmake install PG_CONFIG=/path/to/pg_config 4. 編譯安裝pgfincore-v1.1（這個擴充套件可以把表放到作業系統的Cache中，提高效能） mv pgfincore-v1.1 postgresql-9.1.1/contrib/pgfincore-v1.1cp pgfincore.control /make cleanmakesumake install PG_CONFIG=/path/to/pg_config 5. 編譯安裝libevent-1.4.14b-stable

./configure && make && make install

6. 編譯安裝pgbouncer 1.4.2

./configure --prefix=/opt/pgbouncer && make && make install

注意：如果找不到相關的xxx.so檔案，請把該檔案的目錄放入檔案，最後執行/sbin/ldconfig –v命令
vim /etc/ld.so.conf include /etc/ld.so.conf.d/*.conf /usr/local/lib/

配置: 1. 配置資料節點資訊 新建使用者 : digoal(nosuperuser) 新建表空間 : digoal, digoal_idx 新建資料庫 : digoal digoal庫新建schema : digoal digoal庫新建過程語言 : plpgsql 允許代理函式連的連線池所在的伺服器連線上面新建的使用者和庫 : 配置pg_hba.conf 配置postgresql.conf : 略 2. 配置plproxy節點資訊(本例與資料節點共用PostgreSQL資料庫例項叢集) 新建使用者 : proxy(nosuperuser) 新建表空間 : 共用digoal 新建資料庫 : proxy proxy庫新建schema : proxy plproxy初始化 : 用超級使用者執行/opt/pgsql/share/contrib/plproxy.sql 建立handler function,language,validator function,foreign data wrapper 更改language可信度(否則普通使用者不可以使用plproxy語言) :    proxy=> \c proxy postgres  update pg_language set lanpltrusted='t'where lanname='plproxy'; 這個操作是為了途方便, 生產中請使用超級使用者建立plproxy函式, 把execute許可權賦予給普通使用者. 3. 配置pgfincore 連線到資料節點\c digoal postgres

CREATE EXTENSION pgfincore;

4. 配置pgbouncer(代理函式連的連線池) 4臺主機都需要配置, [email protected]-digoal-> cat config1999.ini [databases]digoal0 = host=172.16.3.150 dbname=digoal port=1921 pool_size=8digoal1= host=172.16.3.39 dbname=digoal port=1921 pool_size=8digoal2= host=172.16.3.40 dbname=digoal port=1921 pool_size=8digoal3= host=172.16.3.33 dbname=digoal port=1921 pool_size=8[pgbouncer]pool_mode = transactionlisten_port =1999unix_socket_dir =/opt/pgbouncer/etclisten_addr =*auth_type = md5auth_file =/opt/pgbouncer/etc/users1999.txtlogfile =/dev/nullpidfile =/opt/pgbouncer/etc/pgbouncer1999.pidmax_client_conn =10000reserve_pool_timeout =0server_reset_query =admin_users = pgbouncer_adminstats_users = pgbouncer_guestignore_startup_parameters = extra_float_digits[email protected]-digoal-> cat users1999.txt "digoal""md5462f71c79368ccf422f8a773ef40074d" 5. 配置pgbouncer(pgbench連的連線池) [email protected]-digoal-> cat config1998.ini [databases]proxy0 = host=172.16.3.150 dbname=proxy port=1921 pool_size=16proxy1= host=172.16.3.39 dbname=proxy port=1921 pool_size=16proxy2= host=172.16.3.40 dbname=proxy port=1921 pool_size=16proxy3= host=172.16.3.33 dbname=proxy port=1921 pool_size=16[pgbouncer]pool_mode = transactionlisten_port =1998unix_socket_dir =/opt/pgbouncer/configlisten_addr =*auth_type = md5auth_file =/opt/pgbouncer/config/users.txtlogfile =/dev/nullpidfile =/opt/pgbouncer/config/pgbouncer1998.pidmax_client_conn =1500reserve_pool_timeout =0server_reset_query =admin_users = pgbouncer_adminstats_users = pgbouncer_guestignore_startup_parameters = extra_float_digits 資料節點, 建立測試表, 插入測試資料: proxy=# \c digoal digoalcreate table user_info(userid int,engname text,cnname text,occupation text,birthday date,signname text,email text,qq numeric,crt_time timestamp without time zone,mod_time timestamp without time zone);create table user_login_rec(userid int,login_time timestamp without time zone,ip inet);create table user_logout_rec(userid int,logout_time timestamp without time zone,ip inet); 測試資料 :  0號節點 insert into user_info (userid,engname,cnname,occupation,birthday,signname,email,qq,crt_time,mod_time)select generate_series(0,50000000,4),'digoal.zhou','德哥','DBA','1970-01-01',E'公益是一輩子的事, I\'m Digoal.Zhou, Just do it!','[email protected]',276732431,clock_timestamp(),NULL; 1號節點 insert into user_info (userid,engname,cnname,occupation,birthday,signname,email,qq,crt_time,mod_time)select generate_series(1,50000000,4),'digoal.zhou','德哥','DBA','1970-01-01',E'公益是一輩子的事, I\'m Digoal.Zhou, Just do it!','[email protected]',276732431,clock_timestamp(),NULL; 2號節點 insert into user_info (userid,engname,cnname,occupation,birthday,signname,email,qq,crt_time,mod_time)select generate_series(2,50000000,4),'digoal.zhou','德哥','DBA','1970-01-01',E'公益是一輩子的事, I\'m Digoal.Zhou, Just do it!','[email protected]',276732431,clock_timestamp(),NULL; 3號節點 insert into user_info (userid,engname,cnname,occupation,birthday,signname,email,qq,crt_time,mod_time)select generate_series(3,50000000,4),'digoal.zhou','德哥','DBA','1970-01-01',E'公益是一輩子的事, I\'m Digoal.Zhou, Just do it!','[email protected]',276732431,clock_timestamp(),NULL; 所有節點 :  set work_mem='2048MB';set maintenance_work_mem='2048MB';alter table user_info add constraint pk_user_info primary key (userid)using index tablespace digoal_idx; 開發:  資料節點 :  實體函式:  登入函式 :  create or replace function f_user_login (i_userid int,OUT o_userid int,OUT o_engname text,OUT o_cnname text,OUT o_occupation text,OUT o_birthday date,OUT o_signname text,OUT o_email text,OUT o_qq numeric)as $BODY$declarebeginselect userid,engname,cnname,occupation,birthday,signname,email,qqinto o_userid,o_engname,o_cnname,o_occupation,o_birthday,o_signname,o_email,o_qqfrom user_info where userid=i_userid;insert into user_login_rec (userid,login_time,ip) values (i_userid,now(),inet_client_addr());return;end;$BODY$language plpgsql; 退出函式 :  create or replace function f_user_logout(i_userid int,OUT o_result int)as $BODY$declarebegininsert into user_logout_rec (userid,logout_time,ip) values (i_userid,now(),inet_client_addr());o_result :=0;return;exception when others theno_result :=1;return;end;$BODY$language plpgsql; 代理節點 :  建立server CREATE SERVER digoal FOREIGN DATA WRAPPER plproxyOPTIONS (connection_lifetime '1800',disable_binary '1',p0 'dbname=digoal0 host=127.0.0.1 port=1999 client_encoding=UTF8',p1 'dbname=digoal1 host=127.0.0.1 port=1999 client_encoding=UTF8',p2 'dbname=digoal2 host=127.0.0.1 port=1999 client_encoding=UTF8',p3 'dbname=digoal3 host=127.0.0.1 port=1999 client_encoding=UTF8'); 建立user mapping CREATE USER MAPPING FOR proxy SERVER digoalOPTIONS (user 'digoal', password 'digoal'); 賦權server

grant usage on foreign server digoal to proxy;

建立代理函式: \c proxy proxy登入函式:CREATE OR REPLACE FUNCTION f_user_login(i_userid int,OUT o_userid int,OUT o_engname text,OUT o_cnname text,OUT o_occupation text,OUT o_birthday date,OUT o_signname text,OUT o_email text,OUT o_qq numeric)as $BODY$    CLUSTER 'digoal';    RUN ON i_userid;    target digoal.f_user_login;$BODY$LANGUAGE plproxy;退出函式:create or replace function f_user_logout(i_userid int,OUT o_result int)as $BODY$    CLUSTER 'digoal';    RUN ON i_userid;    target digoal.f_user_logout;$BODY$language plproxy; pgbench壓力測試 :  [email protected]-digoal-> cat begin.sh#!/bin/bashnohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/login.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy0 >>./login_0.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/login.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy1 >>./login_1.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/login.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy2 >>./login_2.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/login.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy3 >>./login_3.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/logout.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy0 >>./logout_0.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/logout.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy1 >>./logout_1.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/logout.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy2 >>./logout_2.log 2>&1 &nohup pgbench -M extended -r -c 8 -f /home/postgres/digoal_bench/logout.sql -j 8 -n -T 180 -h 127.0.0.1 -p 1998 -U proxy proxy3 >>./logout_3.log 2>&1 &[email protected]> cat login.sql\setrandom userid 0 50000000SELECT f_user_login(:userid);[email protected]> cat logout.sql\setrandom userid 0 50000000SELECT f_user_logout(:userid); cat .pgpass 略 測試結果 :  [email protected]-digoal-> cat login_0.log transaction type:Custom queryscaling factor:1query mode: extendednumber of clients:8number of threads:8duration:180 snumber of transactions actually processed:665468tps=3695.624216(including connections establishing)tps =3695.675102(excluding connections establishing)statement latencies in milliseconds:0.002366        \setrandom userid 0500000002.158355        SELECT f_user_login(:userid);[email protected]-digoal-> cat login_1.log transaction type:Custom queryscaling factor:1query mode: extendednumber of clients:8number of threads:8duration:180 snumber of transactions actually processed:665288tps=3694.720318(including connections establishing)tps =3694.777428(excluding connections establishing)statement latencies in milliseconds:0.002289        \setrandom userid 0500000002.159063        SELECT f_user_login(:userid);[email protected]-digoal-> cat login_2.log transaction type:Custom queryscaling factor:1query mode: extendednumber of clients:8number of threads:8duration:180 snumber of transactions actually processed:645371tps=3585.275832(including connections establishing)tps =3585.340161(excluding connections establishing)statement latencies in milliseconds:0.002341        \setrandom userid 0500000002.225684        SELECT f_user_login(:userid);