1. 程式人生 > 實用技巧 >postgres之通過 python和shell指令碼遷移資料

postgres之通過 python和shell指令碼遷移資料

說明:通過python3每次讀取檔案的前三行傳遞到shell指令碼執行遷移任務

指令碼如下

python指令碼

[root@node-251 aws-uat-prep]# cat pg-move.py 
#!/usr/bin/python3
import sys,os

with open('pg.txt','r') as f:
    content=f.readlines()

content=[content[i].strip('\n') for i in range(len(content))]
a=0
b=3
while content[a:b]:
      result=content[a:b]
      result
=' '.join(result) content[a:b]='' os.system("/tmp/aws-uat-prep/pg-mv.sh {}".format(result))

shell指令碼:通過配置/root/.pgpass來讓psql不用手動輸入密碼執行資料遷入

[root@node-251 aws-uat-prep]# cat pg-mv.sh 
#!/bin/bash
#將cdp_edge上面schema的資料複製一份到cdp_edge_prep對應的schema上面 cd
/tmp/aws-uat-prep rm -f *.sql schema="cml" #postgres的schema
yy=$* #接收python傳過來的字串,也就是三個表 for i in $yy;do echo $i echo $i >>info.txt pg_dump "host=cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com port=5432 user=dbadmin password=G-PsscYdl1qmNh2bw|9X dbname=cdp_edge" -t ${schema}.$i --no-tablespaces -f ${i}.sql echo
"alter table ${schema}.${i} owner to cdp_2c" >> ${i}.sql sed -i "/CREATE TABLE/i\ drop table if exists ${schema}.${i};" ${i}.sql #sed -i "s/public\./${schema}\./g" ${i}.sql #sed -i 's/OWNER TO dbadmin/OWNER TO cdp_2c/g' ${i}.sql done # for i in `ls *.sql`;do psql --host=cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com --port=5432 --username=dbadmin --dbname=cdp_edge_prep -f $i;done for i in `ls *.sql`;do psql -h cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com -p 5432 cdp_edge_prep dbadmin -f $i;done

免密執行操作

[root@node-251 aws-uat-prep]# cat ~/.pgpass 
cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com:5432:cdp_edge_prep:dbadmin:密碼

上面配置後下面的命令可以免密執行

psql -h cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com -p 5432 cdp_edge_prep dbadmin  -f 檔案 ##免密執行命令
psql -h cdpsupportquotation-uat-pg.cojemua46qv2.us-east-1.rds.amazonaws.com -p 5432 cdp_edge_prep dbadmin ## 登陸

注意使用特殊符號的處理比如冒號: htZul;+0NNb\:UK12Ip

其他