oracle goldengate for downstreams 配置--for 資料庫引數配置和ogg程序引數配置
1.整體資料庫配置
三個源端和一個downstream下游抽取資料庫配置:
alter system set enable_goldengate_replication=TRUE;
Alter database add supplemental log data;
Alter database add supplemental log data(primary key,unique index) columns;
Alter database add supplemental log data(all) columns;
Alter database force logging;
select SUPPLEMENTAL_LOG_DATA_MIN,SUPPLEMENTAL_LOG_DATA_PK,SUPPLEMENTAL_LOG_DATA_UI,SUPPLEMENTAL_LOG_DATA_FK,SUPPLEMENTAL_LOG_DATA_ALL,force_logging
from v$database;
2.SOURCE DB1、2、3端資料庫引數調整和配置:
1.db1,db2,db3三個庫建立表空間和使用者
create tablespace ogg datafile ‘xxxxx.dbf’ size 1g autoextend on next 1g;
alter tablespace ogg add datafile ‘xxxx.dbf’ size 1g autoextend on next 1g;
create user ogg account identified by oracle ;
2.授權:
begin
dbms_goldengate_auth.grant_admin_privilege(grantee=>'OGG',
GRANT_SELECT_PRIVILEGES=>TRUE,
PRIVILEGE_TYPE=>'CAPTURE',
DO_GRANTS=>TRUE);
END;
/
EXEC DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE('OGG');
EXEC DBMS_GOLDENGATE_AUTH.GRANT_ADMIN_PRIVILEGE('OGG');
GRANT SELECT ON V_$DATABASE TO OGG ;
GRANT EXECUTE ON OGG.UPDATESEQUENCE TO OGG;
GRANT EXECUTE ON OGG.REPLICATESEQUENCE TO OGG;
GRANT SELECT ON SYS.SEQ$ TO OGG;
GRANT SELECT ON SYS.USER$ TO OGG;
GRANT SELECT ON SYS.OBJ$ TO OGG;
GRANT SELECT ON DBA_SEQUENCES TO OGG;
GRANT CONNECT, RESOURCE TO OGG;
GRANT CREATE SESSION, ALTER SESSION TO OGG;
GRANT SELECT ANY DICTIONARY, SELECT ANY TABLE TO OGG;
GRANT ALTER ANY TABLE TO OGG;
GRANT FLASHBACK ANY TABLE TO OGG;
GRANT EXECUTE ON DBMS_FLASHBACK TO OGG;
GRANT SELECT ON DBA_CLUSTERS TO OGG;
GRANT SELECT ANY TRANSACTION TO OGG;
一般我都是直接給:
grant connect,resource,dba to ogg;
3.建表:
create table ogg.t1 as select OBJECT_ID,OWNER,OBJECT_NAME,CREATED from dba_segments ;
create table ogg.t2 (
pid number(20) primary key not null ,
product varchar2(40),
location varchar2(40),
produce varchar2(40)
);
--暫時不插入資料,後面ogg程序配置完畢後插入
begin
for v_count in 1..100000000 loop
insert into ogg.t2 values(v_count,'rose','hangzhou10','zhejiang');
commit;
end loop;
end;
/
4.配置tns連線資訊三個sourceDB和一個下游抽取db-mid都要配置
dba1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.147.249.104)(PORT = 21521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dba1)
)
)
dba2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.147.249.104)(PORT = 21521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dba2)
)
)
dba3 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.147.249.104)(PORT = 21521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = dba3)
)
)
mid =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 10.147.249.105)(PORT = 21521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = mid)
)
)
5.設定本地歸檔路徑和配置日誌傳輸引數
dba1:
ALTER SYSTEM SET LOG_ARCHIVE_DEST_1='LOCATION=/u01/arch1/ VALID_FOR=(ONLINE_LOGFILE, PRIMARY_ROLE)';
ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dba1,mid)';
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_1=ENABLE ;
設定service 歸檔傳到目標端路徑位置:
目標端建立3個目錄,分別存放db1 db2 db3的歸檔日誌:
alter system set log_archive_dest_5='SERVICE=mid ASYNC OPTIONAL NOREGISTER TEMPLATE=/u01/arch_dba1/dba1_arch_%t_%s_%r.dbf VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) REOPEN=300 DB_UNIQUE_NAME=mid'
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_5=ENABLE;
dba2:
ALTER SYSTEM SET LOG_ARCHIVE_DEST_1='LOCATION=/u02/arch2/ VALID_FOR=(ONLINE_LOGFILE, PRIMARY_ROLE)';
ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dba2,mid)';
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_1=ENABLE ;
設定service目標端歸檔存放位置:
alter system set log_archive_dest_6='SERVICE=mid ASYNC OPTIONAL NOREGISTER TEMPLATE=/u01/arch_dba2/dba2_arch_%t_%s_%r.dbs VALID_FOR=(ONLINE_LOGFILES,PRIMARY_ROLE) REOPEN=300 DB_UNIQUE_NAME=mid'
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_6=ENABLE;
dba3:
ALTER SYSTEM SET LOG_ARCHIVE_DEST_1='LOCATION=/u03/arch3/ VALID_FOR=(ONLINE_LOGFILE, PRIMARY_ROLE)';
ALTER SYSTEM SET LOG_ARCHIVE_CONFIG='DG_CONFIG=(dba3,mid)';
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_1=ENABLE ;
設定service目標端歸檔日誌存放位置:
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_7=ENABLE;
附錄TNS配置檔案截圖
:
3.downstream DB-mid端資料庫引數調整和配置:
1.ogg使用者建立和授權完成;
2.新增standby_redo(standby redo配置和DG (n+1)*(thread) 原則)
alter database add standby logfile group 11 '/u01/app/oracle/oradata/MID/standby_redo11.log' size 200m;
alter database add standby logfile group 12 '/u01/app/oracle/oradata/MID/standby_redo12.log' size 200m;
alter database add standby logfile group 13 '/u01/app/oracle/oradata/MID/standby_redo13.log' size 200m;
alter database add standby logfile group 14 '/u01/app/oracle/oradata/MID/standby_redo14.log' size 200m;
3.配置arch standby_redo歸檔位置
配置本庫歸檔日誌即(mid-db)
下游抽取庫配置:
alter system set log_archive_config='DG_CONFIG=(dba1,dba2,dba3,mid)';
本身歸檔位置:
alter system set log_archive_dest_1='LOCATION=/u01/arch_mid/ VALID_FOR=(ONLINE_LOGFILE,PRIMARY_ROLE)' DB_UNIQUE_NAME=mid;
配置downstream_real_time_mine Y的standby_redo日誌即接收過來的歸檔日誌
alter system set log_archive_dest_5='LOCATION=/u01/arch_dba1/ VALID_FOR=(STANDBY_LOGFILE,PRIMARY_ROLE)';
ALTER SYSTEM SET LOG_ARCHIVE_DEST_STATE_5=ENABLE;
置於dba2 dba3的日誌位置為什麼不用配置?因為在源端的時候我們已經指定了日誌傳輸位置。
那麼dba1同時也指定了日誌傳輸位置,為啥還要配置呢?downstream_real_time_mine Y可以實時解析standby_redo,也就是說dba1的redo會以dg的形式傳輸到mid下游庫來解析的,所以standby_redo歸檔到/u01/arch_dba1目錄下。
注意:三個源端---日誌傳輸到mid db下游庫做日誌分析,是利用了dg的特性,
但是隻能有一個downstream_real_time_mine Y
其他兩個downstream_real_time_mine N且必須置於N
在這裡我們讓dba1是Y,dba2、dba3是N配置OGG程序。
4.downstream-架構的密碼檔案配置:
密碼檔案配置:
dba1生成密碼檔案:
cd $ORACLE_HOME/dbs
orapwd file=orapwdba1 password=Golden_1230 force=y sys=y
cp orapwdba1 orapwdba2
cp orapwdba1 orapwdba3
cp orapwdba1 mid
密碼檔案保證這幾個庫密碼一致,日誌傳輸正常進行。---至此為止日誌傳輸算是正常進行了!!!
5.downstream-架構-install goldengate software
OK goldengate software install finished !!!
6.downstream-架構-configre goldengate extract
6.1編輯MGR引數檔案
PORT 7809
DYNAMICPORTLIST 7810-7900
AUTORESTART EXTRACT * ,WAITMINUTES 2, RETRIES 10
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45
PURGEOLDEXTRACTS /ogg/dirdat/*,USECHECKPOINTS, minkeephours 2
啟動mgr程序:
start mgr
6.2編輯extract for dba1的抽取程序extdb1引數檔案
--------配置整合抽取模式
1.編輯extract程序引數檔案;
EXTRACT extdb1
USERID ogg@dba1, PASSWORD oracle
TRANLOGOPTIONS mininguser ogg@mid miningpassword oracle
TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine Y)
TRANLOGOPTIONS integratedparams ( max_sga_size 2048 ) -- , parallelism 4)
cachemgr cachesize 2g
-- tranlogoptions _NOREADAHEAD ANY
getupdatebefores
GETTRUNCATES
REPORTCOUNT EVERY 15 MINUTES, RATE
numfiles 5000
DISCARDFILE ./dirrpt/extdb1.dsc,APPEND,MEGABYTES 1024
WARNLONGTRANS 2h,CHECKINTERVAL 10m
EXTTRAIL ./dirdat/a1
FETCHOPTIONS NOUSESNAPSHOT
DBOPTIONS ALLOWUNUSEDCOLUMN
DYNAMICRESOLUTION
TABLE OGG.T1;
TABLE OGG.T2;
2.登入資料庫和mid下游抽取庫;
dblogin USERID ogg@dba1, PASSWORD oracle
miningdblogin userid ogg@mid ,PASSWORD oracle
3.在資料庫中註冊extract抽取程序;
register extract extdb1 database
4.新增整合抽取程序,tranlog begin now;
add extract extdb1 integrated tranlog begin now
5.分配程序組使用的trail檔案;
add exttrail ./dirdat/a1 extract extdb1 megabytes 1024
6.啟動extdb1抽取程序;
start extdb1
-------非整合抽取模式-即經典抽取模式classical抽取模式
編輯引數檔案ext11
EXTRACT ext11
USERID ogg@dba1, PASSWORD oracle
--TRANLOGOPTIONS mininguser ogg@mid miningpassword oracle
--TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine Y)
--TRANLOGOPTIONS integratedparams ( max_sga_size 2048 ) -- , parallelism 4)
cachemgr cachesize 2g
-- tranlogoptions _NOREADAHEAD ANY
getupdatebefores
GETTRUNCATES
REPORTCOUNT EVERY 15 MINUTES, RATE
numfiles 5000
DISCARDFILE ./dirrpt/ext11.dsc,APPEND,MEGABYTES 1024
WARNLONGTRANS 2h,CHECKINTERVAL 10m
EXTTRAIL ./dirdat/zz
FETCHOPTIONS NOUSESNAPSHOT
DBOPTIONS ALLOWUNUSEDCOLUMN
--DYNAMICRESOLUTION
tranlogoptions dblogreader
TABLE OGG.T1;
TABLE OGG.T2;
新增ext11抽取程序,並分配./dirdat/zz trail檔案給ext11程序組使用:
GGSCI (qhdb2.zpepc.com.cn as ogg@dba1) 160> add extract ext11 ,tranlog ,begin now
EXTRACT added.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba1) 161> add exttrail ./dirdat/zz ,extract ext11 ,megabytes 2000
EXTTRAIL added.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba1) 162> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING EXT1 00:07:09 00:00:02
EXTRACT STOPPED EXT11 00:00:00 00:00:22
啟動經典模式抽取程序:
start ext11
6.3編輯extract for dba1的抽取程序ext2引數檔案
編輯ext2引數檔案:
GGSCI (qhdb2.zpepc.com.cn as ogg@dba2) 13> view param ext2
EXTRACT ext2
USERID ogg@dba2, PASSWORD oracle
TRANLOGOPTIONS mininguser ogg@mid miningpassword oracle
TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine N)
TRANLOGOPTIONS integratedparams ( max_sga_size 2048 ) -- , parallelism 4)
cachemgr cachesize 2g
-- tranlogoptions _NOREADAHEAD ANY
getupdatebefores
GETTRUNCATES
REPORTCOUNT EVERY 15 MINUTES, RATE
numfiles 5000
DISCARDFILE ./dirrpt/ext2.dsc,APPEND,MEGABYTES 1024
WARNLONGTRANS 2h,CHECKINTERVAL 10m
EXTTRAIL ./dirdat/t2
FETCHOPTIONS NOUSESNAPSHOT
DBOPTIONS ALLOWUNUSEDCOLUMN
--DYNAMICRESOLUTION
--tranlogoptions dblogreader
TABLE OGG.T1;
TABLE OGG.T2;
註冊extract -ext2程序到資料庫:
GGSCI (qhdb2.zpepc.com.cn as ogg@dba2) 14> register extract ext2 database
2021-07-08 16:34:54 INFO OGG-15418 Usage of template clause for log archive destination '6' will cause redo transport to only send archived redo to downstream.
2021-07-08 16:35:06 INFO OGG-02003 Extract EXT2 successfully registered with database at SCN 139768734.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba2) 15> add extract ext2 integrated tranlog begin now
EXTRACT (Integrated) added.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba2) 16> add exttrail ./dirdat/t2 extract ext2 megabytes 2000
EXTTRAIL added.
6.2編輯extract for dba1的抽取程序ext3引數檔案
編輯ext3引數檔案
GGSCI (qhdb2.zpepc.com.cn) 1> dblogin USERID ogg@dba3, PASSWORD oracle
Successfully logged into database.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba3) 2> miningdblogin userid ogg@mid ,PASSWORD oracle
Successfully logged into mining database.
EXTRACT ext3
USERID ogg@dba3, PASSWORD oracle
TRANLOGOPTIONS mininguser ogg@mid miningpassword oracle
TRANLOGOPTIONS INTEGRATEDPARAMS (downstream_real_time_mine N)
TRANLOGOPTIONS integratedparams ( max_sga_size 2048 ) -- , parallelism 4)
cachemgr cachesize 2g
-- tranlogoptions _NOREADAHEAD ANY
getupdatebefores
GETTRUNCATES
REPORTCOUNT EVERY 15 MINUTES, RATE
numfiles 5000
DISCARDFILE ./dirrpt/ext3.dsc,APPEND,MEGABYTES 1024
WARNLONGTRANS 2h,CHECKINTERVAL 10m
EXTTRAIL ./dirdat/t3
FETCHOPTIONS NOUSESNAPSHOT
DBOPTIONS ALLOWUNUSEDCOLUMN
--DYNAMICRESOLUTION
--tranlogoptions dblogreader
TABLE OGG.T1;
TABLE OGG.T2;
註冊ext3到資料庫:
GGSCI (qhdb2.zpepc.com.cn as ogg@dba3) 5> register extract ext3 database
2021-07-08 16:40:09 INFO OGG-15418 Usage of template clause for log archive destination '7' will cause redo transport to only send archived redo to downstream.
2021-07-08 16:40:18 INFO OGG-02003 Extract EXT3 successfully registered with database at SCN 139427050.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba3) 6> add extract ext3 integrated tranlog begin now
EXTRACT (Integrated) added.
GGSCI (qhdb2.zpepc.com.cn as ogg@dba3) 7> add exttrail ./dirdat/t3 ,extract ext3 megabytes 1000
EXTTRAIL added.
檢視dirdat目錄下trail檔案產生的情況:
7.downstream-架構-configre goldengate datapump
配置datapump投遞程序
配置datapump投遞程序 for ext1
編輯引數檔案dpet1
EXTRACT dpet1
PASSTHRU
RMTHOST 127.0.0.1, MGRPORT 7809, compress
RMTTRAIL ./dirdat/local/t1 -- , FORMAT RELEASE 12.2
TABLE OGG.T1;
TABLE OGG.T2;
新增datapump投遞程序:
新增dpet1投遞程序
GGSCI (qhdb2.zpepc.com.cn as ogg@dba1) 7>
add extract dpet1 ,exttrailsource ./dirdat/aa
EXTRACT added.
新增遠端trail檔案(備註:goldengate目標端的trail檔案是在源端定義的,所以在源端新增目標端的trail檔案)
GGSCI (qhdb2.zpepc.com.cn as ogg@dba1) 8>
add RMTTRAIL ./dirdat/local/t1 ,extract dpet1 ,megabytes 2000
RMTTRAIL added.
啟動dpet1投遞程序:
./dirdat/local/t1xxxxx trail檔案生成!
8.downstream-架構-configre goldengate replicat
配置replicat複製程序
replicat程序有兩種模式:
1.傳統的replicat複製;
編輯全域性引數檔案:
GGSCI (qhdb2.zpepc.com.cn) 2> edit param ./GLOBALS
GGSCHEMA ogg
checkpointtable ogg.ckpt
新增checkpointtable
GGSCI (qhdb2.zpepc.com.cn) 4> dblogin USERID ogg@tagdb,PASSWORD oracle
Successfully logged into database.
GGSCI (qhdb2.zpepc.com.cn as ogg@mid) 5> add checkpointtable ogg.ckpt
Successfully created checkpoint table ogg.ckpt.
新增replicat 程序 - rept1
REPLICAT rept1
--setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
--setenv (ORACLE_SID='mid')
--setenv (ORACLE_HOME='/u01/app/oracle/product/19.3.0/db_home1')
--setenv (ORA_TZFILE='/u01/app/oracle/product/19.3.0/db_home1/oracore/zoneinfo/timezone_32.dat')
USERID ogg@tagdb, PASSWORD oracle
SQLEXEC 'ALTER SESSION SET CONSTRAINTS=DEFERRED'
REPORT AT 01:59
REPORTCOUNT EVERY 30 MINUTES, RATE
REPERROR DEFAULT, ABEND
--reperror -1 , DISCARD
--reperror 1403 , DISCARD
--ASSUMETARGETDEFS
MaxDiscardRecs 200
--HANDLECOLLISIONSM-#M-(
--batchsql BATCHESPERQUEUE 100, OPSPERBATCH 8000
--GROUPTRANSOPS 2000
--MAXTRANSOPS 500
assumetargetdefs
--sourcedefs ./dirdef/xxxx.def
DISCARDFILE ./dirrpt/rept1.dsc, purge, MEGABYTES 1024
DISCARDROLLOVER AT 02:30 on sunday
GETTRUNCATES
ALLOWNOOPUPDATES
--sourcecharset passthru
MAP OGG.t1 , target ogg.t1 ;
----filter ( @getenv('TRANSACTION','CSN') > 15756503460690 ) ;
新增rept1程序:
add replicat rept1 ,exttrail ./dirdat/local/t1 checkpointtable ogg.ckpt
2.integrated replicat 複製;(不需要checkpointtable)
add replicat rept1 integrated exttrail ./dirdat/local/t1
-------新增完畢後,start * 啟動程序即可。!!!