OGG19版本源端新增欄位,目標端複製程序不報錯，使用MAPALLCOLUMNS進行測試
阿新 • • 發佈：2021-08-04
一、需求，OGG19版本源端新增欄位,目標端複製程序不報錯,而是直接跳過這個欄位進行同步資料了
現在客戶提了一個需求，能否讓OGG在不同步DDL的情況下,源端新增欄位後，目標端OGG複製程序abend提示表結構不一致！
找了半天，找到一個可能相關的OGG引數，接下來使用這個引數進行測試！！！MAPALLCOLUMNS引數
著急結果的朋友們直接看此處總結： 無論何種辦法，源端新增欄位，目標端沒有同步這個欄位的情況下,OGG會自動跳過！因此需要規範化操作！！！
Oracle優先保障資料同步的實時性
1. 正常OGG鏈路同步測試,正常
2.使用MAPALLCOLUMNS引數表級別生效,模擬源端新增欄位測試
3.對新增欄位重新add trandata測試,新增欄位在目標端直接忽略跳過,OGG不報錯
4.對測試表級別使用NOMAPALLCOLUMNS引數,無效果,OGG不報錯
5.對複製程序層面使用MAPALLCOLUMNS引數,測試新增欄位,OGG目標端不報錯;
6.對複製程序層面使用MAPALLCOLUMNS引數 + assumetargetdefs引數,測試新增欄位,OGG目標端不報錯
備註說明：本次測試源端、目標端環境為同資料庫型別，如果是Oracle->Mysql 有同事在引數使用MAPALLCOLUMNS，如果未手工同步DDL，即使OGG版本19一樣abend!!!
二、測試步驟
2.1. 正常OGG鏈路同步測試,正常
EXT_GBK SETENV (ORACLE_SID = "gbkt1") EXTTRAIL /u01/ogg/base/dirdat/ext_gbk/cc TABLE YZ.TEST00A1; TABLE YZ.TEST00A2; SQL> desc YZ.TEST00A2 Name Null? Type ----------------------------------------- -------- ---------------------------- OWNER VARCHAR2(30) OBJECT_NAME VARCHAR2(128) SUBOBJECT_NAME VARCHAR2(30) OBJECT_ID NUMBER DATA_OBJECT_ID NUMBER OBJECT_TYPE VARCHAR2(19) CREATED DATE LAST_DDL_TIME DATE TIMESTAMP VARCHAR2(19) STATUS VARCHAR2(7) TEMPORARY VARCHAR2(1) GENERATED VARCHAR2(1) SECONDARY VARCHAR2(1) NAMESPACE NUMBER EDITION_NAME VARCHAR2(30) GGSCI (t1) 52> info d_gbk_a,detail Trail Name Seqno RBA Max MB Trail Type /u01/ogg/base/dirdat/gbk/aa 9 19439384 200 RMTTRAIL Extract Source Begin End /u01/ogg/base/dirdat/ext_gbk/cc000000021 2021-07-01 11:04 2021-06-13 05:04 GGSCI (t2) 10> info rep_gbk,detail Log Read Checkpoint File /u01/ogg/base/dirdat/gbk/aa000000009 map yz.TEST00A1 ,target bak_yz.TEST00A1; map yz.TEST00A2 ,target bak_yz.TEST00A2,MAPALLCOLUMNS;
--測試環境效能太差,從整合抽取降級為經典模式抽取 https://blog.csdn.net/zzt_2009/article/details/105581190 EXTRACT RUNNING EXT_GBK 00:03:11 00:00:03 Log Read Checkpoint Oracle Integrated Redo Logs GGSCI (t1 as ogg@gbkt1) 200> dblogin USERID ogg,PASSWORD OGG Successfully logged into database. GGSCI (t1 as ogg@gbkt1) 201> info EXTRACT EXT_GBK downgrade ERROR: Extract EXT_GBK is not ready to be downgraded because recovery SCN values are not set. SQL> select current_scn from v$database; CURRENT_SCN ----------- 4063221 >stop ext_gbk >alter EXTRACT EXT_GBK,scn 4063221 > info EXTRACT EXT_GBK downgrade Extract EXT_GBK is ready to be downgraded from integrated capture. Archive logs corresponding to SCN 4063221
and higher must be accessible by the downgraded extract. >ALTER EXTRACT EXT_GBK DOWNGRADE INTEGRATED TRANLOG Extract EXT_GBK successfully downgraded from integrated capture. > start ext_gbk
--測試資料一致 select count(*) from yz.test00a1; COUNT(*) ---------- 86581 select count(*) from yz.test00a2; select count(*) from bak_yz.test00a1; select count(*) from bak_yz.test00a2; create public database link gbk_db_link1 connect to SYSTEM identified by oracle using '10.0.0.31:1521/gbkt1'; create table bak_yz.TEST00A1 as select * from yz.test00a1@gbk_db_link1; create table bak_yz.TEST00A2 as select * from yz.test00a1@gbk_db_link1; -OGG TEST delete yz.test00a1 where rownum=1; delete yz.test00a2 where rownum=1; commit; Target 資料正常同步！！！
2.2.使用MAPALLCOLUMNS引數表級別生效,模擬源端新增欄位測試
測試 Source 新增欄位 alter table yz.test00a1 add test001 varchar2(20); SQL> insert into yz.test00a1(OBJECT_ID,TEST001) values(6666666,'qwer'); SQL> commit; select OBJECT_ID,TEST001 from yz.TEST00A1 where object_id=6666666; OBJECT_ID TEST001 ---------- -------------------- 6666666 qwer target select OBJECT_ID from bak_yz.TEST00A1 where object_id=6666666; OBJECT_ID ---------- 6666666 --另一個表！ alter table yz.test00a2 add test001 varchar2(20); insert into yz.test00a2(OBJECT_ID,TEST001) values(6666666,'qwer'); commit; select OBJECT_ID,TEST001 from yz.TEST00A2 where object_id=6666666; OBJECT_ID TEST001 ---------- -------------------- 6666666 qwer Target SQL> select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666666; OBJECT_ID ---------- 6666666 --無報錯！！！
2.3.對新增欄位重新add trandata測試,新增欄位在目標端直接忽略跳過,OGG不報錯
---Source 對錶重新add trandata GGSCI (t1 as ogg@gbkt1) 256> delete trandata yz.test00a2 GGSCI (t1 as ogg@gbkt1) 257> add trandata yz.test00a2 GGSCI (t1 as ogg@gbkt1) 258> info trandata yz.TEST00A2 Columns supplementally logged for table YZ.TEST00A1: "CREATED", "DATA_OBJECT_ID", "EDITION_NAME", "GENERATED",
"LAST_DDL_TIME", "NAMESPACE", "OBJECT_ID", "OBJECT_NAME", "OBJECT_TYPE", "OWNER", "SECONDARY", "STATUS",
"SUBOBJECT_NAME", "TEMPORARY", "TIMESTAMP". Columns supplementally logged for table YZ.TEST00A1: "CREATED", "DATA_OBJECT_ID", "EDITION_NAME", "GENERATED",
"LAST_DDL_TIME", "NAMESPACE", "OBJECT_ID", "OBJECT_NAME", "OBJECT_TYPE", "OWNER", "SECONDARY", "STATUS",
"SUBOBJECT_NAME", "TEMPORARY", "TEST001", "TIMESTAMP". Source 再次插入測試 insert into yz.test00a1(OBJECT_ID,TEST001) values(6666667,'qwer'); commit; select OBJECT_ID,TEST001 from yz.TEST00A1 where object_id=6666667; Target select OBJECT_ID from bak_yz.TEST00A1 where object_id=6666667; OBJECT_ID ---------- 6666667 insert into yz.test00a2(OBJECT_ID,TEST001) values(6666667,'qwer'); commit; select OBJECT_ID,TEST001 from yz.TEST00A2 where object_id=6666667; Target select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666667; OBJECT_ID ---------- 6666667 不報錯！！！
2.4.對測試表級別使用NOMAPALLCOLUMNS引數,無效果,OGG不報錯
--換一個引數 NOMAPALLCOLUMNS -- map yz.TEST00A2 ,target bak_yz.TEST00A2,noMAPALLCOLUMNS; 再次測試 alter table yz.test00a2 add test002 varchar2(20); insert into yz.test00a2(OBJECT_ID,TEST002) values(6666668,'qwer'); commit; select OBJECT_ID,TEST002 from yz.TEST00A2 where object_id=6666668; OBJECT_ID TEST002 ---------- -------------------- 6666668 qwer Target select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666668; OBJECT_ID ---------- 6666668 NOMAPALLCOLUMNS source insert into yz.test00a2(OBJECT_ID,TEST002) values(6666669,'qwer'); commit; target select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666669; OBJECT_ID ---------- 6666669
2.5.對複製程序層面使用MAPALLCOLUMNS引數,測試新增欄位,OGG目標端不報錯;
--引數級別 map yz.TEST00A2 ,target bak_yz.TEST00A2; MAPALLCOLUMNS map yz.TEST00A1 ,target bak_yz.TEST00A1; Source 新增欄位 alter table yz.test00a1 add test001 varchar2(20); SQL> insert into yz.test00a1(OBJECT_ID,TEST001) values(6666666,'qwer'); SQL> commit; select OBJECT_ID,TEST001 from yz.TEST00A1 where object_id=6666666; OBJECT_ID TEST001 ---------- -------------------- 6666666 qwer target select OBJECT_ID from bak_yz.TEST00A1 where object_id=6666666; OBJECT_ID ---------- 6666666 --另一個表！ alter table yz.test00a2 add test001 varchar2(20); insert into yz.test00a2(OBJECT_ID,TEST001) values(6666666,'qwer'); commit; select OBJECT_ID,TEST001 from yz.TEST00A2 where object_id=6666666; OBJECT_ID TEST001 ---------- -------------------- 6666666 qwer Target SQL> select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666666; OBJECT_ID ---------- 6666666 --無報錯！！！
2.6.對複製程序層面使用MAPALLCOLUMNS引數 + assumetargetdefs引數,測試新增欄位,OGG目標端不報錯
!!! 使用引數！ GGSCI (t2) 16> stop rep_gbk assumetargetdefs GGSCI (t2) 18> start rep_gbk 再次測試 Source 新增欄位 alter table yz.test00a1 add test002 varchar2(20); insert into yz.test00a1(OBJECT_ID,TEST002) values(6666666,'qwer'); commit; select OBJECT_ID,TEST002 from yz.TEST00A1 where object_id=6666666; OBJECT_ID TEST002 ---------- -------------------- 6666666 6666666 qwer target select OBJECT_ID from bak_yz.TEST00A1 where object_id=6666666; OBJECT_ID ---------- 6666666 6666666 --另一個表！ alter table yz.test00a2 add test002 varchar2(20); insert into yz.test00a2(OBJECT_ID,TEST002) values(6666666,'qwer'); commit; select OBJECT_ID,TEST002 from yz.TEST00A2 where object_id=6666666; OBJECT_ID TEST002 ---------- -------------------- 6666666 6666666 qwer Target SQL> select OBJECT_ID from bak_yz.TEST00A2 where object_id=6666666; OBJECT_ID ---------- 6666666 6666666