Spark轉GemFire任務(一)
ADMG-40 2.2.2.4 - Global Tax Warehouse (GTW) CoA mapping -- Both BtB and Project 1
Revision 7/15, change all latam to mdg
Create GFS account mapped to GTW using FAGL_011ZC, write to /tax/gfs_gtw_mapping
and go through /btb_mdg/ska1 to check all 10 digit GL account can be mapped, if not, write to exception region /tax/master_data_exception
Use BTB_MDG.SKA1 (if BTB_MDG is not ingested, use BTB_LATAM.SKA1):
- For every record of SKA1 (select every record from SKA1 where KTOPL = ‘JNJG’ and XSPEB <> ‘X’ ) -- collection A
collectionA:
Path: /btb_mdg/ska1
Filter: KTOPL = ‘JNJG’ and XSPEB <> ‘X’
Column: saknr
自己本身
- select records in BTB_MDG.FAGL_011ZC (where VERSN=’GTW’ AND KTOPL=’JNJG’) – collection B
collectionB:
Path: /btb_mdg/fagl_011zc
Filter: trim(versn) = ‘GTW’ and ktopl = ‘JNJG’
Column: from vonkt
To biskt
自己本身的
- match each A.SAKNR with [B. VONKT , B. BISKT], B.VONKT is the starting number, B. BISKT is the ending number, both numbers are included in the range, when a match is found,
Validate wheather each item in collectionA in the range of collectionB
判斷是否 A包含於B
4. Get gfsCode (A.SAKNR) and gtwCode (B.ERGSL) description:
for gfsCode -- /btb_mdg/skat: spras:E AND ktopl: JNJG AND saknr:%A.SAKNR%, let it be D, D.txt50 is the gfsCode description.
gtwCode -- /btb_mdg/fagl_011qt : versn:GTW AND spras:E AND ergsl:%gtwCode%, let it be E, E.txt45 is the gtwCode description
gfsCode:
Select txt50 as gfsCodeDescription from /btb_mdg/skat where spras = ‘E’ and ktopl = ‘JNJG’ and saknr in collectionA.saknr
和A的saknr有關系
gtwCode:
Select txt45 as gtwCodeDescription from /btb_mdg/fagl_011qt where trim(versn) = ‘GTW’ and spras = ‘E’ and ergsl in collectionB.ergsl
和B的ergsl有關系
- save it into /tax/gfs_gtw_mapping: <gfs:A.SAKNR, gfs_description:D.txt50, gtw:B.ERGSL, gtw_description:E.txt45>
Insert into /tax/gfs_gtw_mapping(
Gfs : collectionA.saknr,
Gfs_description : txt50,
Gtw : collectionB.ergsl,
Gtw_Description : txt45
)
Validation:
3.1 If there is no match, write exceptionCode:INVALID_GTW_MAPPING, excepitonMessage: No Match on GTW Code for GFS code % A.SAKNR%
items in CollectionA but not in the range of CollectionB will be written into Exception
exceptionCode : INVALID_GTW_MAPPING
exceptionMessage : No Match on GTW Code for GFS code % A.SAKNR%
all [B. VONKT , B. BISKT] should not have overlap, check all [B. VONKT , B. BISKT] do not overlap by sorting B. VONKT and B. BISKT and make sure B.VONKT(n+1) < B. BISKT (n), ), if there is invalid range, exceptionCode: INVALID_GFS_MAPPING, exceptionMessage overlapping range is found, [B(n). valfrom, B(n).valto] and [B(n+1). valfrom, B(n+1).valto],
All the ranges in CollectionB cannot have overlap,if it does,written into Exception
exceptionCode: INVALID_GFS_MAPPING
exceptionMessage overlapping range is found, [B(n). valfrom, B(n).valto] and [B(n+1). valfrom, B(n+1).valto]
6. Saving exception:
Exception should be saved to /tax/master_data_exception
{
sourceSystem: btb_mdg,
exceptionCode:
exceptionMessage:
timestamp
}
Spark轉GemFire任務(一)