分析ADG網卡流量出現規律性毛刺現象
阿新 • • 發佈：2018-01-26
ror 1.2 napi 自動 ram 分鐘 ogr Owner ras 一、介紹
客戶ADG環境，剛上線的業務系統，通過ZABBIX進行監控，發現每半個小時會出現ADG網卡流量超過1M，持續2分鐘後，平均網絡流量在100K左右。
客戶ADG環境，剛上線的業務系統，通過ZABBIX進行監控，發現每半個小時會出現ADG網卡流量超過1M，持續2分鐘後，平均網絡流量在100K左右。
二、分析
1、ADG 環境備庫流量主要來自主庫redo的數據傳輸，或者出現網絡問題，導致歸檔無法傳輸，恢復後歸檔日誌的傳輸。
2、查看歸檔日誌產生的頻率及大小，檢查告警日誌是否有歸檔日誌傳輸。
3、經過排查並未發現歸檔日誌傳輸。檢查數據庫job,發現並沒有客戶創建的job。
4、檢查系統自帶的Job，dba_scheduler_jobs,我們會發現很多系統自帶的job.
5、數據庫自身自動寫入的數據，而且有定時計劃的可能會導致這個問題。
6、查看awr的寫入的頻率發現是每半個小時執行一次。
以下是測試環境的的awrinfo.sql結果，大家需要熟悉了解。
SQL> @?/rdbms/admin/awrinfo.sql This script will report general AWR information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Specify the Report File Name ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The default report file name is awrinfo.txt. To use this name, press <return> to continue, otherwise enter an alternative. Enter value for report_name: Using the report name awrinfo.txt No errors. No errors. ~~~~~~~~~~~~~~~ AWR INFO Report ~~~~~~~~~~~~~~~ Report generated at 11:06:42 on Jan 26, 2018 ( Friday ) in Timezone +08:00 Warning: Non Default AWR Setting! -------------------------------------------------------------------------------- Snapshot interval is 60 minutes and Retention is 8 days DB_ID DB_NAME HOST_PLATFORM INST STARTUP_TIME LAST_ASH_SID PAR ------------ --------- ---------------------------------------- ----- ----------------- ------------ --- * 1486195868 ORCL roidb01 - Linux x86 64-bit 1 09:30:24 (01/26) 32052 NO ######################################################## (I) AWR Snapshots Information ######################################################## ***************************************************** (1a) SYSAUX usage - Schema breakdown (dba_segments) ***************************************************** | | Total SYSAUX size 660.9 MB ( 2% of 32,768.0 MB MAX with AUTOEXTEND ON ) | | Schema APEX_040200 occupies 247.6 MB ( 37.5% ) | Schema SYS occupies 200.1 MB ( 30.3% ) | Schema MDSYS occupies 93.8 MB ( 14.2% ) | Schema XDB occupies 68.5 MB ( 10.4% ) | Schema ORDDATA occupies 16.1 MB ( 2.4% ) | Schema SYSTEM occupies 15.8 MB ( 2.4% ) | Schema WMSYS occupies 7.2 MB ( 1.1% ) | Schema DVSYS occupies 4.4 MB ( 0.7% ) | Schema CTXSYS occupies 3.7 MB ( 0.6% ) | Schema AUDSYS occupies 1.5 MB ( 0.2% ) | Schema GSMADMIN_INT occupies 1.4 MB ( 0.2% ) | Schema DBSNMP occupies 0.5 MB ( 0.1% ) | Schema ORDSYS occupies 0.4 MB ( 0.1% ) | ******************************************************** (1b) SYSAUX occupants space usage (v$sysaux_occupants) ******************************************************** | | Occupant Name Schema Name Space Usage | -------------------- -------------------- ---------------- | SDO MDSYS 93.8 MB | XDB XDB 68.5 MB | SM/OTHER SYS 53.9 MB | SM/AWR SYS 40.4 MB | AO SYS 38.8 MB | XSOQHIST SYS 38.8 MB | ORDIM/ORDDATA ORDDATA 16.1 MB | LOGMNR SYSTEM 13.9 MB | SM/OPTSTAT SYS 11.4 MB | SM/ADVISOR SYS 8.4 MB | WM WMSYS 7.2 MB | TEXT CTXSYS 3.7 MB | SQL_MANAGEMENT_BASE SYS 2.4 MB | PL/SCOPE SYS 1.6 MB | AUDSYS AUDSYS 1.5 MB | LOGSTDBY SYSTEM 1.5 MB | STREAMS SYS 1.0 MB | JOB_SCHEDULER SYS 0.8 MB | EM_MONITORING_USER DBSNMP 0.5 MB | ORDIM ORDSYS 0.4 MB | SMON_SCN_TIME SYS 0.4 MB | AUTO_TASK SYS 0.3 MB | AUDIT_TABLES SYS 0.0 MB | EM SYSMAN 0.0 MB | EXPRESSION_FILTER EXFSYS 0.0 MB | ORDIM/ORDPLUGINS ORDPLUGINS 0.0 MB | ORDIM/SI_INFORMTN_SC SI_INFORMTN_SCHEMA 0.0 MB | STATSPACK PERFSTAT 0.0 MB | TSM TSMSYS 0.0 MB | ULTRASEARCH WKSYS 0.0 MB | ULTRASEARCH_DEMO_USE WK_TEST 0.0 MB | XSAMD OLAPSYS 0.0 MB | | Others (Unaccounted space) 255.8 MB | ****************************************** (1c) SYSAUX usage - Unregistered Schemas ****************************************** | This section displays schemas that are not registered | in V$SYSAUX_OCCUPANTS | | Schema APEX_040200 occupies 247.6 MB | Schema DVSYS occupies 4.4 MB | Schema GSMADMIN_INT occupies 1.4 MB | | Total space 253.4 MB | ************************************************************* (1d) SYSAUX usage - Unaccounted space in registered schemas ************************************************************* | | This section displays unaccounted space in the registered | schemas of V$SYSAUX_OCCUPANTS. | | Unaccounted space in SYS/SYSTEM 2.4 MB | | Total space 2.4 MB | ************************************* (2) Size estimates for AWR snapshots ************************************* | | Estimates based on 60 mins snapshot INTERVAL: | AWR size/day 485.3 MB (20,704 K/snap * 24 snaps/day) | AWR size/wk 3,396.8 MB (size_per_day * 7) per instance | | Estimates based on 2 snaps in past 24 hours: | AWR size/day 606.6 MB (20,704 K/snap and 2 snaps in past 1.6 hours) | AWR size/wk 4,245.9 MB (size_per_day * 7) per instance | ********************************** (3a) Space usage by AWR components (per database) ********************************** COMPONENT MB % AWR KB_PER_SNAP MB_PER_DAY MB_PER_WEEK TABLE% : INDEX% --------- --------- ------ ------------ ---------- ----------- ---------------- FIXED 25.6 63.4 13,120 384.4 2,690.6 48% : 52% SPACE 3.1 7.6 1,568 45.9 321.6 51% : 49% SQLPLAN 2.4 6.0 1,248 36.6 255.9 82% : 18% EVENTS 2.3 5.6 1,152 33.8 236.3 47% : 53% SQL 1.8 4.5 928 27.2 190.3 52% : 48% RAC 0.6 1.5 320 9.4 65.6 50% : 50% SQLTEXT 0.4 0.9 192 5.6 39.4 83% : 17% ASH 0.4 0.9 192 5.6 39.4 50% : 50% SQLBIND 0.3 0.6 128 3.8 26.3 50% : 50% ********************************** (3b) Space usage within AWR Components (> 500K) ********************************** COMPONENT MB SEGMENT_NAME - % SPACE_USED SEGMENT_TYPE --------- --------- --------------------------------------------------------------------- --------------- FIXED 2.0 WRH$_SYSSTAT_PK.WRH$_SYSSTA_1486195868_0 - 66% INDEX PARTITION FIXED 0.9 WRH$_LATCH.WRH$_LATCH_1486195868_0 - 80% TABLE PARTITION FIXED 0.8 WRH$_LATCH_PK.WRH$_LATCH_1486195868_0 - 72% INDEX PARTITION FIXED 0.8 WRH$_SYSMETRIC_HISTORY_INDEX - 26% INDEX FIXED 0.8 WRH$_SYSSTAT.WRH$_SYSSTA_1486195868_0 - 85% TABLE PARTITION FIXED 0.6 WRH$_SYSMETRIC_HISTORY - 19% TABLE SQLPLAN 2.0 WRH$_SQL_PLAN - 67% TABLE SQL 0.5 WRH$_SQLSTAT.WRH$_SQLSTA_1486195868_0 - 81% TABLE PARTITION ********************************** (4) Space usage by non-AWR components (> 500K) ********************************** COMPONENT MB SEGMENT_NAME SEGMENT_TYPE --------- --------- --------------------------------------------------------------------- --------------- NON_AWR 51.2 MDSYS.SYS_LOB0000083702C00006$$ LOBSEGMENT NON_AWR 42.2 SYS.SYS_LOB0000006421C00005$$ LOBSEGMENT NON_AWR 23.2 APEX_040200.SYS_LOB0000088338C00007$$ LOBSEGMENT NON_AWR 19.0 APEX_040200.WWV_FLOW_PAGE_PLUGS TABLE NON_AWR 17.2 SYS.SYS_LOB0000085557C00004$$ LOBSEGMENT NON_AWR 11.0 APEX_040200.WWV_FLOW_STEP_ITEMS TABLE NON_AWR 10.0 APEX_040200.WWV_FLOW_TEMPLATES TABLE NON_AWR 9.2 APEX_040200.SYS_LOB0000088569C00009$$ LOBSEGMENT NON_AWR 9.0 MDSYS.SDO_CS_SRS TABLE NON_AWR 8.1 APEX_040200.SYS_LOB0000087756C00016$$ LOBSEGMENT NON_AWR 8.0 SYS.HEATMAP SYSTEM STATISTI CS NON_AWR 7.2 SYS.SYS_LOB0000085542C00004$$ LOBSEGMENT NON_AWR 6.0 APEX_040200.WWV_FLOW_STEP_PROCESSING TABLE NON_AWR 5.2 APEX_040200.SYS_LOB0000087810C00010$$ LOBSEGMENT NON_AWR 5.1 SYS.SYS_LOB0000006421C00004$$ LOBSEGMENT NON_AWR 4.0 SYSTEM.SYS_LOB0000001340C00009$$ LOBSEGMENT NON_AWR 4.0 APEX_040200.WWV_FLOW_REGION_REPORT_COLUMN TABLE NON_AWR 4.0 APEX_040200.WWV_FLOW_STEPS TABLE NON_AWR 3.2 SYS.SYS_LOB0000085537C00004$$ LOBSEGMENT NON_AWR 3.2 APEX_040200.SYS_LOB0000088421C00019$$ LOBSEGMENT NON_AWR 3.2 SYS.SYS_LOB0000085547C00004$$ LOBSEGMENT NON_AWR 3.0 APEX_040200.WWV_FLOW_STEP_BUTTONS TABLE NON_AWR 3.0 APEX_040200.WWV_FLOW_WORKSHEET_COLUMNS TABLE NON_AWR 3.0 APEX_040200.WWV_FLOW_INSTALL_SCRIPTS TABLE NON_AWR 3.0 APEX_040200.WWV_FLOW_PAGE_PLUG_TEMPLATES TABLE NON_AWR 2.2 SYS.SYS_LOB0000085552C00004$$ LOBSEGMENT NON_AWR 2.2 APEX_040200.SYS_LOB0000088585C00007$$ LOBSEGMENT NON_AWR 2.2 APEX_040200.SYS_LOB0000087738C00022$$ LOBSEGMENT NON_AWR 2.2 APEX_040200.SYS_LOB0000087738C00028$$ LOBSEGMENT NON_AWR 2.2 APEX_040200.SYS_LOB0000087667C00011$$ LOBSEGMENT NON_AWR 2.2 SYS.SYS_LOB0000085562C00004$$ LOBSEGMENT NON_AWR 2.1 SYS.SYS_LOB0000008312C00038$$ LOBSEGMENT NON_AWR 2.0 SYS.I_WRI$_OPTSTAT_HH_OBJ_ICOL_ST INDEX NON_AWR 2.0 APEX_040200.WWV_FLOW_WORKSHEETS TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_FLASH_MAP_REGIONS TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_LIST_ITEMS TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_LIST_TEMPLATES TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_ROW_TEMPLATES TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_STEP_VALIDATIONS TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_STEP_ITEM_HELP TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_STEP_ITEMS_U_NAME INDEX NON_AWR 2.0 APEX_040200.WWV_FLOW_STEP_BRANCHES TABLE NON_AWR 2.0 APEX_040200.WWV_FLOW_PAGE_PLUGS_IDX_U1 INDEX NON_AWR 2.0 APEX_040200.WWV_FLOW_PAGE_PLUGS_IDX3 INDEX NON_AWR 2.0 MDSYS.SDO_COORD_REF_SYS TABLE COMPONENT MB SEGMENT_NAME SEGMENT_TYPE --------- --------- --------------------------------------------------------------------- --------------- NON_AWR 2.0 XDB.XDB$ELEMENT TABLE NON_AWR 1.2 SYS.SYS_LOB0000008304C00004$$ LOBSEGMENT NON_AWR 1.2 AUDSYS.SYS_LOB0000091784C00014$$.SYS_LOB_P186 LOB PARTITION NON_AWR 1.2 APEX_040200.SYS_LOB0000089973C00010$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000088446C00006$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000088432C00007$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000088387C00088$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000088332C00021$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000088326C00005$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000087738C00025$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000087700C00007$$ LOBSEGMENT NON_AWR 1.2 APEX_040200.SYS_LOB0000087667C00024$$ LOBSEGMENT NON_AWR 1.2 MDSYS.SYS_LOB0000086565C00003$$ LOBSEGMENT NON_AWR 1.2 MDSYS.SYS_LOB0000084268C00003$$ LOBSEGMENT NON_AWR 1.2 ORDDATA.SYS_LOB0000077885C00005$$ LOBSEGMENT NON_AWR 1.2 ORDDATA.SYS_LOB0000077702C00005$$ LOBSEGMENT NON_AWR 1.2 XDB.SYS_LOB0000019097C00025$$ LOBSEGMENT NON_AWR 1.2 XDB.SYS_LOB0000019043C00023$$ LOBSEGMENT NON_AWR 1.2 SYS.SYS_LOB0000006450C00009$$.SYS_LOB_P228 LOB PARTITION NON_AWR 1.1 SYS.SYS_LOB0000010986C00001$$ LOBSEGMENT NON_AWR 1.1 APEX_040200.SYS_LOB0000088559C00009$$ LOBSEGMENT NON_AWR 1.1 APEX_040200.SYS_LOB0000087999C00013$$ LOBSEGMENT NON_AWR 1.1 SYS.SYS_LOB0000006450C00009$$.SYS_LOB_P353 LOB PARTITION NON_AWR 1.0 SYS.WRI$_OPTSTAT_OPR_TASKS TABLE NON_AWR 1.0 APEX_040200.WWV_FLOW_MESSAGES$ TABLE NON_AWR 1.0 SYS.SYS$SERVICE_METRICS_TAB TABLE NON_AWR 1.0 SYS.I_WRI$_OPTSTAT_H_OBJ#_ICOL#_ST INDEX NON_AWR 1.0 SYS.LTXID_TRANS.LTXID_TRANS_1 TABLE PARTITION NON_AWR 0.9 SYS.I_WRI$_OPTSTAT_HH_ST INDEX NON_AWR 0.9 APEX_040200.WWV_FLOW_STEP_ITEM_IDX3 INDEX NON_AWR 0.9 APEX_040200.WWV_FLOW_LIST_OF_VALUES_DATA TABLE NON_AWR 0.9 APEX_040200.WWV_FLOW_PAGE_PLUGS_PK INDEX NON_AWR 0.9 APEX_040200.WWV_FLOW_CAL_TEMPLATES TABLE NON_AWR 0.8 APEX_040200.WWV_FLOW_STEP_ITEMS_PK INDEX NON_AWR 0.8 APEX_040200.WWV_FLOW_PAGE_PLUGS_FK INDEX NON_AWR 0.8 APEX_040200.WWV_FLOW_STEP_ITEM_IDX4 INDEX NON_AWR 0.8 APEX_040200.WWV_FLOW_STEP_BUTTONS_IDX2 INDEX NON_AWR 0.8 APEX_040200.WWV_FLOW_STEP_BUTTONS_FKIDX INDEX NON_AWR 0.8 APEX_040200.WWV_FLOW_STEP_ITEM_IDX1 INDEX NON_AWR 0.7 APEX_040200.WWV_FLOW_PAGE_PLUGS_IDX1 INDEX NON_AWR 0.7 APEX_040200.WWV_FLOW_STEP_ITEM_IDX5 INDEX NON_AWR 0.7 APEX_040200.WWV_FLOW_STEP_ITEM_IDX2 INDEX NON_AWR 0.6 SYS.I_WRI$_OPTSTAT_OPR_TASKS_TGST INDEX NON_AWR 0.6 APEX_040200.WWV_FLOW_PAGE_DA_ACTIONS TABLE NON_AWR 0.6 APEX_040200.WWV_FLOW_WORKSHEET_COLUMNS_UK2 INDEX NON_AWR 0.6 APEX_040200.WWV_FLOW_MENU_OPTIONS TABLE NON_AWR 0.6 APEX_040200.WWV_FLOW_STEP_COMPUTATIONS TABLE COMPONENT MB SEGMENT_NAME SEGMENT_TYPE --------- --------- --------------------------------------------------------------------- --------------- NON_AWR 0.6 APEX_040200.WWV_FLOW_PAGE_PLUGS_IDX2 INDEX NON_AWR 0.6 APEX_040200.WWV_FLOW_LISTS_OF_VALUES$ TABLE NON_AWR 0.6 MDSYS.SDO_COORD_OPS TABLE NON_AWR 0.6 SYS.AW_OBJ$ TABLE NON_AWR 0.6 XDB.XDB$COMPLEX_TYPE TABLE NON_AWR 0.6 MDSYS.SDO_COORD_OP_PARAM_VALS TABLE NON_AWR 0.6 MDSYS.COORD_OP_PARA_VAL_PRIM INDEX NON_AWR 0.6 APEX_040200.WWV_FLOW_LOV_DATA_IDX1 INDEX ********************************** (5a) AWR snapshots - last 50 ********************************** Total snapshots in DB 1486195868 Instance 1 = 2 DBID SNAP_ID INST FLUSH_ELAPSED ENDTM STARTUP_TIME STATUS ERRCNT ---------- ---------- ----- -------------------- ----------------- ----------------- ------ ------ 1486195868 15 1 +00000 00:00:00.6 09:41:13 (01/26) 09:30:24 (01/26) 0 0 1486195868 16 1 +00000 00:00:00.3 11:00:18 (01/26) 09:30:24 (01/26) 0 0 ********************************** (5b) AWR snapshots with errors or invalid ********************************** no rows selected ********************************** (5c) AWR snapshots -- OLDEST Non-Baselined snapshots ********************************** DBID INST SNAP_ID ENDTM STATUS ERROR_COUNT ---------- ----- ---------- ----------------- ------ ----------- 1486195868 1 15 09:41:13 (01/26) 0 0 ********************************** (6) AWR Control Settings - interval, retention ********************************** DBID LSNAPID LSPLITID LSNAPTIME LPURGETIME FLAG INTERVAL RETENTION VRSN ----------- -------- -------- -------------- -------------- ----- ----------------- ----------------- ---- 1486195868 16 0 01/26 11:00:19 01/26 10:00:11 2 +00000 01:00:00.0 +00008 00:00:00.0 11 ********************************** (7a) AWR Contents - row counts for each snapshots ********************************** SNAP_ID INST ASH SQL SQBND FILES SEGST SYSEVT ---------- ----- ---------- ---------- ---------- ---------- ---------- ---------- 15 1 4 89 190 4 65 89 16 1 12 96 243 4 66 90 ********************************** (7b) AWR Contents - average row counts per snapshot ********************************** SNAP_COUNT INST ASH SQLSTAT SQLBIND FILES SEGSTAT SYSEVENT ---------- ----- ---------- ---------- ---------- ---------- ---------- ---------- 2 1 8 92.5 216.5 4 65.5 89.5 ********************************** (7c) AWR total item counts - names, text, plans ********************************** SQLTEXT SQLPLAN SQLBMETA SEGOBJ DATAFILE TEMPFILE ---------- ---------- ---------- ---------- ---------- ---------- 326 4983 1232 206 4 1 ######################################################## (II) Advisor Framework Info ######################################################## ********************************** (1) Advisor Tasks - Last 50 ********************************** OWNER/ADVISOR TASK_ID/NAME CREATED EXE_DURATN EXE_CREATN HOW_C STATUS -------------- -------------------------------- ---------------- ---------- ---------- ----- ------------ SYS/SQL Tuning 1/SYS_AUTO_SQL_TUNING_TASK 05:52:52 (07/07) AUTO SYS/SPM Evolve 2/SYS_AUTO_SPM_EVOLVE_TASK 05:52:52 (07/07) AUTO SYS/ADDM 42/ADDM:1486195868_1_16 11:00:19 (01/26) 0 0 AUTO COMPLETED ********************************** (2) Advisor Task - Oldest 5 ********************************** OWNER/ADVISOR TASK_ID/NAME CREATED EXE_DURATN EXE_CREATN HOW_C STATUS -------------- -------------------------------- ---------------- ---------- ---------- ----- ------------ SYS/SQL Tuning 1/SYS_AUTO_SQL_TUNING_TASK 05:52:52 (07/07) AUTO SYS/SPM Evolve 2/SYS_AUTO_SPM_EVOLVE_TASK 05:52:52 (07/07) AUTO SYS/ADDM 42/ADDM:1486195868_1_16 11:00:19 (01/26) 0 0 AUTO COMPLETED ********************************** (3) Advisor Tasks With Errors - Last 50 ********************************** no rows selected ######################################################## (III) ASH Usage Info ######################################################## ********************************** (1a) ASH histogram (past 3 days) ********************************** NUM_ACTIVE_SESSIONS NUM_SAMPLES -------------------- ------------ 0000 - 0004 12 ********************************** (1b) ASH histogram (past 1 day) ********************************** NUM_ACTIVE_SESSIONS NUM_SAMPLES -------------------- ------------ 0000 - 0004 12 ********************************** (2a) ASH details (past 3 days) ********************************** INST MIN_TIME MAX_TIME NUM_SAMPLES NUM_ROWS AVG_ACTIVE ---- ---------------- ---------------- ------------ ------------ ---------- 1 09:41:09 (01/26) 10:45:00 (01/26) 383 12 0.03 ********************************** (2b) ASH details (past 1 day) ********************************** INST MIN_TIME MAX_TIME NUM_SAMPLES NUM_ROWS AVG_ACTIVE ---- ---------------- ---------------- ------------ ------------ ---------- 1 09:41:09 (01/26) 10:45:00 (01/26) 383 12 0.03 ********************************** (2c) ASH sessions (Fg Vs Bg) (past 1 day across all instances in RAC) ********************************** Foreground % 8.3 Background % 91.7 MMNL % 0.0 End of Report Report written to awrinfo.txt
分析ADG網卡流量出現規律性毛刺現象