1. 程式人生 > >記憶體分配統計檢視 | 全方位認識 sys 系統庫

記憶體分配統計檢視 | 全方位認識 sys 系統庫

在上一篇《按 file 分組統計檢視 | 全方位認識 sys 系統庫》中,我們介紹了sys 系統庫中按 file 分組統計的檢視,本期的內容將為大家介紹記憶體事件和innodb buffer pool記憶體分配的統計檢視。下面請跟隨我們一起開始 sys 系統庫的系統學習之旅吧。

PS:

  • 對innodb buffer pool的統計檢視對資料庫的效能影響較大(可能會導致效能陡降),它主要是提供給專業DBA人員做問題分析排查使用,一般情況下不要隨意使用

  • 對innodb buffer pool的統計檢視資料來源於information_schema系統庫,考慮到大家可能有MySQL 5.7之前的版本中使用需求,所以本文中特意列出了對innodb buffer pool的統計檢視的select語句文字

01.innodb_buffer_stats_by_schema,x$innodb_buffer_stats_by_schema

按照schema分組的 InnoDB buffer pool統計資訊,預設按照分配的buffer size大小降序排序--allocated欄位。資料來源:information_schema.innodb_buffer_page

檢視select語句文字如下:

# 不帶x$字首的檢視select語句文字
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  sys.format_bytes(SUM(IF(ibp.compressed_size = 0, 16384, compressed_size))) AS allocated,
  sys.format_bytes(SUM(ibp.data_size)) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(SUM(ibp.number_records)/COUNT(DISTINCT ibp.index_name)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

# 帶x$字首的檢視select語句文字
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) AS allocated,
  SUM(ibp.data_size) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(IFNULL(SUM(ibp.number_records)/NULLIF(COUNT(DISTINCT ibp.index_name), 0), 0)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 06:15:41> select * from innodb_buffer_stats_by_schema;
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| object_schema | allocated  | data      | pages | pages_hashed | pages_old | rows_cached |
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| InnoDB System | 23.73 MiB  | 21.76 MiB |  1519 |            0 |        24 |      21474 |
| mysql        | 240.00 KiB | 14.57 KiB |    15 |            0 |        15 |        179 |
| xiaoboluo    | 128.00 KiB | 38.93 KiB |    8 |            0 |        5 |        982 |
| sys          | 16.00 KiB  | 354 bytes |    1 |            0 |        1 |          6 |
| 小蘿蔔        | 16.00 KiB  | 135 bytes |    1 |            0 |        1 |          3 |
+---------------+------------+-----------+-------+--------------+-----------+-------------+
5 rows in set (0.43 sec)

# 帶x$字首的檢視
[email protected]
: sys 06:15:54> select * from x$innodb_buffer_stats_by_schema; +---------------+-----------+----------+-------+--------------+-----------+-------------+ | object_schema | allocated | data | pages | pages_hashed | pages_old | rows_cached | +---------------+-----------+----------+-------+--------------+-----------+-------------+ | InnoDB System | 24887296 | 22809628 | 1519 | 0 | 24 | 21498 | | mysql | 245760 | 14917 | 15 | 0 | 15 | 179 | | xiaoboluo | 131072 | 39865 | 8 | 0 | 5 | 982 | | sys | 16384 | 354 | 1 | 0 | 1 | 6 | | 小蘿蔔 | 16384 | 135 | 1 | 0 | 1 | 3 | +---------------+-----------+----------+-------+--------------+-----------+-------------+ 5 rows in set (0.42 sec)

 檢視欄位含義如下:

  • object_schema:schema級別物件的名稱,如果該表屬於Innodb儲存引擎,則該欄位顯示為InnoDB System,如果是其他引擎,則該欄位顯示為每個schema name.

  • allocated:當前已分配給schema的總記憶體位元組數

  • data:當前已分配給schema的資料部分使用的記憶體位元組總數

  • pages:當前已分配給schema記憶體總頁數

  • pages_hashed:當前已分配給schema的自適應hash索引頁總數

  • pages_old:當前已分配給schema的舊頁總數(位於LRU列表中的舊塊子列表中的頁數)

  • rows_cached:buffer pool中為schema緩衝的總資料行數

02.innodb_buffer_stats_by_table,x$innodb_buffer_stats_by_table

按照schema和表分組的 InnoDB buffer pool 統計資訊,與sys.innodb_buffer_stats_by_schema檢視類似,但是本檢視是按照schema name和table name分組。資料來源:information_schema.innodb_buffer_page

檢視select語句文字如下:

# 不帶x$字首的檢視select語句文字
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', -1), '`', '') AS object_name,
  sys.format_bytes(SUM(IF(ibp.compressed_size = 0, 16384, compressed_size))) AS allocated,
  sys.format_bytes(SUM(ibp.data_size)) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(SUM(ibp.number_records)/COUNT(DISTINCT ibp.index_name)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema, object_name
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

# 帶x$字首的檢視select語句文字
SELECT IF(LOCATE('.', ibp.table_name) = 0, 'InnoDB System', REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', 1), '`', '')) AS object_schema,
  REPLACE(SUBSTRING_INDEX(ibp.table_name, '.', -1), '`', '') AS object_name,
  SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) AS allocated,
  SUM(ibp.data_size) AS data,
  COUNT(ibp.page_number) AS pages,
  COUNT(IF(ibp.is_hashed = 'YES', 1, NULL)) AS pages_hashed,
  COUNT(IF(ibp.is_old = 'YES', 1, NULL)) AS pages_old,
  ROUND(IFNULL(SUM(ibp.number_records)/NULLIF(COUNT(DISTINCT ibp.index_name), 0), 0)) AS rows_cached
FROM information_schema.innodb_buffer_page ibp
WHERE table_name IS NOT NULL
GROUP BY object_schema, object_name
ORDER BY SUM(IF(ibp.compressed_size = 0, 16384, compressed_size)) DESC;

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 12:41:25> select * from innodb_buffer_stats_by_table limit 3;
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
| object_schema | object_name | allocated | data      | pages | pages_hashed | pages_old | rows_cached |
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
| InnoDB System | SYS_TABLES  | 11.58 MiB | 10.63 MiB |  741 |            0 |        3 |      36692 |
| luoxiaobo    | t_luoxiaobo | 80.00 KiB | 29.21 KiB |    5 |            0 |        0 |        1658 |
| InnoDB System | SYS_COLUMNS | 48.00 KiB | 16.03 KiB |    3 |            0 |        3 |        239 |
+---------------+-------------+-----------+-----------+-------+--------------+-----------+-------------+
3 rows in set (0.12 sec)

# 帶x$字首的檢視
[email protected] : sys 12:41:41> select * from x$innodb_buffer_stats_by_table limit 3;
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
| object_schema | object_name | allocated | data    | pages | pages_hashed | pages_old | rows_cached |
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
| InnoDB System | SYS_TABLES  |  12140544 | 11154757 |  741 |            0 |        3 |      36702 |
| luoxiaobo    | t_luoxiaobo |    81920 |    29913 |    5 |            0 |        0 |        1658 |
| InnoDB System | SYS_COLUMNS |    49152 |    16412 |    3 |            0 |        3 |        239 |
+---------------+-------------+-----------+----------+-------+--------------+-----------+-------------+
3 rows in set (0.12 sec)

檢視欄位含義如下:

  • object_name:表級別物件名稱,通常是表名

  • 其他欄位含義與sys.innodb_buffer_stats_by_schema檢視欄位含義相同,詳見 innodb_buffer_stats_by_schema,x$innodb_buffer_stats_by_schema檢視解釋部分。但這些欄位是按照object_name表級別統計的

03.memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes

按照客戶端主機名分組的記憶體使用統計資訊,預設情況下按照當前記憶體使用量降序排序,資料來源:performance_schema.memory_summary_by_host_by_event_name

* 開啟所有的memory型別的instruments:update setup_instruments set enabled='yes' where name like '%memory/%';

  • memory型別的事件預設情況下只啟用了performance_schema自身的instruments,要監控使用者訪問,需要單獨配置,如下: 

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 10:01:37> select * from memory_by_host_by_current_bytes limit 3;
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| host        | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| 10.10.20.14 |              58256 | 35.83 MiB        | 645 bytes        | 14.20 MiB        | 2.36 GiB        |
| localhost  |                32 | 903.11 KiB        | 28.22 KiB        | 819.00 KiB        | 7.69 MiB        |
| background  |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 352.57 KiB      |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

# 帶x$字首的檢視
[email protected] : sys 10:02:19> select * from x$memory_by_host_by_current_bytes limit 3;
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| host        | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| 10.10.20.14 |              58256 |          37569266 |          644.8995 |          14885584 |      2538394110 |
| localhost  |                18 |            891658 |        49536.5556 |            838656 |        9821551 |
| background  |                  5 |              176 |          35.2000 |              160 |          361068 |
+-------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.00 sec)

檢視欄位含義如下:

  • host:客戶端連線的主機名或IP。在Performance Schema表中的HOST列為NULL的行在這裡假定為後臺執行緒,且在該檢視host列顯示為background

  • current_count_used:當前已分配的且未釋放的記憶體塊對應的記憶體分配次數(記憶體事件呼叫次數,該欄位是快捷值,來自:performance_schema.memory_summary_by_host_by_event_name表的記憶體總分配次數字段COUNT_ALLOC - 記憶體釋放次數COUNT_FREE)

  • current_allocated:當前已分配的且未釋放的記憶體位元組數

  • current_avg_alloc:當前已分配的且未釋放的記憶體塊對應的平均每次記憶體分配的記憶體位元組數(current_allocated/current_count_used)

  • current_max_alloc:當前已分配的且未釋放的單次最大記憶體分配位元組數

  • total_allocated:總的已分配記憶體位元組數

04.memory_by_thread_by_current_bytes,x$memory_by_thread_by_current_bytes

按照thread ID分組的記憶體使用統計資訊(只統計前臺執行緒),預設情況下按照當前記憶體使用量進行降序排序,資料來源:performance_schema.memory_summary_by_thread_by_event_name、performance_schema.threads

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected]lhost : sys 10:04:07> select * from memory_by_thread_by_current_bytes limit 3;
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| thread_id | user                    | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
|        45 | [email protected]          |                34 | 4.91 MiB          | 147.98 KiB        | 4.00 MiB          | 29.36 MiB      |
|        41 | innodb/dict_stats_thread |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 346.31 KiB      |
|        47 | [email protected]          |                  3 | 112 bytes        | 37 bytes          | 80 bytes          | 8.17 KiB        |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.13 sec)

# 帶x$字首的檢視
[email protected] : sys 10:04:58> select * from x$memory_by_thread_by_current_bytes limit 3;
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| thread_id | user                    | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
|        45 | [email protected]          |                19 |          5102538 |      268554.6316 |          4194304 |        44995979 |
|        41 | innodb/dict_stats_thread |                  5 |              176 |          35.2000 |              160 |          354620 |
|        47 | [email protected]          |                  3 |              112 |          37.3333 |                80 |            8368 |
+-----------+--------------------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.12 sec)

檢視欄位含義如下:

  • thread_id:內部thread ID

  • user:對於前臺執行緒,該欄位顯示為account名稱,對於後臺執行緒,該欄位顯示後臺執行緒名稱

  • 其他欄位含義與sys.memory_by_host_by_current_bytes檢視的欄位含義相同,詳見 memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes檢視解釋部分。但是與該檢視不同的是本檢視是按照執行緒分組統計的

05.memory_by_user_by_current_bytes,x$memory_by_user_by_current_bytes

按照使用者分組的記憶體使用統計資訊,預設按照當前記憶體使用量進行降序排序,資料來源:performance_schema.memory_summary_by_user_by_event_name

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 10:05:02> select * from memory_by_user_by_current_bytes limit 3;
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| user      | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| admin      |              58291 | 36.71 MiB        | 660 bytes        | 14.20 MiB        | 2.41 GiB        |
| background |                  5 | 176 bytes        | 35 bytes          | 160 bytes        | 358.17 KiB      |
| qfsys      |                  0 | 0 bytes          | 0 bytes          | 0 bytes          | 0 bytes        |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

# 帶x$字首的檢視
[email protected] : sys 10:05:21> select * from x$memory_by_user_by_current_bytes limit 3;
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| user      | current_count_used | current_allocated | current_avg_alloc | current_max_alloc | total_allocated |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
| admin      |              58278 |          38460932 |          659.9563 |          14885584 |      2586890836 |
| background |                  5 |              176 |          35.2000 |              160 |          366828 |
| qfsys      |                  0 |                0 |            0.0000 |                0 |              0 |
+------------+--------------------+-------------------+-------------------+-------------------+-----------------+
3 rows in set (0.01 sec)

檢視欄位含義如下:

  • user:客戶端使用者名稱。對於後臺執行緒,該欄位顯示為background,對於前臺執行緒,該欄位顯示user名稱(不是account,不包含host部分)

  • 其他欄位含義與sys.memory_by_host_by_current_bytes檢視的欄位含義相同,詳見 memory_by_host_by_current_bytes,x$memory_by_host_by_current_bytes檢視解釋部分。但是與該檢視不同的是這裡是按照使用者名稱分組統計的

06.memory_global_by_current_bytes,x$memory_global_by_current_bytes

按照記憶體分配型別(事件型別)分組的記憶體使用統計資訊,預設情況下按照當前記憶體使用量進行降序排序,資料來源:performance_schema.memory_summary_global_by_event_name

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 10:05:24> select * from memory_global_by_current_bytes limit 3;
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| event_name                                                      | current_count | current_alloc | current_avg_alloc | high_count | high_alloc | high_avg_alloc |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| memory/innodb/lock0lock                                        |          9166 | 14.20 MiB    | 1.59 KiB          |      9166 | 14.20 MiB  | 1.59 KiB      |
| memory/performance_schema/events_statements_history_long        |            1 | 13.66 MiB    | 13.66 MiB        |          1 | 13.66 MiB  | 13.66 MiB      |
| memory/performance_schema/events_statements_history_long.tokens |            1 | 9.77 MiB      | 9.77 MiB          |          1 | 9.77 MiB  | 9.77 MiB      |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
3 rows in set (0.01 sec)

# 帶x$字首的檢視
[email protected] : sys 10:07:19> select * from x$memory_global_by_current_bytes limit 3;
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| event_name                                                      | current_count | current_alloc | current_avg_alloc | high_count | high_alloc | high_avg_alloc |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
| memory/innodb/lock0lock                                        |          9166 |      14885584 |        1624.0000 |      9166 |  14885584 |      1624.0000 |
| memory/performance_schema/events_statements_history_long        |            1 |      14320000 |    14320000.0000 |          1 |  14320000 |  14320000.0000 |
| memory/performance_schema/events_statements_history_long.tokens |            1 |      10240000 |    10240000.0000 |          1 |  10240000 |  10240000.0000 |
+-----------------------------------------------------------------+---------------+---------------+-------------------+------------+------------+----------------+
3 rows in set (0.00 sec)

檢視欄位含義如下:

  • EVENT_NAME:記憶體事件名稱

  • CURRENT_COUNT:當前已分配記憶體且未釋放的記憶體事件發生的總次數(記憶體分配次數)

  • current_alloc:當前已分配記憶體且未釋放的記憶體位元組數

  • current_avg_alloc:當前已分配記憶體且未釋放的記憶體事件的平均記憶體位元組數(平均每次記憶體分配的位元組數)

  • high_count:記憶體事件發生的歷史最高位(高水位)次數(來自performance_schema.memory_summary_global_by_event_name表中的HIGH_COUNT_USED欄位:如果CURRENT_COUNT_USED增加1是一個新的最高值,則該欄位值相應增加 )

  • high_alloc:記憶體分配的歷史最高位(高水位)位元組數(來自performance_schema.memory_summary_global_by_event_name表中的HIGH_NUMBER_OF_BYTES_USED欄位:如果CURRENT_NUMBER_OF_BYTES_USED增加N之後是一個新的最高值,則該欄位值相應增加)

  • high_avg_alloc:記憶體事件發生的歷史最高位(高水位)次數對應的平均每次記憶體分配的位元組數(high_number_of_bytes_used/high_count_used)

07.memory_global_total,x$memory_global_total

當前總記憶體使用量統計(注意:只包含自memory型別的instruments啟用以來被監控到的記憶體事件,在啟用之前的無法監控,so..如果你不是在server啟動之前就在配置檔案中配置啟動memory型別的instruments,那麼此值可能並不可靠,當然如果你的server執行時間足夠長,那麼該值也具有一定參考價值),資料來源:performance_schema.memory_summary_global_by_event_name

下面我們看看使用該檢視查詢返回的結果。

# 不帶x$字首的檢視
[email protected] : sys 10:07:22> select * from memory_global_total limit 3;
+-----------------+
| total_allocated |
+-----------------+
| 168.91 MiB      |
+-----------------+
1 row in set (0.01 sec)

# 帶x$字首的檢視
[email protected] : sys 10:08:24> select * from x$memory_global_total limit 3;
+-----------------+
| total_allocated |
+-----------------+
|      177099388 |
+-----------------+
1 row in set (0.00 sec)

檢視欄位含義如下:

  • total_allocated:在server中分配的記憶體總位元組數

本期內容就介紹到這裡,本期內容參考連結如下:

https://dev.mysql.com/doc/refman/5.7/en/sys-memory-global-total.html

https://dev.mysql.com/doc/refman/5.7/en/sys-innodb-buffer-stats-by-table.html

https://dev.mysql.com/doc/refman/5.7/en/sys-memory-by-host-by-current-bytes.html

https://dev.mysql.com/doc/refman/5.7/en/sys-memory-by-thread-by-current-bytes.html

https://dev.mysql.com/doc/refman/5.7/en/sys-memory-by-user-by-current-bytes.html

https://dev.mysql.com/doc/refman/5.7/en/sys-memory-global-by-current-bytes.html

https://dev.mysql.com/doc/refman/5.7/en/sys-innodb-buffer-stats-by-schema.html

| 作者簡介

羅小波·沃趣科技高階資料庫技術專家

IT從業多年,歷任運維工程師,高階運維工程師,運維經理,資料庫工程師,曾參與版本釋出系統,輕量級監控系統,運維管理平臺,資料庫管理平臺的設計與編寫,熟悉MySQL的體系結構時,InnoDB儲存引擎,喜好專研開源技術,追求完美。