1. 程式人生 > >MySQL壓測--參數壓測

MySQL壓測--參數壓測

自動 mysqld 嚴格 where myisam toolbar isa ddl data

我們知道,MySQL為了保證數據的盡量不丟失,一般會把參數sync_binlog 和innodb_flush_log_at_trx_commit都設置為1,即雙1原則,但是,凡是都有兩面性,高安全和高性能往往是讓人很難取舍的,需要根據你的實際業務需求進行抉擇;


1.測試背景

首先分別了解一下這兩個參數的官方解釋:

sync_binlog

Controls how often the MySQL server synchronizes the binary log to disk.
控制MySQL server 同步binlog到磁盤的頻率

sync_binlog=0: Disables synchronization of the binary log to disk by the MySQL server. Instead, the MySQL server relies on the operating system to flush the binary log to disk from time to time as it does for any other file. This setting provides the best performance, but in the event of a power failure or operating system crash, it is possible that the server has committed transactions that have not been synchronized to the binary log.
關閉MySQL server binlog同步到磁盤,由OS不時的刷新到磁盤,這是MySQL最好的性能,但是當斷電或者系統崩潰,MySQL server已經提交的事物可能還沒有同步到binlog,導致事物丟失。

sync_binlog=1: Enables synchronization of the binary log to disk before transactions are committed. This is the safest setting but can have a negative impact on performance due to the increased number of disk writes. In the event of a power failure or operating system crash, transactions that are missing from the binary log are only in a prepared state. This permits the automatic recovery routine to roll back the transactions, which guarantees that no transaction is lost from the binary log.
在事物提交之前同步binlog到磁盤,這是最安全的,但是由於增加了磁盤的寫操作,會有一些消極的性能影響。當斷電或者系統崩潰,從binlog中丟失的事物僅僅處於準備狀態。這個允許自動恢復程序去rollback事物,保證binlog沒有事物丟失。

sync_binlog=N, where N is a value other than 0 or 1: The binary log is synchronized to disk after N binary log commit groups have been collected. In the event of a power failure or operating system crash, it is possible that the server has committed transactions that have not been flushed to the binary log. This setting can have a negative impact on performance due to the increased number of disk writes. A higher value improves performance, but with an increased risk of data loss.
在N個binlog提交組被收集到之後,binglog同步到磁盤。當斷電或者系統崩潰,已經提交的事物可能還沒有刷新到binlog。這樣的配置增加了磁盤的寫操作,會有一些消極的性能影響。N值越高,數據庫性能越好,但是數據越可能丟失。

For the greatest possible durability and consistency in a replication setup that uses InnoDB with transactions, use these settings:
為了使使用InnoDB的復制設置具有最大的持久性和一致性,請使用以下設置:

sync_binlog=1.
innodb_flush_log_at_trx_commit=1.

Caution
Many operating systems and some disk hardware fool the flush-to-disk operation. They may tell mysqld that the flush has taken place, even though it has not. In this case, the durability of transactions is not guaranteed even with the recommended settings, and in the worst case, a power outage can corrupt InnoDB data. Using a battery-backed disk cache in the SCSI disk controller or in the disk itself speeds up file flushes, and makes the operation safer. You can also try to disable the caching of disk writes in hardware caches.
許多操作系統和一些磁盤硬件欺騙了刷新磁盤的操作。它們可能會告訴sqmyld flush已經發生了,盡管沒有發生。在這種情況下,即使使用推薦的設置,事務的持久性也不能得到保證,在最壞的情況下,斷電可能損壞InnoDB數據。在SCSI磁盤控制器或磁盤本身中使用電池支持的磁盤緩存可以加速文件刷新,並使操作更加安全。您還可以嘗試禁用硬件緩存中的磁盤寫緩存。



innodb_flush_log_at_trx_commit


Controls the balance between strict ACID compliance for commit operations and higher performance that is possible when commit-related I/O operations are rearranged and done in batches. You can achieve better performance by changing the default value but then you can lose up to a second of transactions in a crash.
控制提交操作嚴格遵守ACID和提高性能之間的平衡,這在批量重新安排和執行與提交相關的I/O操作時是可能的,通過更改默認值,您可以獲得更好的性能,但是在崩潰時,你可能會損失多達一秒鐘的事務

The default value of 1 is required for full ACID compliance. With this value, the contents of the InnoDB log buffer are written out to the log file at each transaction commit and the log file is flushed to disk.
默認值1是必須的對ACID性.InnoDB日誌緩沖區的內容在每個事務提交時被寫到日誌文件,日誌文件被刷新到磁盤。

With a value of 0, the contents of the InnoDB log buffer are written to the log file approximately once per second and the log file is flushed to disk. No writes from the log buffer to the log file are performed at transaction commit. Once-per-second flushing is not guaranteed to happen every second due to process scheduling issues. Because the flush to disk operation only occurs approximately once per second, you can lose up to a second of transactions with any mysqld process crash.
值為0時,InnoDB日誌緩沖區的內容大約每秒寫入一次日誌文件,日誌文件被刷新到磁盤。事務提交時不執行從日誌緩沖區寫入日誌文件的操作。由於進程調度問題,不能保證每秒一次刷新。由於對磁盤的刷新操作大約每秒鐘只發生一次,因此您可以在任何mysqld進程中損失最多一秒鐘的事務

With a value of 2, the contents of the InnoDB log buffer are written to the log file after each transaction commit and the log file is flushed to disk approximately once per second. Once-per-second flushing is not 100% guaranteed to happen every second, due to process scheduling issues. Because the flush to disk operation only occurs approximately once per second, you can lose up to a second of transactions in an operating system crash or a power outage.
當值為2時,InnoDB日誌緩沖區的內容在每次事務提交後被寫入日誌文件,日誌文件大約每秒被刷新一次。由於進程調度問題,每秒一次的刷新不能100%保證每秒發生。由於磁盤刷新操作大約每秒發生一次,所以在操作系統崩潰或斷電時,最多可能損失1秒的事務。

InnoDB log flushing frequency is controlled by innodb_flush_log_at_timeout, which allows you to set log flushing frequency to N seconds (where N is 1 ... 2700, with a default value of 1). However, any mysqld process crash can erase up to N seconds of transactions.
DDL changes and other internal InnoDB activities flush the InnoDB log independent of the innodb_flush_log_at_trx_commit setting.
InnoDB crash recovery works regardless of the innodb_flush_log_at_trx_commit setting. Transactions are either applied entirely or erased entirely.
For durability and consistency in a replication setup that uses InnoDB with transactions:
If binary logging is enabled, set sync_binlog=1.
Always set innodb_flush_log_at_trx_commit=1.

Caution
Many operating systems and some disk hardware fool the flush-to-disk operation. They may tell mysqld that the flush has taken place, even though it has not. In this case, the durability of transactions is not guaranteed even with the recommended settings, and in the worst case, a power outage can corrupt InnoDB data. Using a battery-backed disk cache in the SCSI disk controller or in the disk itself speeds up file flushes, and makes the operation safer. You can also try to disable the caching of disk writes in hardware caches.


2.測試配置

這裏TPCC和MySQL分別在兩臺機器上面,是為了排除TPCC運行過程中對MySQL服務資源的占用的影響

IP:172.16.101.54

CPU:2core

Memory:8G

軟件:tpcc-mysql-master


IP:172.16.101.55

CPU:2core

Memory:6G

軟件:MySQL5.7.21


MySQL其他部分參數配置:

key_buffer_size = 8M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 0M
query_cache_type= off
max_allowed_packet = 64M
myisam_sort_buffer_size=128M
tmp_table_size=32M
table_open_cache = 2000
thread_cache_size = 8
wait_timeout = 300
interactive_timeout = 300
max_connections = 8000
max_user_connections=0
innodb_thread_concurrency = 4 
transaction_isolation = READ-COMMITTED
gtid-mode = on
enforce-gtid-consistency = true
log_slave_updates = on
innodb_buffer_pool_size = 5120M
innodb_log_buffer_size = 16M
innodb_lock_wait_timeout = 100



3.測試過程

測試主要分兩種情況,通過tpcc測得MySQL server的tps和qps,每一種情況測試三遍,取平均值,並做成表格和折線圖。


innodb_flush_log_at_trx_commit = 1 && sync_binlog = 1

or

innodb_flush_log_at_trx_commit = 0 && sync_binlog = 0


4.測試結果

首先說明一下,因為手頭上只有這樣低配置的服務器,所以測試結果不是那麽的可靠,所以僅僅參考測試思想和過程即可,不要太在意測試結果,哈哈。


(1)TpmC

即每分鐘的事物數

線程數48163264128256512102415362048
i=1&s=1864.4331411.3672020.6672465.8672666.5332666.6332614.3332431.2672011.2331507.7331266.1
i=0&s=02761.3833160.052835.8172510.6832366.0172303.252367.5172251.8171834.5831439.6841166.3

技術分享圖片


從圖中我們可以看到:

  • 並發0-32的時候,雙0的MySQL TpmC遠遠大於雙1;

  • 並發>32後,兩者之間的TpmC數量差不多


(2)雙1情況下的 slow queries,open tables,Queries per second avg等指標的情況


技術分享圖片


從圖中我們可以看到:

  • 隨著並發越來越大,slow queries也越來越多;

  • 當並發達到512左右,open tables也達到參數table_open_cache = 2000的設定值;

  • 隨著並發越來越大,QPS越來越小,剛開始並不明顯,一度處於平衡,知道並發太大,QPS性能下降;


參考鏈接:

《MySQL壓測--TPCC安裝,測試》 http://blog.51cto.com/darrenmemos/2130202

《MySQL壓測--註意事項和FAQ》 http://blog.51cto.com/darrenmemos/2132084



MySQL壓測--參數壓測