1. 程式人生 > >【MySQL】sysbench壓測伺服器及結果解讀

【MySQL】sysbench壓測伺服器及結果解讀

 

主要壓測範圍包括CPU測試、磁碟IO測試、執行緒測試、OLTP測試等,那麼sysbench就可以滿足我們的壓測需求。下面我們簡單來看下sysbench的安裝使用以及壓測結果的解讀。

一、sysbench安裝

sysbench安裝其實非常簡單,可以參考下git,這裡我們講下原始碼編譯安裝。
上傳sysbench到伺服器、解壓、安裝:

[[email protected]~] yum -y install make automake libtool pkgconfig libaio-devel
[[email protected]~] git clone
 

[[email protected] sysbench]# ls -l
total 108
-rwxr-xr-x 1 root root 26 Nov 23 15:39 autogen.sh
-rw-r--r-- 1 root root 14511 Nov 23 15:39 ChangeLog
drwxr-xr-x 2 root root 26 Nov 23 15:39 config
-rw-r--r-- 1 root root 11396 Nov 23 15:39 configure.ac
-rw-r--r-- 1 root root 18092 Nov 23 15:39 COPYING
drwxr-xr-x 3 root root 135 Nov 23 15:39 debian
-rwxr-xr-x 1 root root 7122 Nov 23 15:39 install-sh
drwxr-xr-x 2 root root 4096 Nov 23 15:39 m4
-rw-r--r-- 1 root root 1547 Nov 23 15:39 Makefile.am
-rwxr-xr-x 1 root root 10266 Nov 23 15:39 missing
-rwxr-xr-x 1 root root 1988 Nov 23 15:39 mkinstalldirs
-rw-r--r-- 1 root root 18867 Nov 23 15:39 README.md
drwxr-xr-x 2 root root 27 Nov 23 15:39 rpm
drwxr-xr-x 2 root root 26 Nov 23 15:39 scripts
drwxr-xr-x 2 root root 31 Nov 23 15:39 snap
drwxr-xr-x 5 root root 4096 Nov 23 15:39 src
drwxr-xr-x 4 root root 85 Nov 23 15:39 tests
drwxr-xr-x 5 root root 55 Nov 23 15:39 third_party
[

[email protected] sysbench]#

 
[root[email protected] tmp]# unzip sysbench-master.zip
[[email protected] tmp]# cd sysbench-master [[email protected] sysbench-master]# ./autogen.sh [[email protected] sysbench-master]# ./configure [[email protected] sysbench-master]# make -j [
[email protected]
sysbench-master]# make install

其實可以檢視下README.md,上面也寫了如何安裝

## Build and Install
 shell
    ./autogen.sh
    # Add --with-pgsql to build with PostgreSQL support
    ./configure
    make -j
    make install

[[email protected] sysbench]# sysbench
sysbench: error while loading shared libraries: libmysqlclient.so.20: cannot open shared object file: No such file or directory

字面意思就是打不到共享庫檔案libmysqlclient.so.20.

先查詢一下本地有沒有這個檔案:

[[email protected] sysbench]# find .  -name /libmysqlclient.so.20                                               ./usr/lib64/mysql/libmysqlclient.so.16
./usr/local/mysql/lib/libmysqlclient.so.20

在/usr/loca/mysql/lib下面有這個檔案,解決方法是先建立一個軟連結到/usr/local/lib

[[email protected] sysbench] ln -s /usr/local/mysql/lib/libmysqlclient.so.20 /usr/local/lib/libmysqlclient.so.20
接著在/etc/ld.so.cnf中加入/usr/loca/lib這一行

[[email protected] sysbench]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib
執行ldconfig -v更新下配置就可以了

[[email protected] sysbench]# /sbin/ldconfig -v
現在sysbench可以執行了!

[[email protected] sysbench]# sysbench --version
sysbench 1.1.0-5cd1244

 

[[email protected] sysbench]# sysbench --help
Usage:
sysbench [options]... [testname] [command]

Commands implemented by most tests: prepare run cleanup help

General options:
--threads=N number of threads to use [1]
--events=N limit for total number of events [0]
--time=N limit for total execution time in seconds [10]
--warmup-time=N execute events for this many seconds with statistics disabled before the actual benchmark run with statistics enabled [0]
--forced-shutdown=STRING number of seconds to wait after the --time limit before forcing shutdown, or 'off' to disable [off]
--thread-stack-size=SIZE size of stack per thread [64K]
--thread-init-timeout=N wait time in seconds for worker threads to initialize [30]
--rate=N average transactions rate. 0 for unlimited rate [0]
--report-interval=N periodically report intermediate statistics with a specified interval in seconds. 0 disables intermediate reports [0]
--report-checkpoints=[LIST,...] dump full statistics and reset all counters at specified points in time. The argument is a list of comma-separated values representing the amount of time in seconds elapsed from start of test when report checkpoint(s) must be performed. Report checkpoints are off by default. []
--debug[=on|off] print more debugging info [off]
--validate[=on|off] perform validation checks where possible [off]
--help[=on|off] print help and exit [off]
--version[=on|off] print version and exit [off]
--config-file=FILENAME File containing command line options
--luajit-cmd=STRING perform LuaJIT control command. This option is equivalent to 'luajit -j'. See LuaJIT documentation for more information

Pseudo-Random Numbers Generator options:
--rand-type=STRING random numbers distribution {uniform, gaussian, special, pareto, zipfian} to use by default [special]
--rand-seed=N seed for random number generator. When 0, the current time is used as an RNG seed. [0]
--rand-spec-iter=N number of iterations for the special distribution [12]
--rand-spec-pct=N percentage of the entire range where 'special' values will fall in the special distribution [1]
--rand-spec-res=N percentage of 'special' values to use for the special distribution [75]
--rand-pareto-h=N shape parameter for the Pareto distribution [0.2]
--rand-zipfian-exp=N shape parameter (exponent, theta) for the Zipfian distribution [0.8]

Log options:
--verbosity=N verbosity level {5 - debug, 0 - only critical messages} [3]

--percentile=N percentile to calculate in latency statistics (1-100). Use the special value of 0 to disable percentile calculations [95]
--histogram[=on|off] print latency histogram in report [off]

General database options:

--db-driver=STRING specifies database driver to use ('help' to get list of available drivers) [mysql]
--db-ps-mode=STRING prepared statements usage mode {auto, disable} [auto]
--db-debug[=on|off] print database-specific debug information [off]


Compiled-in database drivers:
mysql - MySQL driver

mysql options:
--mysql-host=[LIST,...] MySQL server host [localhost]
--mysql-port=[LIST,...] MySQL server port [3306]
--mysql-socket=[LIST,...] MySQL socket
--mysql-user=STRING MySQL user [sbtest]
--mysql-password=STRING MySQL password []
--mysql-db=STRING MySQL database name [sbtest]
--mysql-ssl[=on|off] use SSL connections, if available in the client library [off]
--mysql-ssl-key=STRING path name of the client private key file
--mysql-ssl-ca=STRING path name of the CA file
--mysql-ssl-cert=STRING path name of the client public key certificate file
--mysql-ssl-cipher=STRING use specific cipher for SSL connections []
--mysql-compression[=on|off] use compression, if available in the client library [off]
--mysql-debug[=on|off] trace all client library calls [off]
--mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205]
--mysql-dry-run[=on|off] Dry run, pretend that all MySQL client API calls are successful without executing them [off]

Compiled-in tests:
fileio - File I/O test
cpu - CPU performance test
memory - Memory functions speed test
threads - Threads subsystem performance test
mutex - Mutex performance test

See 'sysbench <testname> help' for a list of options for each test.

二、壓測過程及壓測結果解讀

1.cpu測試

[[email protected] storage]# sysbench --test=cpu --cpu-max-prime=2000000 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Prime numbers limit: 2000000 Initializing worker threads... Threads started! CPU speed: events per second: 0.63 Throughput: events/s (eps): 0.6334 time elapsed: 11.0520s total number of events: 7 Latency (ms): min: 1571.86 avg: 1578.84 max: 1620.53 95th percentile: 1618.78 sum: 11051.91 Threads fairness: events (avg/stddev): 7.0000/0.00 execution time (avg/stddev): 11.0519/0.00 

cpu測試主要是進行素數的加法運算,上面我們的例子中,指定了最大的質數發生器數量為 2000000,可以看出伺服器此次測試 執行時間 大約為11.05秒
2.磁碟IO測試

[[email protected] storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw prepare
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3)

128 files, 245760Kb each, 30720Mb total
Creating files for the test... Extra file open flags: (none) Creating file test_file.0 Creating file test_file.1 Creating file test_file.2 Creating file test_file.3 Creating file test_file.4 Creating file test_file.5 Creating file test_file.6 Creating file test_file.7 Creating file test_file.8 Creating file test_file.9 Creating file test_file.10 Creating file test_file.11 Creating file test_file.12 Creating file test_file.13 Creating file test_file.14 Creating file test_file.15 Creating file test_file.16 Creating file test_file.17 Creating file test_file.18 Creating file test_file.19 Creating file test_file.20 Creating file test_file.21 Creating file test_file.22 Creating file test_file.23 Creating file test_file.24 Creating file test_file.25 Creating file test_file.26 Creating file test_file.27 Creating file test_file.28 Creating file test_file.29 Creating file test_file.30 Creating file test_file.31 Creating file test_file.32 Creating file test_file.33 Creating file test_file.34 Creating file test_file.35 Creating file test_file.36 Creating file test_file.37 Creating file test_file.38 Creating file test_file.39 Creating file test_file.40 Creating file test_file.41 Creating file test_file.42 Creating file test_file.43 Creating file test_file.44 Creating file test_file.45 Creating file test_file.46 Creating file test_file.47 Creating file test_file.48 Creating file test_file.49 Creating file test_file.50 Creating file test_file.51 Creating file test_file.52 Creating file test_file.53 Creating file test_file.54 Creating file test_file.55 Creating file test_file.56 Creating file test_file.57 Creating file test_file.58 Creating file test_file.59 Creating file test_file.60 Creating file test_file.61 Creating file test_file.62 Creating file test_file.63 Creating file test_file.64 Creating file test_file.65 Creating file test_file.66 Creating file test_file.67 Creating file test_file.68 Creating file test_file.69 Creating file test_file.70 Creating file test_file.71 Creating file test_file.72 Creating file test_file.73 Creating file test_file.74 Creating file test_file.75 Creating file test_file.76 Creating file test_file.77 Creating file test_file.78 Creating file test_file.79 Creating file test_file.80 Creating file test_file.81 Creating file test_file.82 Creating file test_file.83 Creating file test_file.84 Creating file test_file.85 Creating file test_file.86 Creating file test_file.87 Creating file test_file.88 Creating file test_file.89 Creating file test_file.90 Creating file test_file.91 Creating file test_file.92 Creating file test_file.93 Creating file test_file.94 Creating file test_file.95 Creating file test_file.96 Creating file test_file.97 Creating file test_file.98 Creating file test_file.99 Creating file test_file.100 Creating file test_file.101 Creating file test_file.102 Creating file test_file.103 Creating file test_file.104 Creating file test_file.105 Creating file test_file.106 Creating file test_file.107 Creating file test_file.108 Creating file test_file.109 Creating file test_file.110 Creating file test_file.111 Creating file test_file.112 Creating file test_file.113 Creating file test_file.114 Creating file test_file.115 Creating file test_file.116 Creating file test_file.117 Creating file test_file.118 Creating file test_file.119 Creating file test_file.120 Creating file test_file.121 Creating file test_file.122 Creating file test_file.123 Creating file test_file.124 Creating file test_file.125 Creating file test_file.126 Creating file test_file.127 32212254720 bytes written in 55.25 seconds (556.00 MiB/sec). [[email protected] storage]# sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw run WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options. WARNING: --num-threads is deprecated, use --threads instead sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 16 Initializing random number generator from current time Extra file open flags: (none) 128 files, 240MiB each 30GiB total file size Block size 16KiB Number of IO requests: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Initializing worker threads... Threads started! Throughput: read: IOPS=68491.05 1070.17 MiB/s (1122.16 MB/s) write: IOPS=45660.96 713.45 MiB/s (748.11 MB/s) fsync: IOPS=146103.57 Latency (ms): min: 0.00 avg: 0.06 max: 3.61 95th percentile: 0.31 sum: 156128.82 

可以看到,磁碟的讀IOPS可以達到68491.05,寫IOPS可以達到45660.96,fsync可以達到146103.57
3.執行緒測試

[[email protected] storage]# sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 64 Initializing random number generator from current time Initializing worker threads... Threads started! Throughput: events/s (eps): 14851.1802 time elapsed: 10.0036s total number of events: 148565 Latency (ms): min: 0.02 avg: 4.31 max: 24.49 95th percentile: 10.46 sum: 639889.93 Threads fairness: events (avg/stddev): 2321.3281/42.94 execution time (avg/stddev): 9.9983/0.00 

(傳送64次/個測試執行緒請求,每次/個執行緒請求產生/生成100個數量,每個執行緒的鎖數量為2) ,測試執行時間為10秒
4.記憶體測試

[[email protected] storage]# sysbench --test=memory --memory-block-size=8k --memory-total-size=40G run
WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Running memory speed test with the following options: block size: 8KiB total size: 40960MiB operation: write scope: global Initializing worker threads... Threads started! Total operations: 5242880 (1443403.42 per second) 40960.00 MiB transferred (11276.59 MiB/sec) Throughput: events/s (eps): 1443403.4239 time elapsed: 3.6323s total number of events: 5242880 Latency (ms): min: 0.00 avg: 0.00 max: 0.02 95th percentile: 0.00 sum: 2926.10 Threads fairness: events (avg/stddev): 5242880.0000/0.00 execution time (avg/stddev): 2.9261/0.00 

上述引數指定了本次測試整個過程是在記憶體中傳輸 40G 的資料量,每個 block 大小為 8K。測試結果顯示:
執行時間為3.6323秒,每秒傳輸速度為11276.59Mb每秒
5.OLTP測試
(1)100執行緒
資料準備:

[[email protected] storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=100 --max-requests=0 prepare sysbench 1.1.0 (using bundled LuaJIT 2.1.0-beta3) Initializing worker threads... Creating table 'sbtest1'... Creating table 'sbtest4'... Creating table 'sbtest2'... Creating table 'sbtest9'... Creating table 'sbtest6'... Creating table 'sbtest8'... Creating table 'sbtest7'... Creating table 'sbtest5'... Creating table 'sbtest10'... Creating table 'sbtest3'... Inserting 20000000 records into 'sbtest1' Inserting 20000000 records into 'sbtest7' Inserting 20000000 records into 'sbtest4' Inserting 20000000 records into 'sbtest9' Inserting 20000000 records into 'sbtest6' Inserting 20000000 records into 'sbtest8' Inserting 20000000 records into 'sbtest2' Inserting 20000000 records into 'sbtest5' Inserting 20000000 records into 'sbtest3' Inserting 20000000 records into 'sbtest10' Creating a secondary index on 'sbtest4'... Creating a secondary index on 'sbtest1'... Creating a secondary index on 'sbtest10'... Creating a secondary index on 'sbtest3'... Creating a secondary index on 'sbtest6'... Creating a secondary index on 'sbtest8'... Creating a secondary index on 'sbtest7'... Creating a secondary index on 'sbtest9'... Creating a secondary index on 'sbtest5'... Creating a secondary index on 'sbtest2'... 

開始測試:

[[email protected] storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua --mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=100 --time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_100.log

測試過程中,可以tail -f /tmp/40_100.log檢視輸出
測試結果:這裡只展示統計部分

SQL statistics:
    queries performed:
        read:                            143195290
        write:                           40912940
        other:                           20456470
        total:                           204564700
    transactions:                        10228235 (2841.15 per sec.)
    queries:                             204564700 (56823.07 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

Throughput:
    events/s (eps):                      2841.1536
    time elapsed:                        3600.0289s
    total number of events:              10228235

Latency (ms):
         min:                                    2.49
         avg:                                   35.19
         max:                                 2017.54
         95th percentile:                       39.65
         sum:                            359973625.38

Threads fairness:
    events (avg/stddev):           102282.3500/5194.20
    execution time (avg/stddev):   3599.7363/0.01

(2)200執行緒
資料準備:

[[email protected] storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=200 --max-requests=0 prepare 

開始測試:

[[email protected] storage]# sysbench /usr/local/share/sysbench/oltp_read_write.lua \
--mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest --mysql-user=tpcc \
--mysql-password=tpcc --table_size=20000000 --tables=10 --threads=200 --time=3600 \ --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_200.log 

測試結果:

SQL statistics:
    queries performed:
        read:                            142876258
        write:                           40821788
        other:                           20410894
        total:                           204108940
    transactions:                        10205447 (2834.80 per sec.)
    queries:                             204108940 (56696.02 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

Throughput:
    events/s (eps):                      2834.8009
    time elapsed:                        3600.0577s
    total number of events:              10205447

Latency (ms):
         min:                                    2.39
         avg:                                   70.55
         max:                                 2011.65
         95th percentile:                       80.03
         sum:                            719978372.22

Threads fairness:
    events (avg/stddev):           51027.2350/3840.19
    execution time (avg/stddev):   3599.8919/0.02

(3)300執行緒
資料準備:

[[email protected] storage]# sysbench /usr/local/share/sysbench/oltp_common.lua \
> --mysql-user=tpcc --mysql-password=tpcc --mysql-socket=/tmp/mysql3306.sock \
> --tables=10 --table_size=20000000 --threads=300 --max-requests=0 prepare 

開始測試:

sysbench /usr/local/share/sysbench/oltp_read_write.lua \
--mysql-host=localhost --mysql-port=3306 --mysql-db=sbtest \
--mysql-user=tpcc --mysql-password=tpcc --table_size=20000000 --tables=10 --threads=300 \
--time=3600 --report-interval=10 --mysql-socket=/tmp/mysql3306.sock run > /tmp/40_300.log 

測試結果:

SQL statistics:
    queries performed:
        read:                            144281438
        write:                           41223268
        other:                           20611634
        total:                           206116340
    transactions:                        10305817 (2862.66 per sec.)
    queries:                             206116340 (57253.12 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

Throughput:
    events/s (eps):                      2862.6561
    time elapsed:                        3600.0891s
    total number of events:              10305817

Latency (ms):
         min:                                    2.64
         avg:                                  104.79
         max:                                 4576.60
         95th percentile:                      123.28
         sum:                           1079986824.04

Threads fairness:
    events (avg/stddev):           34352.7233/2459.96
    execution time (avg/stddev):   3599.9561/0.03

生成壓測圖
安裝gnuplot

plot 'D:\40_100.log' using 9 with linepoint title '100 threads','D:\40_200.log' using 9 with linepoint title '200 threads','D:\40_300.log' using 9 with linepoint title '300 threads'; 
  多執行緒QPS對比圖   多執行緒TPS對比圖

三、壓測注意事項

1.在OLTP測試過程中,資料庫初始化後,冷備data目錄,每次測試完畢後刪除當前資料目錄,拷貝data目錄重新啟動,為的是每次開始測試前環境保持一致。
2.測試完成後,需要再系統層做一些清理的工作
shell>sync #重新整理髒資料到磁碟
shell>echo 3 >/proc/sys/vm/drop_cache #清除cache
shell>swapoff -a && swapon -a 釋放swap
3.壓測過程中需要修改引數max_prepared_stmt_count預設為16382,修改為65536
4.如果是SSD硬碟的話,innodb_io_capacity和innodb_io_capacity_max兩個引數也要相應調大,建議調整到50000以上。
5.對於測試資料的準備,我們通過show engine innodb status觀察Free buffers,儘量模擬生產環境,讓測試資料填滿整個buffer pool。我這裡innodb_buffer_pool_size設定的40G,測試表數量10張,每張表資料2000萬,差不多填滿buffer pool。