經常要對一些新儲存系統進行 I/O Benchmark 測試，每次測試又有可能針對不同的目的，但基本也都是圍繞資料庫轉悠，心血來潮，對幾個常見的工具做個比較。

要強調的幾點:
ORION –Oracle I/O Numbers Calibration Tool 還是比較全面的針對資料庫應用的 IO 測試工具。現在 Oracle 釋出了不少平臺的移植版本。該工具也比較好用。

資料庫應用必需要考慮非同步 I/O 的因素，否則結果會有很大偏差，當然如果只測試儲存能力的話，到可以忽略。AIO 壓力測試可以考慮以下 AIO-Stress

Unix 命令 dd 雖然很土，但還是一個測試 I/O 的基本手段和方法.有的時候即使沒別的工具只用它也能發現很多問題。另外一個需要注意的就是字元裝置和塊裝置的差別啦。更新: 就當我說得是 GNU dd

吧，謝謝下面留言的朋友。

有些工具因為用過很久了，記憶難免有問題，表格中會有誤導。僅供參考。

---------------------------------------------------------------------------------------------------

Linux IO測試實踐  （原創文章）

以下資料的測試時間是 2013-7-28 23-24點 ，正是阿里雲伺服器不忙的時間。

方法1  hdparm  阿里雲伺服器

-------------------------------

CentOS 5.7作業系統 (阿里雲伺服器系統盤) 順序讀取測試

[[email protected] ~]# hdparm -t /dev/hda

/dev/hda:

 Timing buffered disk reads:  942 MB in  3.00 seconds = 313.98 MB/sec(第一次)

 Timing buffered disk reads:  934 MB in  3.00 seconds = 311.30 MB/sec(第二次)

Timing buffered disk reads:  162 MB in  3.00 seconds =  53.93 MB/sec  (2013-7-28 14:00)

CentOS 5.7作業系統 (阿里雲伺服器虛擬盤) 順序讀取測試

[[email protected] ~]# hdparm -t /dev/xvdb

/dev/xvdb:

 Timing buffered disk reads:  122 MB in  3.02 seconds =  40.38 MB/sec(第一次)

 Timing buffered disk reads:  134 MB in  3.03 seconds =  44.22 MB/sec(第二次)

參考：http://blog.chinaunix.net/uid-186064-id-2823326.html

方法1  hdparm   MAC OS 10.7 筆記本

-------------------------------

MAC OS 10.7 作業系統 （5400轉筆記本硬碟） 順序讀取測試

預設沒有hdparm，所有沒做測試

方法2  dd  公司實體機 15000轉sas

-------------------------------

[[email protected] /]# dd if=/dev/zero of=/zookeeper.dbf bs=8k count=200000 conv=fdatasync 200000+0 records in 200000+0 records out 1638400000 bytes (1.6 GB) copied, 14.2775 seconds, 115 MB/s 注：關於   conv=fdatasync       請參考： 正確的使用dd進行磁碟讀寫速度測試
http://elf8848.iteye.com/blogs/2089055
 linux 同步IO: sync、fsync與fdatasync 編輯
http://elf8848.iteye.com/blog/2088986

方法2  dd  阿里雲伺服器

-------------------------------

CentOS 5.7作業系統（阿里雲伺服器系統盤） 順序讀取測試

time dd if=/dev/hda of=/dev/null bs=8k count=102400

102400+0 records in

102400+0 records out

838860800 bytes (839 MB) copied, 2.9264 seconds, 287 MB/s (第一次)(為什麼這麼高？)

838860800 bytes (839 MB) copied, 3.0851 seconds, 272 MB/s (第二次)

838860800 bytes (839 MB) copied, 16.7665 seconds, 50.0 MB/s(第三次)（與上一次隔了一個小時，波動？）

838860800 bytes (839 MB) copied, 15.0574 seconds, 55.7 MB/s(第四次)

CentOS 5.7作業系統（阿里雲伺服器系統盤） 順序寫入測試

[[email protected] /]# time dd if=/dev/zero of=bigfile2 bs=8k count=102400

102400+0 records in

102400+0 records out

838860800 bytes (839 MB) copied, 13.3033 seconds, 63.1 MB/s(第一次)

838860800 bytes (839 MB) copied, 9.51204 seconds, 88.2 MB/s(第二次)

CentOS 5.7作業系統（阿里雲伺服器虛擬盤） 順序讀取測試

[[email protected] ~]# time dd if=/dev/xvdb of=/dev/null bs=8k count=102400

102400+0 records in

102400+0 records out

838860800 bytes (839 MB) copied, 9.49292 seconds, 88.4 MB/s (第一次)

838860800 bytes (839 MB) copied, 16.1384 seconds, 52.0 MB/s (第二次)

838860800 bytes (839 MB) copied, 14.7076 seconds, 57.0 MB/s (第三次)

838860800 bytes (839 MB) copied, 13.9852 seconds, 60.0 MB/s (第四次)

CentOS 5.7作業系統（阿里雲伺服器虛擬盤） 順序寫入測試

[[email protected] data0]# time dd if=/dev/zero of=bigfile2 bs=8k count=102400  （bigfile2檔案寫入到當前目錄）

102400+0 records in

102400+0 records out

838860800 bytes (839 MB) copied, 15.0549 seconds, 55.7 MB/s(第一次)

838860800 bytes (839 MB) copied, 12.8344 seconds, 65.4 MB/s(第二次)

請參考：

http://space.itpub.net/441887/viewspace-630694

http://blog.csdn.net/youyudehexie/article/details/8182494

方法2  dd  MAC OS 10.7 筆記本

-------------------------------

MAC OS 10.7 作業系統（5400轉筆記本硬碟） 順序讀取測試

[email protected] ~$ sudo time dd if=/dev/disk0 of=/dev/null bs=8k count=102400

102400+0 records in

102400+0 records out

838860800 bytes transferred in 29.715178 secs (28230045 bytes/sec)

       29.72 real         0.13 user         6.62 sys

MAC OS 10.7 作業系統（5400轉筆記本硬碟） 順序寫入測試

[email protected] ~$ dd if=/dev/zero of=bigfile bs=8k count=102400

102400+0 records in

102400+0 records out

838860800 bytes transferred in 16.930649 secs (49546878 bytes/sec)

寫測試請參考：

http://langui.sh/2011/04/02/using-dd-in-os-x/

http://askubuntu.com/questions/215262/dd-dev-disk4-permission-denied-error-when-making-liveusb-on-mac-os-x

---------------------------------------------------------------------------------------------------

近來想了解一下開發環境的IO效能，分別用dd/orion/iozone/bonnie++四種工具測試了一下 原文地址：http://space.itpub.net/441887/viewspace-630694 開發環境系統配置如下: Intel SR1625 server, 2 CPU, 32GB記憶體, 用主機板自帶卡做了raid1+0，8個7200轉SATA硬碟 作業系統是RHEL 5.3 64位 因為實體記憶體是32GB，因此整個過程都選用了60GB+的資料量來測試，以避免cache的影響 1. 首先用自帶的dd命令先測一下, 塊大小為8k ---------------------------------------- dd只能提供一個大概的測試結果，而且是連續IO而不是隨機IO 讀測試 # time dd if=/dev/sda2 f=/dev/null bs=8k count=8388608 8388608+0 records in 8388608+0 records out 68719476736 bytes (69 GB) copied, 516.547 seconds, 133 MB/s real    8m36.926s user    0m0.117s sys     0m55.216s 寫測試 # time dd if=/dev/zero f=/opt/iotest bs=8k count=8388608 8388608+0 records in 8388608+0 records out 68719476736 bytes (69 GB) copied, 888.398 seconds, 77.4 MB/s real    14m48.743s user    0m3.678s sys     2m47.158s 讀寫測試 # time dd if=/dev/sda2 f=/opt/iotest bs=8k count=8388608 8388608+0 records in 8388608+0 records out 68719476736 bytes (69 GB) copied, 1869.89 seconds, 36.8 MB/s real    31m10.343s user    0m2.613s sys     3m25.548s 2. 接下來用Oracle的orion工具來測一下 ---------------------------------------- 解壓即可使用 # gzip orion_linux_x86-64.gz 測非同步IO時需要libaio庫 # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib64 # echo $LD_LIBRARY_PATH :/opt/oracle/product/10.2.0/lib:/usr/lib64 建立配置檔案mytest.lun,列出要測試的分割槽即可. 注意檔名字首要跟下面的 testname一致 # vi mytest.lun 檢視mytest.jun內容 # cat mytest.lun /dev/sda2 先來個simple test # ./orion_linux_x86-64 -run simple -testname mytest -num_disks 8 檢視測試結果 # cat mytest_20081111_1431_summary.txt ORION VERSION 11.1.0.7.0 Commandline: -run simple -testname mytest -num_disks 8 This maps to this test: Test: mytest Small IO size: 8 KB Large IO size: 1024 KB IO Types: Small Random IOs, Large Random IOs Simulated Array Type: CONCAT Write: 0% Cache Size: Not Entered Duration for each Data Point: 60 seconds Small Columns:,      0 Large Columns:,      0,      1,      2,      3,      4,      5,      6,      7,      8,      9,     10,     11,     12,     13,     14,     15,     16 Total Data Points: 38 Name: /dev/sda2 Size: 629143441920 1 FILEs found. Maximum Large MBPS=56.97 @ Small=0 and Large=7 Maximum Small IOPS=442 @ Small=40 and Large=0 Minimum Small Latency=14.62 @ Small=1 and Large=0 最大MBPS為56.97，最大IOPS為442 再測一下8k隨機讀操作 # ./orion_linux_x86-64 -run advanced -testname mytest -num_disks 8 -size_small 8 -size_large 8 -type rand & 看看結果 # cat mytest_20081111_1519_summary.txt ORION VERSION 11.1.0.7.0 Commandline: -run advanced -testname mytest -num_disks 8 -size_small 8 -size_large 8 -type rand This maps to this test: Test: mytest Small IO size: 8 KB Large IO size: 8 KB IO Types: Small Random IOs, Large Random IOs Simulated Array Type: CONCAT Write: 0% Cache Size: Not Entered Duration for each Data Point: 60 seconds Small Columns:,      0 Large Columns:,      0,      1,      2,      3,      4,      5,      6,      7,      8,      9,     10,     11,     12,     13,     14,     15,     16 Total Data Points: 38 Name: /dev/sda2 Size: 629143441920 1 FILEs found. Maximum Large MBPS=3.21 @ Small=0 and Large=13 Maximum Small IOPS=448 @ Small=38 and Large=0 Minimum Small Latency=15.16 @ Small=1 and Large=0 最大MBPS為3.21(這麼低？？)，最大IOPS為448 再測一下1M順序讀操作, 失敗了, 原因不明... # ./orion_linux_x86-64 -run advanced -testname mytest -num_disks 8 -size_small 1024 -size_large 1024 -type seq ORION: ORacle IO Numbers -- Version 11.1.0.7.0 mytest_20081114_1349 Test will take approximately 73 minutes Larger caches may take longer rwbase_run_test: rwbase_reap_req failed rwbase_run_process: rwbase_run_test failed rwbase_rwluns: rwbase_run_process failed orion_warm_cache: Warming cache failed. Continuing 看看結果 # cat mytest_20081111_1620_summary.txt ORION VERSION 11.1.0.7.0 Commandline: -run advanced -testname mytest -num_disks 8 -size_small 1024 -size_large 1024 -type seq This maps to this test: Test: mytest Small IO size: 1024 KB Large IO size: 1024 KB IO Types: Small Random IOs, Large Sequential Streams Number of Concurrent IOs Per Stream: 4 Force streams to separate disks: No Simulated Array Type: CONCAT Write: 0% Cache Size: Not Entered Duration for each Data Point: 60 seconds 沒結果，失敗 ---------------------------------------- 安裝 # tar -xvf iozone3_345.tar # make linux-AMD64 指定64G的檔案，只測read/reread和write/rewrite，記錄大小從4k-16k.同時生成一個excel檔案iozone.wks # ./iozone -Rab iozone.wks -s64G -i 0 -i 1 -y 4k -q 16k         Iozone: Performance Test of File I/O                 Version $Revision: 3.345 $                 Compiled for 64 bit mode.                 Build: linux-AMD64         Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins                      Al Slater, Scott Rhine, Mike Wisner, Ken Goss                      Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,                      Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,                      Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,                      Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,                      Fabrice Bacchella, Zhenghua Xue, Qin Li.         Run began: Tue Nov 11 10:23:25 2008         Excel chart generation enabled         Auto Mode         File size set to 67108864 KB         Using Minimum Record Size 4 KB         Using Maximum Record Size 16 KB         Command line used: ./iozone -Rab iozone.wks -s64G -i 0 -i 1 -y 4k -q 16k         Output is in Kbytes/sec         Time Resolution = 0.000001 seconds.         Processor cache size set to 1024 Kbytes.         Processor cache line size set to 32 bytes.         File stride size set to 17 * record size.                                                             random  random    bkwd   record   stride                                                KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread         67108864       4   72882   69470   104898   125512         67108864       8   72083   69256   133689   109061         67108864      16   73375   69155   142019   116034 iozone test complete. Excel output is below: "Writer report"         "4"  "8"  "16" "67108864"   72882  72083  73375 "Re-writer report"         "4"  "8"  "16" "67108864"   69470  69256  69155 "Reader report"         "4"  "8"  "16" "67108864"   104898  133689  142019 "Re-Reader report"         "4"  "8"  "16" "67108864"   125512  109061  116034 可以看到，8k的寫是72M/s左右，讀是133M/s左右，跟dd的結果比較接近 測一下64G檔案8k隨機讀寫 # ./iozone -Rab iozone.wks -s64G -i 2 -y 8k -q 8k         Iozone: Performance Test of File I/O                 Version $Revision: 3.345 $                 Compiled for 64 bit mode.                 Build: linux-AMD64         Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins                      Al Slater, Scott Rhine, Mike Wisner, Ken Goss                      Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,                      Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,                      Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,                      Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,                      Fabrice Bacchella, Zhenghua Xue, Qin Li.         Run began: Fri Nov 14 15:52:01 2008         Excel chart generation enabled         Auto Mode         File size set to 67108864 KB         Using Minimum Record Size 8 KB         Using Maximum Record Size 8 KB         Command line used: ./iozone -Rab iozone.wks -s64G -i 2 -y 8k -q 8k         Output is in Kbytes/sec         Time Resolution = 0.000001 seconds.         Processor cache size set to 1024 Kbytes.         Processor cache line size set to 32 bytes.         File stride size set to 17 * record size.                                                             random  random    bkwd   record   stride                                                KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread         67108864       8 Error reading block at 6501007360 read: Success 出錯了(??) ---------------------------------------- 安裝 # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib64 # ./configure # make # make install 開始測試，預設檔案大小是記憶體的2倍 # bonnie++ -d /opt/IOTest/ -m sva17 -u root Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03e       ------Sequential Output------ --Sequential Input- --Random-                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP sva17           63G 52391  84 35222   7 34323   6 56362  88 131568  10 176.7   0                     ------Sequential Create------ --------Random Create--------                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ sva17,63G,52391,84,35222,7,34323,6,56362,88,131568,10,176.7,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++ 順序寫: 按字元是52391KB/s，CPU佔用率84%；按塊是35222KB/s，CPU佔用率7% 順序讀: 按字元是56362KB/s, CPU佔用率88%；按塊是131568KB/s，CPU佔用率10% 隨機讀寫: 176.7次/s，CPU佔用率0% 後兩項全是++ (沒結果?) 結論:不同測試工具構建出來的測試環境不同，側重點也不一樣，得到的結果可能相差比較大。 MBPS: dd和iozone比較接近，讀寫分別是130+和70+。 orion讀57左右，寫沒測(會刪掉分割槽內所有檔案!) bonnie++按塊讀是130左右，寫是35左右；按字元讀是56左右，寫是52左右 IOPS: dd 無結果 orion 440左右(只讀) iozone 出錯 bonnie++ 176.7 (讀寫)