一、壓測工具

通過 Linux dd 命令測試

二、IO 測試

• 關於 IO 的限制

  • --blkio-weight=0
    • Block IO weight (relative weight) accepts a weight value between 10 and 1000.
  • --blkio-weight-device=""
    • Block IO weight (relative device weight, format: DEVICE_NAME:WEIGHT)
    • 針對特定裝置的權重比
  • --device-read-bps=""
    • Limit read rate from a device (format: <device-path>:<number>[<unit>]
      ). Number is a positive integer. Unit can be one of kb, mb, or gb.
    • 按每秒讀取塊裝置的資料量設定上限
  • --device-write-bps=""
    • Limit write rate from a device (format: <device-path>:<number>[<unit>]). Number is a positive integer. Unit can be one of kb, mb, or gb.
    • 按每秒寫入塊裝置的資料量設定上限
  • --device-read-iops=""
    • Limit read rate (IO per second) from a device (format: <device-path>:<number>
      ). Number is a positive integer.
    • 按照每秒讀操作次數設定上限
  • --device-write-iops=""
    • Limit write rate (IO per second) from a device (format: <device-path>:<number>). Number is a positive integer.
    • 按照每秒寫操作次數設定上限

➜  ~ docker help run | grep -E 'bps|IO'
Usage:  docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
  --blkio-weight                  Block IO (relative weight), between 10 and 1000
 

  --blkio-weight-device=[]        Block IO weight (relative device weight)
  --device-read-bps=[]            Limit read rate (bytes per second) from a device
  --device-read-iops=[]           Limit read rate (IO per second) from a device
  --device-write-bps=[]           Limit write rate (bytes per second) to a device
  --device-write-iops=[]          Limit write rate (IO per second) to a device
➜  ~

2.1 --blkio-weight、--blkio-weight-device

• --blkio-weight

預設，所有的容器對於 IO 操作「block IO bandwidth – blkio」都擁有相同優先順序。可以通過 --blkio-weight 修改容器 blkio 權重。--blkio-weight 權重值在 10 ~ 1000 之間。

Note: The blkio weight setting is only available for direct IO. Buffered IO is not currently supported.

使用 blkio weight 還需要注意 IO 的排程必須為 CFQ：

➜  ~ cat /sys/block/sda/queue/scheduler
noop [deadline] cfq
➜  ~ sudo sh -c "echo cfq > /sys/block/sda/queue/scheduler"
➜  ~ cat /sys/block/sda/queue/scheduler
noop deadline [cfq]

按照 Docker 官方文件的介紹測試：

➜  ~ docker run -it --rm --blkio-weight 100 ubuntu-stress:latest /bin/bash
[email protected]0b6770ee80e0:/#
➜  ~ docker run -it --rm --blkio-weight 1000 ubuntu-stress:latest /bin/bash
[email protected]6778b6b39686:/#

在執行的容器上同時執行如下命令，統計測試時間：

[email protected]0b6770ee80e0:/# time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 122.442 s, 8.8 MB/s

real    2m2.524s
user    0m0.008s
sys     0m0.492s
[email protected]6778b6b39686:/# time dd if=/dev/zero of=test.out bs=1M count=1024 oflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 122.493 s, 8.8 MB/s

real    2m2.574s
user    0m0.020s
sys     0m0.480s
[email protected]6778b6b39686:/#

官方的測試說明是：

You’ll find that the proportion of time is the same as the proportion of blkio weights of the two containers.

• --blkio-weight-device="DEVICE_NAME:WEIGHT"

--blkio-weight-device 可以指定某個裝置的權重大小，如果同時指定 --blkio-weight 則以 --blkio-weight 為全域性預設配置，針對指定裝置以 --blkio-weight-device 指定裝置值為主。

➜  ~ docker run -it --rm --blkio-weight-device "/dev/sda:100" ubuntu-stress:latest /bin/bash

2.2 --device-read-bps、--device-write-bps

限制容器的寫入速度是 1mb「<device-path>:<limit>[unit]，單位可以是 kb、mb、gb 正整數」:

➜  ~ docker run -it --rm --device-write-bps /dev/sda:1mb ubuntu-stress:latest /bin/bash
[email protected]:/# dd if=/dev/zero of=test.out bs=1M count=100 oflag=direct
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 100.064 s, 1.0 MB/s    # 可以得知寫入的平均速度是 1.0 MB/s

通過 iotop 獲取測試過程中的 bps 也是 1.0 MB 為上限：

讀 bps 限制使用方式同寫 bps 限制：

➜  ~ docker run -it --rm --device-read-bps /dev/sda:1mb ubuntu-stress:latest /bin/bash

2.3 --device-read-iops、--device-write-iops

限制容器 write iops 為 5「<device-path>:<limit>，必須為正整數」：

➜  ~ docker run -it --rm --device-write-iops /dev/sda:5 ubuntu-stress:latest /bin/bash
[email protected]:/# dd if=/dev/zero of=test.out bs=1M count=100 oflag=direct
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 42.6987 s, 2.5 MB/s

通過 iostat 監控 tps「此處即為 iops」 基本上持續在 10 左右「會有些偏差」：

➜  ~ iostat 1
... ...
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.13    0.00    0.13   23.46    0.00   75.28

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sda              10.00         0.00      2610.00          0       5220
... ...

讀 iops 限制使用方式同寫 iops 限制：

➜  ~ docker run -it --rm --device-read-iops /dev/sda:5 ubuntu-stress:latest /bin/bash

注： 在容器中通過 dd 測試讀速度並沒有看到很好的效果，經查沒有找到磁碟讀操作的好工具，所以文中沒有介紹讀測試。

三、原始碼解析

libcontainer 主要操作是對 cgroup 下相關檔案根據選項寫操作，具體更進一步的資源限制操作可以看 cgroup 的實現方式。

本文轉自:http://blog.opskumu.com/docker-io-limit.html