disk_benckmark - charlesfg/TPCx-V_setup GitHub Wiki

Host

  • /dev/dm-1 - 4G r75% w25% - iops ~ 1.3K
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.1.3
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 4096MB)
Jobs: 1 (f=1): [m] [100.0% done] [4771KB/1550KB/0KB /s] [1192/387/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=7174: Tue Feb  7 08:59:06 2017
  read : io=3071.7MB, bw=4040.1KB/s, iops=1010, runt=778378msec
  write: io=1024.4MB, bw=1347.6KB/s, iops=336, runt=778378msec
  cpu          : usr=1.73%, sys=7.26%, ctx=944011, majf=0, minf=7
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=786347/w=262229/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=3071.7MB, aggrb=4040KB/s, minb=4040KB/s, maxb=4040KB/s, mint=778378msec, maxt=778378msec
  WRITE: io=1024.4MB, aggrb=1347KB/s, minb=1347KB/s, maxb=1347KB/s, mint=778378msec, maxt=778378msec

Disk stats (read/write):
    dm-1: ios=787122/262833, merge=0/0, ticks=49884180/116492, in_queue=50007664, util=100.00%, aggrios=799723/304818, aggrmerge=56/3595, aggrticks=56505660/427656, aggrin_queue=56938168, aggrutil=100.00%
  sda: ios=799723/304818, merge=56/3595, ticks=56505660/427656, in_queue=56938168, util=100.00%

  • /dev/dm-1 - 1G r75% w25% - iops ~ 1.3K
root@oxum:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.1.3
Starting 1 process
Jobs: 1 (f=1): [m] [100.0% done] [3864KB/1292KB/0KB /s] [966/323/0 iops] [eta 00m:00s] 
test: (groupid=0, jobs=1): err= 0: pid=6538: Tue Feb  7 09:42:23 2017
  read : io=784996KB, bw=4431.6KB/s, iops=1107, runt=177137msec
  write: io=263580KB, bw=1488.0KB/s, iops=372, runt=177137msec
  cpu          : usr=1.65%, sys=7.45%, ctx=234054, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=4431KB/s, minb=4431KB/s, maxb=4431KB/s, mint=177137msec, maxt=177137msec
  WRITE: io=263580KB, aggrb=1488KB/s, minb=1488KB/s, maxb=1488KB/s, mint=177137msec, maxt=177137msec

Disk stats (read/write):
    dm-1: ios=196249/66086, merge=0/0, ticks=11230108/76380, in_queue=11307028, util=100.00%, aggrios=196338/98752, aggrmerge=8/3884, aggrticks=11283908/346680, aggrin_queue=11630868, aggrutil=100.00%
  sda: ios=196338/98752, merge=8/3884, ticks=11283908/346680, in_queue=11630868, util=100.00%

  • asd
root@oxum:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test.delete --filename=/var/test.delete --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test.delete: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.1.3
Starting 1 process
test.delete: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m] [99.5% done] [4599KB/1418KB/0KB /s] [1149/354/0 iops] [eta 00m:01s]
test.delete: (groupid=0, jobs=1): err= 0: pid=14285: Tue Feb  7 09:53:08 2017
  read : io=784996KB, bw=4288.6KB/s, iops=1072, runt=183045msec
  write: io=263580KB, bw=1439.1KB/s, iops=359, runt=183045msec
  cpu          : usr=1.49%, sys=6.89%, ctx=235561, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=4288KB/s, minb=4288KB/s, maxb=4288KB/s, mint=183045msec, maxt=183045msec
  WRITE: io=263580KB, aggrb=1439KB/s, minb=1439KB/s, maxb=1439KB/s, mint=183045msec, maxt=183045msec

Disk stats (read/write):
    dm-2: ios=196100/99667, merge=0/0, ticks=11590684/310164, in_queue=11903256, util=100.00%, aggrios=196400/98045, aggrmerge=8/3718, aggrticks=11764188/264828, aggrin_queue=12029268, aggrutil=100.00%
  sda: ios=196400/98045, merge=8/3718, ticks=11764188/264828, in_queue=12029268, util=100.00%

VM

  • tpc-g4b1
➜ (root@tpc-g4b1)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75 
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [3756KB/1294KB/0KB /s] [939/323/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=6255: Tue Feb  7 09:56:58 2017
  read : io=784996KB, bw=3682.1KB/s, iops=920, runt=213145msec
  write: io=263580KB, bw=1236.7KB/s, iops=309, runt=213145msec
  cpu          : usr=1.44%, sys=5.62%, ctx=218482, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=3682KB/s, minb=3682KB/s, maxb=3682KB/s, mint=213145msec, maxt=213145msec
  WRITE: io=263580KB, aggrb=1236KB/s, minb=1236KB/s, maxb=1236KB/s, mint=213145msec, maxt=213145msec

Disk stats (read/write):
  xvda: ios=194710/65845, merge=1435/308, ticks=11643252/1855927, in_queue=13500801, util=100.00%

Latency:

root@oxum:~# runatall 'ioping -q -c 10 .'
tpc-g1a

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 3.60 s, 36 KiB read, 2 iops, 10.0 KiB/s
generated 10 requests in 9.55 s, 40 KiB, 1 iops, 4.19 KiB/s
min/avg/max/mdev = 15.2 ms / 399.5 ms / 859.4 ms / 264.0 ms
tpc-g1b1

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 6.00 s, 36 KiB read, 1 iops, 6.00 KiB/s
generated 10 requests in 9.54 s, 40 KiB, 1 iops, 4.19 KiB/s
min/avg/max/mdev = 244.5 ms / 666.1 ms / 1.20 s / 309.3 ms
tpc-g1b2

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 5.17 s, 36 KiB read, 1 iops, 6.96 KiB/s
generated 10 requests in 10.5 s, 40 KiB, 0 iops, 3.82 KiB/s
min/avg/max/mdev = 143.1 ms / 574.3 ms / 1.46 s / 386.6 ms
tpc-g2a

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 3.54 s, 36 KiB read, 2 iops, 10.2 KiB/s
generated 10 requests in 9.60 s, 40 KiB, 1 iops, 4.17 KiB/s
min/avg/max/mdev = 1.74 ms / 393.6 ms / 599.3 ms / 202.4 ms
tpc-g2b1

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.51 s, 36 KiB read, 1 iops, 7.99 KiB/s
generated 10 requests in 9.67 s, 40 KiB, 1 iops, 4.14 KiB/s
min/avg/max/mdev = 106.3 ms / 500.6 ms / 809.8 ms / 229.6 ms
tpc-g2b2

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.16 s, 36 KiB read, 2 iops, 8.65 KiB/s
generated 10 requests in 9.42 s, 40 KiB, 1 iops, 4.25 KiB/s
min/avg/max/mdev = 230.6 ms / 462.4 ms / 1.15 s / 262.0 ms
tpc-g3a

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.36 s, 36 KiB read, 2 iops, 8.25 KiB/s
generated 10 requests in 9.26 s, 40 KiB, 1 iops, 4.32 KiB/s
min/avg/max/mdev = 221.0 ms / 484.6 ms / 1.03 s / 232.9 ms
tpc-g3b1

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.61 s, 36 KiB read, 1 iops, 7.81 KiB/s
generated 10 requests in 9.88 s, 40 KiB, 1 iops, 4.05 KiB/s
min/avg/max/mdev = 64.3 ms / 512.0 ms / 1.16 s / 310.6 ms
tpc-g3b2

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 3.85 s, 36 KiB read, 2 iops, 9.36 KiB/s
generated 10 requests in 9.29 s, 40 KiB, 1 iops, 4.31 KiB/s
min/avg/max/mdev = 286.5 ms / 427.3 ms / 814.6 ms / 170.7 ms
tpc-g4a

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 8.71 s, 36 KiB read, 1 iops, 4.13 KiB/s
generated 10 requests in 11.5 s, 40 KiB, 0 iops, 3.48 KiB/s
min/avg/max/mdev = 315.7 ms / 968.2 ms / 2.24 s / 567.6 ms
tpc-g4b1

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.78 s, 36 KiB read, 1 iops, 7.53 KiB/s
generated 10 requests in 10.9 s, 40 KiB, 0 iops, 3.66 KiB/s
min/avg/max/mdev = 207.1 ms / 531.1 ms / 1.23 s / 297.4 ms
tpc-g4b2

--- . (ext4 /dev/xvda1) ioping statistics ---
9 requests completed in 4.30 s, 36 KiB read, 2 iops, 8.37 KiB/s
generated 10 requests in 9.27 s, 40 KiB, 1 iops, 4.31 KiB/s
min/avg/max/mdev = 150.7 ms / 478.0 ms / 890.1 ms / 221.8 ms

All VMs at the same time:

➜ (root@tpc-drive)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [99.6% done] [4043KB/1310KB/0KB /s] [1010/327/0 iops] [eta 00m:10s]
test: (groupid=0, jobs=1): err= 0: pid=13978: Tue Feb  7 10:53:14 2017
  read : io=784996KB, bw=299758B/s, iops=73, runt=2681611msec
  write: io=263580KB, bw=100650B/s, iops=24, runt=2681611msec
  cpu          : usr=0.09%, sys=0.36%, ctx=185095, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=292KB/s, minb=292KB/s, maxb=292KB/s, mint=2681611msec, maxt=2681611msec
  WRITE: io=263580KB, aggrb=98KB/s, minb=98KB/s, maxb=98KB/s, mint=2681611msec, maxt=2681611msec

Disk stats (read/write):
  xvda: ios=195106/68009, merge=2312/2655, ticks=126436799/38531869, in_queue=164968721, util=100.00%

➜ (root@tpc-g1a)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=9283: Tue Feb  7 10:48:49 2017
  read : io=784996KB, bw=283397B/s, iops=69, runt=2836421msec
  write: io=263580KB, bw=95157B/s, iops=23, runt=2836421msec
  cpu          : usr=0.09%, sys=0.35%, ctx=192374, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=276KB/s, minb=276KB/s, maxb=276KB/s, mint=2836421msec, maxt=2836421msec
  WRITE: io=263580KB, aggrb=92KB/s, minb=92KB/s, maxb=92KB/s, mint=2836421msec, maxt=2836421msec

Disk stats (read/write):
  xvda: ios=195616/67912, merge=2336/2252, ticks=138165611/38589748, in_queue=176757984, util=100.00%

➜ (root@tpc-g1b1)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [267KB/51KB/0KB /s] [66/12/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3763: Tue Feb  7 10:49:59 2017
  read : io=784996KB, bw=277333B/s, iops=67, runt=2898447msec
  write: io=263580KB, bw=93120B/s, iops=22, runt=2898447msec
  cpu          : usr=0.09%, sys=0.35%, ctx=188687, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=270KB/s, minb=270KB/s, maxb=270KB/s, mint=2898447msec, maxt=2898447msec
  WRITE: io=263580KB, aggrb=90KB/s, minb=90KB/s, maxb=90KB/s, mint=2898447msec, maxt=2898447msec

Disk stats (read/write):
  xvda: ios=194909/67797, merge=1348/2136, ticks=138506114/39495796, in_queue=178004020, util=100.00%

➜ (root@tpc-g1b2)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [147KB/0KB/0KB /s] [36/0/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=20936: Tue Feb  7 10:50:55 2017
  read : io=784996KB, bw=273420B/s, iops=66, runt=2939930msec
  write: io=263580KB, bw=91806B/s, iops=22, runt=2939930msec
  cpu          : usr=0.08%, sys=0.33%, ctx=186801, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=267KB/s, minb=267KB/s, maxb=267KB/s, mint=2939930msec, maxt=2939930msec
  WRITE: io=263580KB, aggrb=89KB/s, minb=89KB/s, maxb=89KB/s, mint=2939930msec, maxt=2939930msec

Disk stats (read/write):
  xvda: ios=194936/67781, merge=1326/2063, ticks=139315061/40709197, in_queue=180032667, util=100.00%

➜ (root@tpc-g2a)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=9591: Tue Feb  7 10:50:59 2017
  read : io=784996KB, bw=272878B/s, iops=66, runt=2945770msec
  write: io=263580KB, bw=91624B/s, iops=22, runt=2945770msec
  cpu          : usr=0.08%, sys=0.33%, ctx=191387, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=266KB/s, minb=266KB/s, maxb=266KB/s, mint=2945770msec, maxt=2945770msec
  WRITE: io=263580KB, aggrb=89KB/s, minb=89KB/s, maxb=89KB/s, mint=2945770msec, maxt=2945770msec

Disk stats (read/write):
  xvda: ios=195555/67852, merge=2728/2139, ticks=142795908/40780010, in_queue=183580344, util=100.00%



➜ (root@tpc-g2b1)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [239KB/59KB/0KB /s] [59/14/0 iops] [eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=3238: Tue Feb  7 10:51:48 2017
  read : io=784996KB, bw=270772B/s, iops=66, runt=2968676msec
  write: io=263580KB, bw=90917B/s, iops=22, runt=2968676msec
  cpu          : usr=0.08%, sys=0.34%, ctx=186256, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2968676msec, maxt=2968676msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2968676msec, maxt=2968676msec

Disk stats (read/write):
  xvda: ios=195086/67918, merge=1348/2192, ticks=140325717/41736036, in_queue=182065377, util=100.00%


➜ (root@tpc-g2b2)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [387KB/167KB/0KB /s] [96/41/0 iops] [eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=3873: Tue Feb  7 10:52:11 2017
  read : io=784996KB, bw=270691B/s, iops=66, runt=2969569msec
  write: io=263580KB, bw=90890B/s, iops=22, runt=2969569msec
  cpu          : usr=0.08%, sys=0.33%, ctx=188790, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2969569msec, maxt=2969569msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2969569msec, maxt=2969569msec

Disk stats (read/write):
  xvda: ios=194846/67800, merge=1399/2192, ticks=142402398/41510487, in_queue=183918198, util=100.00%

➜ (root@tpc-g3a)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [480KB/132KB/0KB /s] [120/33/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=9035: Tue Feb  7 10:52:15 2017
  read : io=784996KB, bw=270215B/s, iops=65, runt=2974801msec
  write: io=263580KB, bw=90730B/s, iops=22, runt=2974801msec
  cpu          : usr=0.08%, sys=0.33%, ctx=189465, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=263KB/s, minb=263KB/s, maxb=263KB/s, mint=2974801msec, maxt=2974801msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2974801msec, maxt=2974801msec

Disk stats (read/write):
  xvda: ios=195407/67923, merge=2397/2424, ticks=143792552/41754389, in_queue=185548790, util=100.00%

➜ (root@tpc-g3b1)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [0KB/0KB/0KB /s] [0/0/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3214: Tue Feb  7 10:52:33 2017
  read : io=784996KB, bw=270566B/s, iops=66, runt=2970939msec
  write: io=263580KB, bw=90848B/s, iops=22, runt=2970939msec
  cpu          : usr=0.08%, sys=0.34%, ctx=187315, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2970939msec, maxt=2970939msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2970939msec, maxt=2970939msec

Disk stats (read/write):
  xvda: ios=194864/67840, merge=1394/2245, ticks=141341836/41409427, in_queue=182752730, util=100.00%


➜ (root@tpc-g4a)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [260KB/76KB/0KB /s] [65/19/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=11795: Tue Feb  7 10:52:36 2017
  read : io=784996KB, bw=270405B/s, iops=66, runt=2972708msec
  write: io=263580KB, bw=90794B/s, iops=22, runt=2972708msec
  cpu          : usr=0.08%, sys=0.34%, ctx=189029, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2972708msec, maxt=2972708msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2972708msec, maxt=2972708msec

Disk stats (read/write):
  xvda: ios=195342/67908, merge=1990/2490, ticks=143224565/41559860, in_queue=184787264, util=100.00%

➜ (root@tpc-g4b1)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
Jobs: 1 (f=1): [m(1)] [100.0% done] [163KB/39KB/0KB /s] [40/9/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=6558: Tue Feb  7 10:51:41 2017
  read : io=784996KB, bw=270390B/s, iops=66, runt=2972872msec
  write: io=263580KB, bw=90789B/s, iops=22, runt=2972872msec
  cpu          : usr=0.08%, sys=0.35%, ctx=189080, majf=0, minf=29
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2972872msec, maxt=2972872msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2972872msec, maxt=2972872msec

Disk stats (read/write):
  xvda: ios=194897/67866, merge=1361/2288, ticks=142005754/41662773, in_queue=183670464, util=100.00%


➜ (root@tpc-g4b2)/root fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.2.8
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [1926KB/615KB/0KB /s] [481/153/0 iops] [eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=6964: Tue Feb  7 10:52:41 2017
  read : io=784996KB, bw=271274B/s, iops=66, runt=2963184msec
  write: io=263580KB, bw=91086B/s, iops=22, runt=2963184msec
  cpu          : usr=0.08%, sys=0.34%, ctx=189001, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=196249/w=65895/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=784996KB, aggrb=264KB/s, minb=264KB/s, maxb=264KB/s, mint=2963184msec, maxt=2963184msec
  WRITE: io=263580KB, aggrb=88KB/s, minb=88KB/s, maxb=88KB/s, mint=2963184msec, maxt=2963184msec

Disk stats (read/write):
  xvda: ios=194820/67757, merge=1372/2287, ticks=142082259/40608563, in_queue=182694483, util=100.00%