Linux 下磁盘分区偏移对于磁盘性能的影响

野性酷女 2022-09-19 08:05 267阅读 0赞

操作环境：

OpenStack Havana

Centos 6.0

fio 2.1.2

本次通过centos6下磁盘分区的偏移不同，测试偏移不同对于磁盘性能的影响

首先使用默认创建磁盘分区方式（默认情况下偏移量为63 block），磁盘分区不对齐

libaio.dat: (groupid=0, jobs=1): err= 0: pid=4777: Thu Dec 19 03:14:48 2013
      read : io=6194.5MB, bw=10571KB/s, iops=165, runt=600052msec
        clat (usec): min=902, max=1535.4K, avg=3496.93, stdev=23268.84
         lat (usec): min=902, max=1535.4K, avg=3497.34, stdev=23268.84
        clat percentiles (usec):
         |  1.00th=[  956],  5.00th=[  972], 10.00th=[  988], 20.00th=[  996],
         | 30.00th=[ 1004], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],
         | 70.00th=[ 1080], 80.00th=[ 1176], 90.00th=[ 8160], 95.00th=[10048],
         | 99.00th=[13120], 99.50th=[42752], 99.90th=[374784], 99.95th=[561152],
         | 99.99th=[888832]
        bw (KB  /s): min=   63, max=32256, per=100.00%, avg=11429.35, stdev=8259.00
      write: io=6202.9MB, bw=10585KB/s, iops=165, runt=600052msec
        clat (msec): min=1, max=1231, avg= 2.53, stdev=16.29
         lat (msec): min=1, max=1231, avg= 2.54, stdev=16.29
        clat percentiles (usec):
         |  1.00th=[ 1080],  5.00th=[ 1096], 10.00th=[ 1112], 20.00th=[ 1128],
         | 30.00th=[ 1144], 40.00th=[ 1144], 50.00th=[ 1160], 60.00th=[ 1192],
         | 70.00th=[ 1208], 80.00th=[ 1240], 90.00th=[ 1496], 95.00th=[ 8512],
         | 99.00th=[15552], 99.50th=[18048], 99.90th=[228352], 99.95th=[362496],
         | 99.99th=[831488]
        bw (KB  /s): min=   53, max=32768, per=100.00%, avg=11476.66, stdev=8307.46
        lat (usec) : 1000=12.61%
        lat (msec) : 2=74.04%, 4=0.86%, 10=8.17%, 20=3.83%, 50=0.12%
        lat (msec) : 100=0.09%, 250=0.13%, 500=0.09%, 750=0.03%, 1000=0.01%
        lat (msec) : 2000=0.01%
      cpu          : usr=0.63%, sys=1.52%, ctx=199539, majf=0, minf=26
      IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         issued    : total=r=99111/w=99245/d=0, short=r=0/w=0/d=0
    
    Run status group 0 (all jobs):
       READ: io=6194.5MB, aggrb=10570KB/s, minb=10570KB/s, maxb=10570KB/s, mint=600052msec, maxt=600052msec
      WRITE: io=6202.9MB, aggrb=10585KB/s, minb=10585KB/s, maxb=10585KB/s, mint=600052msec, maxt=600052msec
    
    Disk stats (read/write):
      vdc: ios=99036/99425, merge=0/181, ticks=342033/465754, in_queue=810322, util=98.37%

在磁盘分区前63个block是用于存放MBR信息的，下面设置为偏移量为64 block进行性能测试

设置偏移量为64 block，操作如下：

[root@centos6 ~]# fdisk /dev/vdc
    
    WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
             switch off the mode (command 'c') and change display units to
             sectors (command 'u').
    
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-624152, default 1): 
    Using default value 1
    Last cylinder, +cylinders or +size{K,M,G} (1-624152, default 624152): 
    Using default value 624152
    
    Command (m for help): x
    
    Expert command (m for help): b
    Partition number (1-4): 1
    New beginning of data (1-629145215, default 63): 64
    
    Expert command (m for help): p
    
    Disk /dev/vdc: 16 heads, 63 sectors, 624152 cylinders
    
    Nr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID
     1 00   1   1    0  15  63 1023         64  629145152 83
     2 00   0   0    0   0   0    0          0          0 00
     3 00   0   0    0   0   0    0          0          0 00
     4 00   0   0    0   0   0    0          0          0 00

测试结果如下：

libaio.dat: (groupid=0, jobs=1): err= 0: pid=6786: Thu Dec 19 03:56:57 2013
      read : io=8240.4MB, bw=14063KB/s, iops=219, runt=600001msec
        clat (usec): min=913, max=1621.6K, avg=3340.33, stdev=23281.19
         lat (usec): min=913, max=1621.6K, avg=3340.74, stdev=23281.19
        clat percentiles (usec):
         |  1.00th=[  964],  5.00th=[  980], 10.00th=[  988], 20.00th=[  996],
         | 30.00th=[ 1012], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],
         | 70.00th=[ 1080], 80.00th=[ 1112], 90.00th=[ 7328], 95.00th=[ 9664],
         | 99.00th=[12864], 99.50th=[52480], 99.90th=[358400], 99.95th=[497664],
         | 99.99th=[897024]
        bw (KB  /s): min=   62, max=33792, per=100.00%, avg=15060.70, stdev=9208.06
      write: io=8228.9MB, bw=14044KB/s, iops=219, runt=600001msec
        clat (msec): min=1, max=906, avg= 1.19, stdev= 2.74
         lat (msec): min=1, max=906, avg= 1.20, stdev= 2.74
        clat percentiles (usec):
         |  1.00th=[ 1096],  5.00th=[ 1112], 10.00th=[ 1112], 20.00th=[ 1128],
         | 30.00th=[ 1144], 40.00th=[ 1144], 50.00th=[ 1160], 60.00th=[ 1176],
         | 70.00th=[ 1192], 80.00th=[ 1208], 90.00th=[ 1240], 95.00th=[ 1272],
         | 99.00th=[ 1368], 99.50th=[ 1640], 99.90th=[ 2704], 99.95th=[ 3472],
         | 99.99th=[ 8896]
        bw (KB  /s): min=   45, max=32000, per=100.00%, avg=15107.37, stdev=9207.78
        lat (usec) : 1000=10.87%
        lat (msec) : 2=81.83%, 4=0.45%, 10=4.71%, 20=1.80%, 50=0.07%
        lat (msec) : 100=0.08%, 250=0.08%, 500=0.07%, 750=0.02%, 1000=0.01%
        lat (msec) : 2000=0.01%
      cpu          : usr=0.85%, sys=2.00%, ctx=264954, majf=0, minf=26
      IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         issued    : total=r=131845/w=131662/d=0, short=r=0/w=0/d=0
    
    Run status group 0 (all jobs):
       READ: io=8240.4MB, aggrb=14063KB/s, minb=14063KB/s, maxb=14063KB/s, mint=600001msec, maxt=600001msec
      WRITE: io=8228.9MB, aggrb=14043KB/s, minb=14043KB/s, maxb=14043KB/s, mint=600001msec, maxt=600001msec
    
    Disk stats (read/write):
      vdc: ios=131804/131853, merge=0/177, ticks=434689/446135, in_queue=880684, util=98.54%

设置偏移量为128 block进行性能测试：

设置步骤如下：

[root@centos6 ~]# fdisk /dev/vdc
    
    WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
             switch off the mode (command 'c') and change display units to
             sectors (command 'u').
    
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-624152, default 1): 
    Using default value 1
    Last cylinder, +cylinders or +size{K,M,G} (1-624152, default 624152): 
    Using default value 624152
    
    Command (m for help): x
    
    Expert command (m for help): b
    Partition number (1-4): 1
    New beginning of data (1-629145215, default 63): 128
    
    Expert command (m for help): p
    
    Disk /dev/vdc: 16 heads, 63 sectors, 624152 cylinders
    
    Nr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID
     1 00   1   1    0  15  63 1023        128  629145088 83
     2 00   0   0    0   0   0    0          0          0 00
     3 00   0   0    0   0   0    0          0          0 00
     4 00   0   0    0   0   0    0          0          0 00
    
    Expert command (m for help): w
    The partition table has been altered!
    
    Calling ioctl() to re-read partition table.

测试结果如下：

libaio.dat: Laying out IO file(s) (1 file(s) / 4096MB)
    Jobs: 1 (f=1): [m] [100.0% done] [26752KB/30528KB/0KB /s] [418/477/0 iops] [eta 00m:00s]
    libaio.dat: (groupid=0, jobs=1): err= 0: pid=7836: Thu Dec 19 04:29:07 2013
      read : io=10562MB, bw=18025KB/s, iops=281, runt=600002msec
        clat (usec): min=907, max=855615, avg=2343.27, stdev=12381.39
         lat (usec): min=907, max=855616, avg=2343.68, stdev=12381.39
        clat percentiles (usec):
         |  1.00th=[  956],  5.00th=[  980], 10.00th=[  988], 20.00th=[  996],
         | 30.00th=[ 1012], 40.00th=[ 1020], 50.00th=[ 1032], 60.00th=[ 1048],
         | 70.00th=[ 1064], 80.00th=[ 1096], 90.00th=[ 4128], 95.00th=[ 8384],
         | 99.00th=[11328], 99.50th=[16512], 99.90th=[183296], 99.95th=[292864],
         | 99.99th=[489472]
        bw (KB  /s): min=  114, max=33024, per=100.00%, avg=18514.19, stdev=9072.17
      write: io=10531MB, bw=17973KB/s, iops=280, runt=600002msec
        clat (usec): min=1046, max=277697, avg=1184.54, stdev=948.23
         lat (usec): min=1056, max=277713, avg=1196.02, stdev=948.30
        clat percentiles (usec):
         |  1.00th=[ 1080],  5.00th=[ 1112], 10.00th=[ 1112], 20.00th=[ 1128],
         | 30.00th=[ 1144], 40.00th=[ 1160], 50.00th=[ 1160], 60.00th=[ 1176],
         | 70.00th=[ 1192], 80.00th=[ 1208], 90.00th=[ 1240], 95.00th=[ 1272],
         | 99.00th=[ 1368], 99.50th=[ 1816], 99.90th=[ 2640], 99.95th=[ 3056],
         | 99.99th=[ 8256]
        bw (KB  /s): min=  118, max=32000, per=100.00%, avg=18474.97, stdev=9036.99
        lat (usec) : 1000=10.97%
        lat (msec) : 2=83.13%, 4=0.82%, 10=3.68%, 20=1.18%, 50=0.06%
        lat (msec) : 100=0.06%, 250=0.07%, 500=0.03%, 750=0.01%, 1000=0.01%
      cpu          : usr=1.10%, sys=2.59%, ctx=339298, majf=0, minf=25
      IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         issued    : total=r=168987/w=168497/d=0, short=r=0/w=0/d=0
    
    Run status group 0 (all jobs):
       READ: io=10562MB, aggrb=18025KB/s, minb=18025KB/s, maxb=18025KB/s, mint=600002msec, maxt=600002msec
      WRITE: io=10531MB, aggrb=17972KB/s, minb=17972KB/s, maxb=17972KB/s, mint=600002msec, maxt=600002msec
    
    Disk stats (read/write):
      vdc: ios=168874/168617, merge=0/145, ticks=388717/445504, in_queue=838295, util=98.15%

测试分析：

从测试结果可以得出结论是，在数据请求快大小为64KB的情况下，磁盘分区偏移量在128 block的时候性能是最好的，默认 63 block的时候是最差的。