Aliyun Linux上挂载NAS文件系统

摘录阿里云官网在Aliyun Linux上挂载NAS文件系统的各个步骤，汇聚在此。

参考资料

阿里云官方相关的文档链接:

挂载步骤

如下是在系统为Alibaba Cloud Enterprise Linux Server 17.01的ECS上挂载通用型NAS(性能型)的步骤。实际操作中，需要将如下命令中的file-system-id.region换为实际创建的NAS挂载点。

  1. 控制台创建nas

  2. ECS服务器安装nfs-utils工具

    1
    sudo yum install nfs-utils

  3. 挂载NAS磁盘到/home/deployer/carl_work/fio_test进行测试

    1
    2
    mkdir -p /home/deployer/carl_work/fio_test
    sudo mount -t nfs -o vers=3,nolock,proto=tcp,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport file-system-id.region.nas.aliyuncs.com:/ /home/deployer/carl_work/fio_test

  4. 修改nfs同时连接数
    root用户执行

    1
    2
    3
    4
    5
    6
    if (lsmod | grep sunrpc); then
    (modinfo sunrpc | grep tcp_max_slot_table_entries) && sysctl -w sunrpc.tcp_max_slot_table_entries=128
    (modinfo sunrpc | grep tcp_slot_table_entries) && sysctl -w sunrpc.tcp_slot_table_entries=128
    fi
    (modinfo sunrpc | grep tcp_max_slot_table_entries) && echo "options sunrpc tcp_max_slot_table_entries=128" >> /etc/modprobe.d/alinas.conf
    (modinfo sunrpc | grep tcp_slot_table_entries) && echo "options sunrpc tcp_slot_table_entries=128" >> /etc/modprobe.d/alinas.conf

    查看修改结果，如果为128说明起效。

    1
    cat /proc/sys/sunrpc/tcp_slot_table_entries

  5. 进行磁盘性能的测试
    参考NAS性能测试，测试命令使用fio, 使用/home/deployer/carl_work/fio_test目录进行测试。
    5.1 安装fio

    1
    sudo yum install fio

    5.2 测试命令

    • 随机读IOPS设置: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    • 随机写IOPS设置: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    • 随机读吞吐: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    • 随机写吞吐: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/

    5.3. 测试结果
    NAS和80G的普通EBS的读写性能测试如下:(详细的测试经过参见文章末尾)
    |比较指标|80G磁盘|NAS服务|
    |–|–|–|
    |随机读IOPS| IOPS 2463| IOPS=29.2k|
    |随机写IOPS| IOPS 2409| IOPS=16.7k|
    |随机读吞吐| BW=112MiB/s| BW=247MiB/s|
    |随机写吞吐| BW=112MiB/s| BW=111MiB/s|

  6. 创建实际挂载目录

    1
    sudo mkdir -p /mnt/mount_point

  7. 挂载nfs系统到目录中
    参考自: Linux系统挂载NFS文件系统

    1
    sudo mount -t nfs -o vers=3,nolock,proto=tcp,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport file-system-id.region.nas.aliyuncs.com:/ /mnt/mount_point

  8. 设置开机挂载NFS
    参考自: Linux系统挂载NFS文件系统
    修改/etc/fstab文件，添加如下内容

    1
    file-system-id.region.nas.aliyuncs.com:/ /mnt/mount_point nfs vers=3,nolock,proto=tcp,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,_netdev,noresvport,x-systemd.automount,x-systemd.requires=systemd-resolved.service,x-systemd.after=systemd-resolved.service 0 0

    根据官方文档Linux系统挂载NFS文件系统中的挂载说明，需要在/etc/rc.local中设置mount -a -t nfs。
    不设置的话，测试下来，挂载多个NAS盘的时候，只会挂载一个盘。

    1
    echo "mount -a -t nfs" >> /etc/rc.local;

  9. 验证开机挂载设置
    使用mount -afv来验证/etc/fstab是否正常工作，正常的话，reboot重启机器，查看重启后是否可以正常工作。

附录: 完整的FIO测试结果

80G EBS测试结果

  • 随机读IOPS设置。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: IOPS 2463

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    Fio: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Fio: Laying out IO file (1 file / 1024MiB)
    Jobs: 1 (f=1): [r(1)][100.0%][r=9856KiB/s,w=0KiB/s][r=2464,w=0 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=19964: Mon Oct 11 09:54:15 2021
       read: IOPS=2463, BW=9854KiB/s (10.1MB/s)(578MiB/60052msec)
        slat (nsec): min=1655, max=96266k, avg=13506.00, stdev=883584.74
        clat (usec): min=220, max=198310, avg=51941.52, stdev=47838.37
         lat (usec): min=227, max=198313, avg=51955.24, stdev=47837.80
        clat percentiles (usec):
         |  1.00th=[   635],  5.00th=[  1037], 10.00th=[  1369], 20.00th=[  1909],
         | 30.00th=[  2409], 40.00th=[  3163], 50.00th=[ 95945], 60.00th=[ 96994],
         | 70.00th=[ 98042], 80.00th=[ 98042], 90.00th=[ 99091], 95.00th=[100140],
         | 99.00th=[101188], 99.50th=[101188], 99.90th=[107480], 99.95th=[107480],
         | 99.99th=[196084]
       bw (  KiB/s): min= 8152, max=11984, per=100.00%, avg=9855.05, stdev=251.69, samples=120
       iops        : min= 2038, max= 2996, avg=2463.73, stdev=62.92, samples=120
      lat (usec)   : 250=0.01%, 500=0.34%, 750=1.53%, 1000=2.64%
      lat (msec)   : 2=17.52%, 4=23.25%, 10=2.62%, 20=0.09%, 50=0.12%
      lat (msec)   : 100=48.72%, 250=3.17%
      cpu          : usr=0.40%, sys=1.72%, ctx=41479, majf=0, minf=137
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=147942,0,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
       READ: bw=9854KiB/s (10.1MB/s), 9854KiB/s-9854KiB/s (10.1MB/s-10.1MB/s), io=578MiB (606MB), run=60052-60052msec
    Disk stats (read/write):
      vda: ios=147568/293, merge=0/130, ticks=7645416/10496, in_queue=7666640, util=99.87%

  • 随机写IOPS设置。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: IOPS 2409

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    Fio: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=9656KiB/s][r=0,w=2414 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=21976: Mon Oct 11 09:57:31 2021
      write: IOPS=2409, BW=9639KiB/s (9871kB/s)(565MiB/60018msec)
        slat (usec): min=2, max=90913, avg=10.30, stdev=582.11
        clat (usec): min=715, max=193557, avg=53103.59, stdev=45225.91
         lat (usec): min=723, max=193569, avg=53114.09, stdev=45225.33
        clat percentiles (msec):
         |  1.00th=[    3],  5.00th=[    4], 10.00th=[    5], 20.00th=[    5],
         | 30.00th=[    6], 40.00th=[    7], 50.00th=[   93], 60.00th=[   95],
         | 70.00th=[   95], 80.00th=[   96], 90.00th=[   97], 95.00th=[   97],
         | 99.00th=[  100], 99.50th=[  101], 99.90th=[  112], 99.95th=[  194],
         | 99.99th=[  194]
       bw (  KiB/s): min= 8680, max=11528, per=99.95%, avg=9634.45, stdev=377.05, samples=120
       iops        : min= 2170, max= 2882, avg=2408.59, stdev=94.27, samples=120
      lat (usec)   : 750=0.01%, 1000=0.01%
      lat (msec)   : 2=0.20%, 4=6.18%, 10=40.10%, 20=0.41%, 50=0.10%
      lat (msec)   : 100=52.48%, 250=0.53%
      cpu          : usr=0.39%, sys=1.42%, ctx=3836, majf=0, minf=9
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=0,144633,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
      WRITE: bw=9639KiB/s (9871kB/s), 9639KiB/s-9639KiB/s (9871kB/s-9871kB/s), io=565MiB (592MB), run=60018-60018msec
    Disk stats (read/write):
      vda: ios=1/195080, merge=0/1791, ticks=0/3220556, in_queue=3229740, util=99.31%
    [deployer@streaming fio_test]$

  • 随机读吞吐。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: BW=112MiB/s

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    Fio: (g=0): rw=randread, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Jobs: 1 (f=1): [r(1)][100.0%][r=9225KiB/s,w=0KiB/s][r=9,w=0 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=27365: Mon Oct 11 10:05:43 2021
       read: IOPS=112, BW=112MiB/s (118MB/s)(6849MiB/61083msec)
        slat (usec): min=65, max=206085, avg=5942.24, stdev=22601.07
        clat (msec): min=48, max=2286, avg=1134.96, stdev=167.64
         lat (msec): min=48, max=2286, avg=1140.91, stdev=167.84
        clat percentiles (msec):
         |  1.00th=[  472],  5.00th=[  986], 10.00th=[ 1070], 20.00th=[ 1099],
         | 30.00th=[ 1099], 40.00th=[ 1116], 50.00th=[ 1116], 60.00th=[ 1150],
         | 70.00th=[ 1183], 80.00th=[ 1200], 90.00th=[ 1217], 95.00th=[ 1301],
         | 99.00th=[ 1687], 99.50th=[ 1854], 99.90th=[ 2198], 99.95th=[ 2198],
         | 99.99th=[ 2299]
       bw (  KiB/s): min= 2048, max=221184, per=99.09%, avg=113777.44, stdev=20600.71, samples=121
       iops        : min=    2, max=  216, avg=111.11, stdev=20.12, samples=121
      lat (msec)   : 50=0.04%, 100=0.25%, 250=0.23%, 500=0.53%, 750=1.50%
      lat (msec)   : 1000=2.99%, 2000=94.09%, >=2000=0.37%
      cpu          : usr=0.04%, sys=1.19%, ctx=2844, majf=0, minf=585
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.5%, >=64=99.1%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=6849,0,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
       READ: bw=112MiB/s (118MB/s), 112MiB/s-112MiB/s (118MB/s-118MB/s), io=6849MiB (7182MB), run=61083-61083msec
    Disk stats (read/write):
      vda: ios=6825/253, merge=0/119, ticks=7425620/156144, in_queue=7606460, util=99.88%
    [deployer@streaming fio_test]$

  • 随机写吞吐。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/
    测试结果: BW=112MiB/s

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/
    Fio: (g=0): rw=randwrite, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Fio: Laying out IO file (1 file / 1024MiB)
    Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=99.1MiB/s][r=0,w=99 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=30203: Mon Oct 11 10:09:10 2021
      write: IOPS=112, BW=112MiB/s (118MB/s)(6768MiB/60349msec)
        slat (usec): min=87, max=999257, avg=1578.07, stdev=29103.92
        clat (msec): min=34, max=2213, avg=1139.56, stdev=195.51
         lat (msec): min=34, max=2214, avg=1141.14, stdev=195.47
        clat percentiles (msec):
         |  1.00th=[  600],  5.00th=[  852], 10.00th=[  919], 20.00th=[ 1099],
         | 30.00th=[ 1116], 40.00th=[ 1116], 50.00th=[ 1133], 60.00th=[ 1167],
         | 70.00th=[ 1167], 80.00th=[ 1183], 90.00th=[ 1200], 95.00th=[ 1603],
         | 99.00th=[ 1854], 99.50th=[ 1972], 99.90th=[ 2165], 99.95th=[ 2198],
         | 99.99th=[ 2198]
       bw (  KiB/s): min= 2048, max=278528, per=100.00%, avg=152799.08, stdev=82913.01, samples=89
       iops        : min=    2, max=  272, avg=149.21, stdev=80.97, samples=89
      lat (msec)   : 50=0.01%, 100=0.03%, 250=0.15%, 500=0.06%, 750=2.36%
      lat (msec)   : 1000=8.88%, 2000=88.30%, >=2000=0.21%
      cpu          : usr=1.15%, sys=1.11%, ctx=305, majf=0, minf=9
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.5%, >=64=99.1%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=0,6768,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
      WRITE: bw=112MiB/s (118MB/s), 112MiB/s-112MiB/s (118MB/s-118MB/s), io=6768MiB (7097MB), run=60349-60349msec
    Disk stats (read/write):
      vda: ios=1/7257, merge=0/338, ticks=660/2976536, in_queue=2980084, util=99.70%
    [deployer@streaming fio_test]$

NAS测试结果

  • 随机读IOPS设置。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: IOPS=29.2k

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    Fio: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Fio: Laying out IO file (1 file / 1024MiB)
    fio: native_fallocate call failed: Operation not supported
    Jobs: 1 (f=1): [r(1)][100.0%][r=116MiB/s,w=0KiB/s][r=29.7k,w=0 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=18130: Mon Oct 11 10:39:30 2021
       read: IOPS=29.2k, BW=114MiB/s (120MB/s)(6853MiB/60004msec)
        slat (nsec): min=1349, max=177040k, avg=12031.52, stdev=164024.33
        clat (usec): min=338, max=330112, avg=4364.10, stdev=3297.08
         lat (msec): min=2, max=330, avg= 4.38, stdev= 3.30
        clat percentiles (msec):
         |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
         | 30.00th=[    4], 40.00th=[    4], 50.00th=[    4], 60.00th=[    5],
         | 70.00th=[    5], 80.00th=[    6], 90.00th=[    7], 95.00th=[    9],
         | 99.00th=[   13], 99.50th=[   18], 99.90th=[   37], 99.95th=[   51],
         | 99.99th=[  122]
       bw (  KiB/s): min=48296, max=138352, per=99.99%, avg=116934.00, stdev=9975.45, samples=120
       iops        : min=12074, max=34588, avg=29233.50, stdev=2493.86, samples=120
      lat (usec)   : 500=0.01%, 1000=0.01%
      lat (msec)   : 2=0.01%, 4=59.62%, 10=38.19%, 20=1.80%, 50=0.34%
      lat (msec)   : 100=0.03%, 250=0.02%, 500=0.01%
      cpu          : usr=5.42%, sys=20.17%, ctx=2285170, majf=0, minf=136
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=1754318,0,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
       READ: bw=114MiB/s (120MB/s), 114MiB/s-114MiB/s (120MB/s-120MB/s), io=6853MiB (7186MB), run=60004-60004msec
    [deployer@streaming fio_test]$

  • 随机写IOPS设置。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: IOPS=16.7k

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=4K -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    Fio: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=67.4MiB/s][r=0,w=17.2k IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=19604: Mon Oct 11 10:41:50 2021
      write: IOPS=16.7k, BW=65.3MiB/s (68.5MB/s)(3918MiB/60022msec)
        slat (nsec): min=1720, max=97046k, avg=12147.27, stdev=105290.92
        clat (msec): min=3, max=393, avg= 7.65, stdev= 8.38
         lat (msec): min=3, max=393, avg= 7.66, stdev= 8.38
        clat percentiles (msec):
         |  1.00th=[    5],  5.00th=[    5], 10.00th=[    6], 20.00th=[    6],
         | 30.00th=[    6], 40.00th=[    7], 50.00th=[    7], 60.00th=[    7],
         | 70.00th=[    8], 80.00th=[    8], 90.00th=[   10], 95.00th=[   12],
         | 99.00th=[   38], 99.50th=[   56], 99.90th=[  121], 99.95th=[  159],
         | 99.99th=[  284]
       bw (  KiB/s): min=45256, max=76696, per=100.00%, avg=66854.28, stdev=6813.82, samples=120
       iops        : min=11314, max=19174, avg=16713.56, stdev=1703.43, samples=120
      lat (msec)   : 4=0.39%, 10=92.74%, 20=4.76%, 50=1.51%, 100=0.44%
      lat (msec)   : 250=0.15%, 500=0.01%
      cpu          : usr=3.33%, sys=11.64%, ctx=1274842, majf=0, minf=9
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=0,1003062,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
      WRITE: bw=65.3MiB/s (68.5MB/s), 65.3MiB/s-65.3MiB/s (68.5MB/s-68.5MB/s), io=3918MiB (4109MB), run=60022-60022msec
    [deployer@streaming fio_test]$

  • 随机读吞吐。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    测试结果: BW=247MiB/s

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randread -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/fio_test
    Fio: (g=0): rw=randread, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Jobs: 1 (f=1): [r(1)][100.0%][r=298MiB/s,w=0KiB/s][r=298,w=0 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=20952: Mon Oct 11 10:43:21 2021
       read: IOPS=247, BW=247MiB/s (259MB/s)(14.6GiB/60421msec)
        slat (usec): min=65, max=2906.5k, avg=327.01, stdev=23775.31
        clat (msec): min=14, max=11144, avg=516.90, stdev=878.02
         lat (msec): min=14, max=11145, avg=517.22, stdev=878.26
        clat percentiles (msec):
         |  1.00th=[   27],  5.00th=[   51], 10.00th=[  110], 20.00th=[  409],
         | 30.00th=[  422], 40.00th=[  430], 50.00th=[  435], 60.00th=[  435],
         | 70.00th=[  439], 80.00th=[  447], 90.00th=[  502], 95.00th=[  818],
         | 99.00th=[ 3910], 99.50th=[ 7483], 99.90th=[11073], 99.95th=[11073],
         | 99.99th=[11208]
       bw (  KiB/s): min= 1988, max=432128, per=100.00%, avg=270847.08, stdev=102807.00, samples=112
       iops        : min=    1, max=  422, avg=264.45, stdev=100.45, samples=112
      lat (msec)   : 20=0.15%, 50=4.77%, 100=4.64%, 250=3.29%, 500=77.02%
      lat (msec)   : 750=4.57%, 1000=1.50%, 2000=2.24%, >=2000=1.82%
      cpu          : usr=0.13%, sys=3.28%, ctx=28905, majf=0, minf=1095
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, >=64=99.6%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=14943,0,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
       READ: bw=247MiB/s (259MB/s), 247MiB/s-247MiB/s (259MB/s-259MB/s), io=14.6GiB (15.7GB), run=60421-60421msec
    [deployer@streaming fio_test]$

  • 随机写吞吐。
    命令: fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/
    测试结果: BW=111MiB/s

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    [deployer@streaming fio_test]$ fio -numjobs=1 -iodepth=128 -direct=1 -ioengine=libaio -sync=1 -rw=randwrite -bs=1M -size=1G -time_based -runtime=60 -name=Fio -directory=/home/deployer/carl_work/
    Fio: (g=0): rw=randwrite, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=128
    fio-3.1
    Starting 1 process
    Jobs: 1 (f=1): [w(1)][100.0%][r=0KiB/s,w=160MiB/s][r=0,w=160 IOPS][eta 00m:00s]
    Fio: (groupid=0, jobs=1): err= 0: pid=21639: Mon Oct 11 10:44:55 2021
      write: IOPS=111, BW=111MiB/s (117MB/s)(6726MiB/60490msec)
        slat (usec): min=84, max=631865, avg=1042.99, stdev=19536.13
        clat (msec): min=301, max=2046, avg=1149.90, stdev=161.34
         lat (msec): min=301, max=2046, avg=1150.94, stdev=162.22
        clat percentiles (msec):
         |  1.00th=[  835],  5.00th=[  902], 10.00th=[ 1070], 20.00th=[ 1116],
         | 30.00th=[ 1116], 40.00th=[ 1133], 50.00th=[ 1133], 60.00th=[ 1167],
         | 70.00th=[ 1183], 80.00th=[ 1183], 90.00th=[ 1200], 95.00th=[ 1418],
         | 99.00th=[ 1871], 99.50th=[ 1955], 99.90th=[ 2005], 99.95th=[ 2039],
         | 99.99th=[ 2039]
       bw (  KiB/s): min=22528, max=280576, per=100.00%, avg=139314.02, stdev=64292.99, samples=97
       iops        : min=   22, max=  274, avg=136.04, stdev=62.79, samples=97
      lat (msec)   : 500=0.19%, 750=0.45%, 1000=8.34%, 2000=90.57%, >=2000=0.45%
      cpu          : usr=1.10%, sys=1.16%, ctx=261, majf=0, minf=9
      IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=0.5%, >=64=99.1%
         submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
         complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
         issued rwt: total=0,6726,0, short=0,0,0, dropped=0,0,0
         latency   : target=0, window=0, percentile=100.00%, depth=128
    Run status group 0 (all jobs):
      WRITE: bw=111MiB/s (117MB/s), 111MiB/s-111MiB/s (117MB/s-117MB/s), io=6726MiB (7053MB), run=60490-60490msec
    Disk stats (read/write):
      vda: ios=255/7703, merge=0/1368, ticks=6396/2727104, in_queue=2739580, util=99.71%
    [deployer@streaming fio_test]$

留言

▲