3 ms·

4027.4MB/s (~32Gbps) with 100GB of random writes isn’t bad at all. Maybe you’re just maxing out the SSDs themselves, not the platform around it?

devereaux 9y ago
Unfortunately, with md6 (using 6 drives) the performance start degrading. It is perfectly reproductible: (...) Run status group 0 (all jobs): WRITE: io=963949MB, aggrb=4016.5MB/s, minb=4016.5MB/s, maxb=4016.5MB/s, mint=240001msec, maxt=240001msec Disk stats (read/write): md6: ios=0/246593949, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=0/41128473, aggrmerge=0/0, aggrticks=0/4307426, aggrin_queue=4365219, aggrutil=89.21% nvme2n1: ios=0/41137801, merge=0/0, ticks=0/2860992, in_queue=2909124, util=81.31% nvme3n1: ios=0/41125375, merge=0/0, ticks=0/2861640, in_queue=2896800, util=80.94% nvme0n1: ios=0/41136785, merge=0/0, ticks=0/7713176, in_queue=7809532, util=89.21% nvme4n1: ios=0/41142237, merge=0/0, ticks=0/3141888, in_queue=3185880, util=81.73% nvme1n1: ios=0/41116666, merge=0/0, ticks=0/6495928, in_queue=6578204, util=87.12% nvme5n1: ios=0/41111977, merge=0/0, ticks=0/2770936, in_queue=2811776, util=81.20% I would welcome any idea. Tell me your email if you want to play with the machine (there's nothing there yet, I was just setting up md5 for the next step: pgsql benchmarks)
- user5994461 9y ago
  It's not degrading, it's the same. You've reached a ceiling on something. Try increasing the queue from 16 to 64.
  - devereaux 9y ago
    Just tried changing a few things with a different benchmark after reading http://middoraid.blogspot.com/2013/01/tweaking.html http://middoraid.blogspot.com/2013/01/tweaking.html, no change.
    - user5994461 9y ago
      Stopped reading after the title. RAID5 is long dead. It's too sensitive to data loss and it has performance issues. Use RAID 1 or RAID 10.
      - devereaux 9y ago
        I can afford to lose the data. I'm doing RAID0. The article was interesting for presenting simple tweaks and a nice table of the results.

devereaux 9y ago

Unfortunately, with md6 (using 6 drives) the performance start degrading. It is perfectly reproductible: (...) Run status group 0 (all jobs): WRITE: io=963949MB, aggrb=4016.5MB/s, minb=4016.5MB/s, maxb=4016.5MB/s, mint=240001msec, maxt=240001msec Disk stats (read/write): md6: ios=0/246593949, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=0/41128473, aggrmerge=0/0, aggrticks=0/4307426, aggrin_queue=4365219, aggrutil=89.21% nvme2n1: ios=0/41137801, merge=0/0, ticks=0/2860992, in_queue=2909124, util=81.31% nvme3n1: ios=0/41125375, merge=0/0, ticks=0/2861640, in_queue=2896800, util=80.94% nvme0n1: ios=0/41136785, merge=0/0, ticks=0/7713176, in_queue=7809532, util=89.21% nvme4n1: ios=0/41142237, merge=0/0, ticks=0/3141888, in_queue=3185880, util=81.73% nvme1n1: ios=0/41116666, merge=0/0, ticks=0/6495928, in_queue=6578204, util=87.12% nvme5n1: ios=0/41111977, merge=0/0, ticks=0/2770936, in_queue=2811776, util=81.20% I would welcome any idea. Tell me your email if you want to play with the machine (there's nothing there yet, I was just setting up md5 for the next step: pgsql benchmarks)

user5994461 9y ago
It's not degrading, it's the same. You've reached a ceiling on something. Try increasing the queue from 16 to 64.
- devereaux 9y ago
  Just tried changing a few things with a different benchmark after reading http://middoraid.blogspot.com/2013/01/tweaking.html http://middoraid.blogspot.com/2013/01/tweaking.html, no change.
  - user5994461 9y ago
    Stopped reading after the title. RAID5 is long dead. It's too sensitive to data loss and it has performance issues. Use RAID 1 or RAID 10.
    - devereaux 9y ago
      I can afford to lose the data. I'm doing RAID0. The article was interesting for presenting simple tweaks and a nice table of the results.