3 ms·

You say 4 Gb/s here but in the thread below show 4GB/s, i.e. 8-10x faster depending on how you think about link speeds. What happens when you test each drive i

You say 4 Gb/s here but in the thread below show 4GB/s, i.e. 8-10x faster depending on how you think about link speeds. What happens when you test each drive individually? Do they all perform identically or is the 6th device slower? An array will be limited by its slowest member even if there is no bus contention. Also, there are many tunable parameters and I am not sure the defaults are equivalent for RAID5 or RAID6 in the Linux-MD layer. What happens if you use all 6 drives in a RAID5 array? What happens if you use 5 drives in a RAID6 array? Did you try changing the chunk size to match the SSD page size? Are you sure the partitions are all aligned properly to avoid any fragmentation during SSD writes? There are countless variables here...

devereaux 9y ago
Sorry for the typo. I will edit the post to show an uppercase B. This is still at least an order of magnitude below the 57 GB/s reported by praseodym This is for a raid0, as I am only concerned with performance here. The chunk size is the default 512K. I can try 4k, but I haven't found the Samsung NVMe SM961 page size. The partitions are of course 4k aligned: Number Start (sector) End (sector) Size Code Name 1 4096 209719295 100.0 GiB FD00 Linux RAID 2 209719296 419434495 100.0 GiB FD00 Linux RAID 3 419434496 629149695 100.0 GiB FD00 Linux RAID 4 629149696 838864895 100.0 GiB FD00 Linux RAID 5 838864896 1048580095 100.0 GiB FD00 Linux RAID 6 1048580096 1258295295 100.0 GiB FD00 Linux RAID 7 1258295296 1468010495 100.0 GiB FD00 Linux RAID 8 1468010496 1677725695 100.0 GiB FD00 Linux RAID 9 1677725696 1875384974 94.3 GiB 8300 Linux filesystem 10 34 4095 2.0 MiB EF02 BIOS boot partition No drive is slower on independant benchmarks. I created the various partitions to also test that scenario by testing various combination of the 6 drives. Performance increases linearly until the 6th drive. If you want to try, give me your email and I'll send you the root password to play with SSH. I was setting md5 for psql tests but there's nothing on the server yet, you can also play as much as you want with /dev/md6 You can also get a AX160-NVMe at Hetzner.de. If you cancel before 14 days, there is no charge.

devereaux 9y ago

Sorry for the typo. I will edit the post to show an uppercase B. This is still at least an order of magnitude below the 57 GB/s reported by praseodym This is for a raid0, as I am only concerned with performance here. The chunk size is the default 512K. I can try 4k, but I haven't found the Samsung NVMe SM961 page size. The partitions are of course 4k aligned: Number Start (sector) End (sector) Size Code Name 1 4096 209719295 100.0 GiB FD00 Linux RAID 2 209719296 419434495 100.0 GiB FD00 Linux RAID 3 419434496 629149695 100.0 GiB FD00 Linux RAID 4 629149696 838864895 100.0 GiB FD00 Linux RAID 5 838864896 1048580095 100.0 GiB FD00 Linux RAID 6 1048580096 1258295295 100.0 GiB FD00 Linux RAID 7 1258295296 1468010495 100.0 GiB FD00 Linux RAID 8 1468010496 1677725695 100.0 GiB FD00 Linux RAID 9 1677725696 1875384974 94.3 GiB 8300 Linux filesystem 10 34 4095 2.0 MiB EF02 BIOS boot partition No drive is slower on independant benchmarks. I created the various partitions to also test that scenario by testing various combination of the 6 drives. Performance increases linearly until the 6th drive. If you want to try, give me your email and I'll send you the root password to play with SSH. I was setting md5 for psql tests but there's nothing on the server yet, you can also play as much as you want with /dev/md6 You can also get a AX160-NVMe at Hetzner.de. If you cancel before 14 days, there is no charge.