3 ms·
I think PCIe TLP overhead and NVMe commands account for the difference between 7 and 8 GB/s.
by wmf 3y ago
I think PCIe TLP overhead and NVMe commands account for the difference between 7 and 8 GB/s.
- mrb 3y agoYou are probably right. Reading some old notes of mine when I was fine-tuning PCIe bandwith on my ZFS server, I had discovered back then that a PCIe Max_Payload_Size of 256 bytes limited usable bandwidth to about 74% of the link's theoretical max. I had calculated that 512 and 1024 bytes (the maximum) would raise it to respectively about 86% and 93% (but my SATA controllers didn't support a value greater than 256.)
- _zoltan_ 3y agoMellanox recommends setting this from the default 512 to 4096 on their NICs.