I'm confused by how high their latencies are, something isn't right (/me goes and spends too much time that I instead should spend on paperwork). The biggest c

I'm confused by how high their latencies are, something isn't right (/me goes and spends too much time that I instead should spend on paperwork). The biggest contributor is that they tested random reads - that'll likely end up bottlenecked by the FTL, not the protocol & interface latency. Which, especially on consumer devices, can add a lot. Testing sequential, non-buffered, 4k IO with fio, on my boot disk, a Samsung SSD 970 PRO 1TB, I see: Average latency of 13.8us (99.9th percentile 33us), with fio running on the socket the NVMe is attached to, and 16.22us on the remote node (99.9th 37us). With polling that goes down to 8.19 / 8.79us. If I instead test random reads, I see 69.68 / 70.63us (99.9th 147us/ 176us) local/remote node respectively, if I use a 128MB file. Unsurprisingly the local/remote difference nearly vanishes, as the transfer itself isn't the bottleneck anymore. Nor does polling help much. With larger files (I tested up to 128GB, didn't have enough free space for more), that increases a bit, but not by much. Even taking the effect of randomness into account, my disk is several years old, and the latencies reported in the post (and linked PDF) are well above what I see, on a two socket system, with plenty other things going on at the same time. I see considerably lower latencies on two different fast server grade SSDs as well (not quite to the optane level though). The PDF's benchmark description says: > The main goal of this work is to evaluate the hardware and not the filesystem. On each device we create a file of size approximately 90% that of a device size. We fill it with specific data to be sure that the space is allocated and that the reading takes place. To assert the latter we verify that received bytes match the ones we expect. I don't really understand why they're doing that. On most disks that will ensure that the FTL cannot operate from cache, even if that cache is generously sized. IME even random workloads tend to have more locality than a uniform distribution over 90% 4TB. But more importantly, this seems to counter the author's stated goals: > In this work we present a detailed overview and evaluation of modern storage reading performance with regard to available Linux synchronous and asynchronous interfaces. But the test setup chooses the scenario that is least likely to be influenced by the choice of interface? There's a lot of odd stuff in the later sections. It's not at all clear what 7.1 really measures. The numbers are too high for QD1 sequential reads (even at 10us latency one would just be a bit below 1GB/s). But too low for high QD sequential reads. So is it buffered - but then I see higher throughput at 4kB? Then 7.2 talks about "AIO" in the graphs but I think just means doing one read per thread. But how would that be "asynchronous reading"?