6 ms·

I mean, I love the insane GB/sec figures, but does anyone else mostly care about IOPS? These state 1.8M read and 1M write, which sounds quite impressive.

CoastalCoder 5y ago
Related: anyone know of a good video or diagram for helping CS students get an intuition regarding the interplay of bandwidth and latency? Including how saturating bandwidth increases latency by causing queueing bottlenecks? I'm looking for something a bit more visual and dynamic than the old "station wagon full of tapes going down the highway" imagery. [EDIT: Just for clarification, I feel like I already have a pretty good grasp on these concepts. I'm looking for good ways to help others at the ~ undergrad level.]
- wtallis 5y ago
  I do some rather coarse measurements of random read throughput vs latency as part of my SSD reviews. See eg. the bottom of https://www.anandtech.com/show/16636/the-inland-performance-plus-2tb-ssd-review-phisons-e18-nvme-controller-tested/3 https://www.anandtech.com/show/16636/the-inland-performance-... Those graphs cut off the essentially vertical latency spike that results from enqueuing requests faster than the sustained rate at which the drive can serve them. For a different view in terms of queue depth rather than throughput, there are some relevant graphs from an older review that predates io_uring: https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p4800x-750gb-handson-review/5 https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p48... Generally speaking, latency starts increasing long before you've reached a drive's throughput limit. Some of this is inevitable, because you have a relatively small number of channels (eg. 8) and dies to access in parallel. Once you're up to the throughput range where you have dozens of requests in flight at a time, you'll have constant collisions where multiple requests want to read from the same plane/die/channel at once, and some of those requests have to be delayed. But that's mostly about contention and link utilization between the SSD controller and the NAND flash itself. The PCIe link is pretty good about handling transactions with consistently low latency even when on average it's mostly busy.
- bombcar 5y ago
  Here's a post on relative latencies that may be useful: https://danluu.com/infinite-disk/ https://danluu.com/infinite-disk/ There was another post I saw recently comparing the increase in disk SIZE over the last 30 years vs the increase in disk SPEED vs LATENCY (so size in GB, speed in GB/s, latency in IOPS) - and how size increases far outstripped speed which outstripped latency, though all had improved. Found it! The key is IOPS/GB as a metric. https://brooker.co.za/blog/2021/03/25/latency-bandwidth.html https://brooker.co.za/blog/2021/03/25/latency-bandwidth.html
- uyt 5y ago
  Are you referring to Little's Law?
- louwrentius 5y ago
  Maybe this doesn’t answer your question exactly but I addressed this topic in two blogpost, maybe it helps. https://louwrentius.com/understanding-storage-performance-iops-and-latency.html https://louwrentius.com/understanding-storage-performance-io... https://louwrentius.com/understanding-iops-latency-and-storage-performance.html https://louwrentius.com/understanding-iops-latency-and-stora...
MrFoof 5y ago
>...but does anyone else mostly care about IOPS IOPS helps, but for the average user, hundreds of thousands is already functionally infinite. What matters at this point is latency. Where you really feel that is Queue Depth 1. Where you read a file that points you to other files, that point you to other files, etc. That is the exact case where the computer is still making you wait. This happens when you start your operating system, it starts services, you launch apps, etc. Driving that latency down is the biggest improvement you'll ever see past where we are today in terms of IOPS and throughput. This is where the latest Optane actually shines. Optane doesn't win on IOPS or throughput, but where it shines is its crazy latency (delivered at relatively low power levels). Where latencies are 10% of that of even the highest end PCIe 4.0 NVMe SSDs. Do something like launch 20 applications at once, and it'll be done in a fraction of a time compared to even something like a Samsung 980 Pro because of latency being more around 10 μs instead of 100 μs. PCIe 5.0 SSDs will cut latencies down to where Optane is today, but driving latency under 1 μs is where we'll get into a new level of crazy.
- Dylan16807 5y ago
  Isn't launching 20 applications at once the realm where flash competes the best?
  - labawi 5y ago
    Technically, it would comparatively excel at launching 64K+ applications at once, but yes - flash is not quite amazing at launching 20 applications serially.
- jiggawatts 5y ago
  I can't upvote this enough. Related: Notice how the public cloud marketing material tends to focus on scalability over other metrics? That's because scaling horizontally for them is easy: They just plop down more "stamps" -- a set of clusters and controllers that is their unit of scale. Need 1,000 more servers in US East? Plop down 10 more stamps of 100 servers. Easy! Except of course this involves an awful lot of networking, with long cable runs and many hops and virtualisation layers. The end result is that you can't get anywhere near the underlying storage latency. Azure's Premium SSD has "write flush" latencies of about 4 milliseconds according to my measurements, which is easily 100x slower than what my laptop can do with a now very outdated Samsung NVMe SSD. Notice that if you go to their marketing page, they talk about "low latency" and "lowest latency", but they have no numbers? Meanwhile the MB/s and IOPS is stated with numbers: https://azure.microsoft.com/en-us/pricing/details/managed-disks/ https://azure.microsoft.com/en-us/pricing/details/managed-di...

CoastalCoder 5y ago

Related: anyone know of a good video or diagram for helping CS students get an intuition regarding the interplay of bandwidth and latency? Including how saturating bandwidth increases latency by causing queueing bottlenecks? I'm looking for something a bit more visual and dynamic than the old "station wagon full of tapes going down the highway" imagery. [EDIT: Just for clarification, I feel like I already have a pretty good grasp on these concepts. I'm looking for good ways to help others at the ~ undergrad level.]

wtallis 5y ago
I do some rather coarse measurements of random read throughput vs latency as part of my SSD reviews. See eg. the bottom of https://www.anandtech.com/show/16636/the-inland-performance-plus-2tb-ssd-review-phisons-e18-nvme-controller-tested/3 https://www.anandtech.com/show/16636/the-inland-performance-... Those graphs cut off the essentially vertical latency spike that results from enqueuing requests faster than the sustained rate at which the drive can serve them. For a different view in terms of queue depth rather than throughput, there are some relevant graphs from an older review that predates io_uring: https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p4800x-750gb-handson-review/5 https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p48... Generally speaking, latency starts increasing long before you've reached a drive's throughput limit. Some of this is inevitable, because you have a relatively small number of channels (eg. 8) and dies to access in parallel. Once you're up to the throughput range where you have dozens of requests in flight at a time, you'll have constant collisions where multiple requests want to read from the same plane/die/channel at once, and some of those requests have to be delayed. But that's mostly about contention and link utilization between the SSD controller and the NAND flash itself. The PCIe link is pretty good about handling transactions with consistently low latency even when on average it's mostly busy.
bombcar 5y ago
Here's a post on relative latencies that may be useful: https://danluu.com/infinite-disk/ https://danluu.com/infinite-disk/ There was another post I saw recently comparing the increase in disk SIZE over the last 30 years vs the increase in disk SPEED vs LATENCY (so size in GB, speed in GB/s, latency in IOPS) - and how size increases far outstripped speed which outstripped latency, though all had improved. Found it! The key is IOPS/GB as a metric. https://brooker.co.za/blog/2021/03/25/latency-bandwidth.html https://brooker.co.za/blog/2021/03/25/latency-bandwidth.html
uyt 5y ago
Are you referring to Little's Law?
louwrentius 5y ago
Maybe this doesn’t answer your question exactly but I addressed this topic in two blogpost, maybe it helps. https://louwrentius.com/understanding-storage-performance-iops-and-latency.html https://louwrentius.com/understanding-storage-performance-io... https://louwrentius.com/understanding-iops-latency-and-storage-performance.html https://louwrentius.com/understanding-iops-latency-and-stora...

MrFoof 5y ago

>...but does anyone else mostly care about IOPS IOPS helps, but for the average user, hundreds of thousands is already functionally infinite. What matters at this point is latency. Where you really feel that is Queue Depth 1. Where you read a file that points you to other files, that point you to other files, etc. That is the exact case where the computer is still making you wait. This happens when you start your operating system, it starts services, you launch apps, etc. Driving that latency down is the biggest improvement you'll ever see past where we are today in terms of IOPS and throughput. This is where the latest Optane actually shines. Optane doesn't win on IOPS or throughput, but where it shines is its crazy latency (delivered at relatively low power levels). Where latencies are 10% of that of even the highest end PCIe 4.0 NVMe SSDs. Do something like launch 20 applications at once, and it'll be done in a fraction of a time compared to even something like a Samsung 980 Pro because of latency being more around 10 μs instead of 100 μs. PCIe 5.0 SSDs will cut latencies down to where Optane is today, but driving latency under 1 μs is where we'll get into a new level of crazy.

Dylan16807 5y ago
Isn't launching 20 applications at once the realm where flash competes the best?
- labawi 5y ago
  Technically, it would comparatively excel at launching 64K+ applications at once, but yes - flash is not quite amazing at launching 20 applications serially.
jiggawatts 5y ago
I can't upvote this enough. Related: Notice how the public cloud marketing material tends to focus on scalability over other metrics? That's because scaling horizontally for them is easy: They just plop down more "stamps" -- a set of clusters and controllers that is their unit of scale. Need 1,000 more servers in US East? Plop down 10 more stamps of 100 servers. Easy! Except of course this involves an awful lot of networking, with long cable runs and many hops and virtualisation layers. The end result is that you can't get anywhere near the underlying storage latency. Azure's Premium SSD has "write flush" latencies of about 4 milliseconds according to my measurements, which is easily 100x slower than what my laptop can do with a now very outdated Samsung NVMe SSD. Notice that if you go to their marketing page, they talk about "low latency" and "lowest latency", but they have no numbers? Meanwhile the MB/s and IOPS is stated with numbers: https://azure.microsoft.com/en-us/pricing/details/managed-disks/ https://azure.microsoft.com/en-us/pricing/details/managed-di...