4 ms·

>...but does anyone else mostly care about IOPS IOPS helps, but for the average user, hundreds of thousands is already functionally infinite. What matters at

by MrFoof 5y ago

>...but does anyone else mostly care about IOPS IOPS helps, but for the average user, hundreds of thousands is already functionally infinite. What matters at this point is latency. Where you really feel that is Queue Depth 1. Where you read a file that points you to other files, that point you to other files, etc. That is the exact case where the computer is still making you wait. This happens when you start your operating system, it starts services, you launch apps, etc. Driving that latency down is the biggest improvement you'll ever see past where we are today in terms of IOPS and throughput. This is where the latest Optane actually shines. Optane doesn't win on IOPS or throughput, but where it shines is its crazy latency (delivered at relatively low power levels). Where latencies are 10% of that of even the highest end PCIe 4.0 NVMe SSDs. Do something like launch 20 applications at once, and it'll be done in a fraction of a time compared to even something like a Samsung 980 Pro because of latency being more around 10 μs instead of 100 μs. PCIe 5.0 SSDs will cut latencies down to where Optane is today, but driving latency under 1 μs is where we'll get into a new level of crazy.

Dylan16807 5y ago
Isn't launching 20 applications at once the realm where flash competes the best?
- labawi 5y ago
  Technically, it would comparatively excel at launching 64K+ applications at once, but yes - flash is not quite amazing at launching 20 applications serially.
jiggawatts 5y ago
I can't upvote this enough. Related: Notice how the public cloud marketing material tends to focus on scalability over other metrics? That's because scaling horizontally for them is easy: They just plop down more "stamps" -- a set of clusters and controllers that is their unit of scale. Need 1,000 more servers in US East? Plop down 10 more stamps of 100 servers. Easy! Except of course this involves an awful lot of networking, with long cable runs and many hops and virtualisation layers. The end result is that you can't get anywhere near the underlying storage latency. Azure's Premium SSD has "write flush" latencies of about 4 milliseconds according to my measurements, which is easily 100x slower than what my laptop can do with a now very outdated Samsung NVMe SSD. Notice that if you go to their marketing page, they talk about "low latency" and "lowest latency", but they have no numbers? Meanwhile the MB/s and IOPS is stated with numbers: https://azure.microsoft.com/en-us/pricing/details/managed-disks/ https://azure.microsoft.com/en-us/pricing/details/managed-di...
- benlivengood 5y ago
  Cloud customers are also living in the land of millisecond latency for most things. Their customers are at the other end of a multi-ms round trip. Sharding reduces the need to push as many IOPS through a single database machine, leaving slack for higher storage latency. For people who really need low latency in the cloud there is EC2 instance store or GCE local SSD.
  - jiggawatts 5y ago
    That's not how latency works. It doesn't matter if the user running Google Chrome is 50ms away from the web server, if that server in turn does 10,000 round-trips to the database server. Suddenly, microseconds matter. Why 10K round-trips you ask? Because of ORMs like Entity Framework. They have a habit of producing wildly inefficient query code, but hey... it saves on developer time! I literally just optimised a large web site that actually had fairly decent SQL queries. I reduced the latency from 1,500 microseconds to about 50 microseconds and the performance of most pages improved by about 150-300 milliseconds! Let me reiterate: A latency difference that can't even be benchmarked easily because it's so small made a very noticeable difference to page load times. The site feels snappier. Navigating it is now a joy instead of a drag.
judge2020 5y ago
IMO it's functionally infinite until it's not, 1gb was a 'functionally infinite' amount of ram back when 64k was considered top tier.
PinkPigeon 5y ago
Hey, thank you, you have just corrected a massive misconception I had about SSD performance. My logic always worked like this: if IOPS is input / output per second, surely that means 10,000 IOPS of, say, the usual random 4kb file write at QD1, then that means I can get 10,000 of those in one second. So, a higher number must be better. I had no idea that IOPS needs to take latency into account. After having done some reading I am still somewhat confused as to how this really works, but for future reference I will use latency as a measurement of performance again, as that is indeed what I thought IOPS was.

by MrFoof 5y ago

Dylan16807 5y ago

Isn't launching 20 applications at once the realm where flash competes the best?

labawi 5y ago
Technically, it would comparatively excel at launching 64K+ applications at once, but yes - flash is not quite amazing at launching 20 applications serially.

jiggawatts 5y ago

I can't upvote this enough. Related: Notice how the public cloud marketing material tends to focus on scalability over other metrics? That's because scaling horizontally for them is easy: They just plop down more "stamps" -- a set of clusters and controllers that is their unit of scale. Need 1,000 more servers in US East? Plop down 10 more stamps of 100 servers. Easy! Except of course this involves an awful lot of networking, with long cable runs and many hops and virtualisation layers. The end result is that you can't get anywhere near the underlying storage latency. Azure's Premium SSD has "write flush" latencies of about 4 milliseconds according to my measurements, which is easily 100x slower than what my laptop can do with a now very outdated Samsung NVMe SSD. Notice that if you go to their marketing page, they talk about "low latency" and "lowest latency", but they have no numbers? Meanwhile the MB/s and IOPS is stated with numbers: https://azure.microsoft.com/en-us/pricing/details/managed-disks/ https://azure.microsoft.com/en-us/pricing/details/managed-di...

benlivengood 5y ago
Cloud customers are also living in the land of millisecond latency for most things. Their customers are at the other end of a multi-ms round trip. Sharding reduces the need to push as many IOPS through a single database machine, leaving slack for higher storage latency. For people who really need low latency in the cloud there is EC2 instance store or GCE local SSD.
- jiggawatts 5y ago
  That's not how latency works. It doesn't matter if the user running Google Chrome is 50ms away from the web server, if that server in turn does 10,000 round-trips to the database server. Suddenly, microseconds matter. Why 10K round-trips you ask? Because of ORMs like Entity Framework. They have a habit of producing wildly inefficient query code, but hey... it saves on developer time! I literally just optimised a large web site that actually had fairly decent SQL queries. I reduced the latency from 1,500 microseconds to about 50 microseconds and the performance of most pages improved by about 150-300 milliseconds! Let me reiterate: A latency difference that can't even be benchmarked easily because it's so small made a very noticeable difference to page load times. The site feels snappier. Navigating it is now a joy instead of a drag.

judge2020 5y ago

IMO it's functionally infinite until it's not, 1gb was a 'functionally infinite' amount of ram back when 64k was considered top tier.

PinkPigeon 5y ago

Hey, thank you, you have just corrected a massive misconception I had about SSD performance. My logic always worked like this: if IOPS is input / output per second, surely that means 10,000 IOPS of, say, the usual random 4kb file write at QD1, then that means I can get 10,000 of those in one second. So, a higher number must be better. I had no idea that IOPS needs to take latency into account. After having done some reading I am still somewhat confused as to how this really works, but for future reference I will use latency as a measurement of performance again, as that is indeed what I thought IOPS was.