3 ms·

The other reason Optane excels is latency. This is why single-threaded use cases, like system boot, love it. IOPS do not tell the whole story here. Consider di

The other reason Optane excels is latency. This is why single-threaded use cases, like system boot, love it. IOPS do not tell the whole story here. Consider disk A, at 1M IOPS, against disk B, at 1k IOPS. Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. Which disk is higher performing? You might say they're both capable of the task, so just get the cheaper one. This is wrong. Assume, for sake of a simple argument, that disk A can respond in 1us to these requests, and disk B in 1ms. That is, they'd be able to hit "max IOPS" on these requests, if only you have enough. Then disk A services the whole thing in 500us per second, or 0.05% used, and disk B in 500ms per second, or 50% used. So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. Either point of view can be correct, at different times or for different systems. Ignorance that both points of view exist is how things like bufferbloat are everywhere now.

Dylan16807 6y ago
> The other reason Optane excels is latency. It's not an "other reason". IO operations at queue depth 1 are a direct measurement of latency. Queue depth 1 is not about throughput at all. > Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. I will assume the workload is light in CPU too, since spending half the time waiting for the disk is apparently not an instant disqualification. > So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. If you're waiting on a bunch of results and disk A gets them to you significantly faster, it sounds like your initial premise of "only 500 IOPS" was wrong or misleading. If you have a response time you want to meet, then you should measure IOPS with a sliding window of that many milliseconds. If it's still 500 at most, then disk B will still be fine for meeting your target. If it turns out that "500" actually comes in bursts of 50 operations, and you want to be done in 10 milliseconds, then you should label your workload as "5k IOPS". Put in a drive that does at least 10k and you'll see all calculations done in under a frame; basically perfect latency. A drive a million times faster would have no practical difference.

Dylan16807 6y ago

> The other reason Optane excels is latency. It's not an "other reason". IO operations at queue depth 1 are a direct measurement of latency. Queue depth 1 is not about throughput at all. > Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. I will assume the workload is light in CPU too, since spending half the time waiting for the disk is apparently not an instant disqualification. > So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. If you're waiting on a bunch of results and disk A gets them to you significantly faster, it sounds like your initial premise of "only 500 IOPS" was wrong or misleading. If you have a response time you want to meet, then you should measure IOPS with a sliding window of that many milliseconds. If it's still 500 at most, then disk B will still be fine for meeting your target. If it turns out that "500" actually comes in bursts of 50 operations, and you want to be done in 10 milliseconds, then you should label your workload as "5k IOPS". Put in a drive that does at least 10k and you'll see all calculations done in under a frame; basically perfect latency. A drive a million times faster would have no practical difference.