3 ms·

They also make glorious boot/OS drives. Those typically see a heavy mix of reads and writes, which is where Optane's dominance over NAND is strongest. The 900P/

They also make glorious boot/OS drives. Those typically see a heavy mix of reads and writes, which is where Optane's dominance over NAND is strongest. The 900P/905P was pretty much the absolute best C: drive money could buy.

cm2187 6y ago
But does that make a noticeable difference? I think a consumer SSD like the samsung 970 EVO PLUS has over 500k IOPS. Would you shave more than milliseconds from the boot time by going higher?
- jiggawatts 6y ago
  IOPS is measuring throughput for parallel operations. Optane SSDs have extremely low latency, which is critical for sequential operations. Most operating systems have long phases of largely single-threaded synchronous I/O, or nearly so. They certainly can't maintain queue depth required to even approach 500K IOPS.
  - Dylan16807 6y ago
    At queue depth 1 the 980 pro delivers 20k reads per second / 45k writes per second. That still sounds like a huge amount. Optane is faster, but is it enough to matter outside some extremely demanding niches?
    - exmadscientist 6y ago
      The other reason Optane excels is latency. This is why single-threaded use cases, like system boot, love it. IOPS do not tell the whole story here. Consider disk A, at 1M IOPS, against disk B, at 1k IOPS. Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. Which disk is higher performing? You might say they're both capable of the task, so just get the cheaper one. This is wrong. Assume, for sake of a simple argument, that disk A can respond in 1us to these requests, and disk B in 1ms. That is, they'd be able to hit "max IOPS" on these requests, if only you have enough. Then disk A services the whole thing in 500us per second, or 0.05% used, and disk B in 500ms per second, or 50% used. So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. Either point of view can be correct, at different times or for different systems. Ignorance that both points of view exist is how things like bufferbloat are everywhere now.
      - Dylan16807 6y ago
        > The other reason Optane excels is latency. It's not an "other reason". IO operations at queue depth 1 are a direct measurement of latency. Queue depth 1 is not about throughput at all. > Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. I will assume the workload is light in CPU too, since spending half the time waiting for the disk is apparently not an instant disqualification. > So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. If you're waiting on a bunch of results and disk A gets them to you significantly faster, it sounds like your initial premise of "only 500 IOPS" was wrong or misleading. If you have a response time you want to meet, then you should measure IOPS with a sliding window of that many milliseconds. If it's still 500 at most, then disk B will still be fine for meeting your target. If it turns out that "500" actually comes in bursts of 50 operations, and you want to be done in 10 milliseconds, then you should label your workload as "5k IOPS". Put in a drive that does at least 10k and you'll see all calculations done in under a frame; basically perfect latency. A drive a million times faster would have no practical difference.
    - imtringued 6y ago
      IOPS are actually meaningless assuming conventional filesystem APIs. In the context of those APIs, IOPS are just a different way of writing MB/s except you get to show off that your block sizes are smaller. Latency is a much bigger factor with conventional filesystem APIs. If you had fully async filesystem APIs that read offset x1 on file f1, offset x1 on file2, offset x100 on file f100 all at the same time then you would see linear scaling and the difference between Optane and NAND would grow smaller. However, that would require learning a new and much harder paradigm (you now have to be aware of parallelism inside SSDs). Optane is a simple upgrade that requires no software changes.
  - deleted 6y ago
    [deleted]
daxfohl 6y ago
Genuinely curious, are disk writes a big part of booting? I'd imagine it to be mostly large sequential reads. Especially since operating systems have a long history of booting from HDD and still need reasonable support for that, I wouldn't expect the boot process has a whole lot of disk thrashing. I also can't think of a reason why it would need to.
- exmadscientist 6y ago
  Preface: My experience is primarily on Windows (required for, among others, crappy enterprise CAD software). Linux is probably different. Most developer workstations have had SSDs for many years. This applies to the Windows 10 developers, too. So they haven't paid any real attention to performance regressions on HDDs, since it still works. It's just slow. But it's a HDD so who cares? That means there's now a lot of I/O going on before and after login, more than would have ever been tolerated in the "old days". Yes, initial boot is just streaming reads, that's easy. But once the OS gets going, it starts logging things... writing. Or finishing up the dregs of the last system update. And NAND SSDs do not handle that mix well at all. They do extremely well with all-reads and very well with all-writes, but a mix will bog them down (block granularity? I don't know why). Optane doesn't do that. It doesn't care about reads versus writes, or small writes between larger reads. It just goes, and it responds fast no matter what the mix. It's not really the sort of thing where you notice "wow, this is so fast". It's the sort of thing where you notice randomly sometime that "hey, I've never actually had to wait for this disk". It's harder to notice the absence of pain compared to the pain itself. With Optane, something else is always the bottleneck. It's not a huge effect, but it was pretty easy to justify the smaller price premium for Optane a few years ago when it was introduced. I fully expected it would carve out a bigger niche for itself at the top of the performance market -- because it really is the top, ahead of anything NAND -- but that hasn't happened. More Intel failings, I guess.
  - daxfohl 6y ago
    https://www.tomshardware.com/reviews/fastest-windows-10-boot-time,5810.html https://www.tomshardware.com/reviews/fastest-windows-10-boot... I should have just googled it. So 4 seconds for optane and 5 for Samsung. I'm surprised, but it looks legit. Though I wonder now if pcie4 changes things since the article was published.
tomnipotent 6y ago
For anyone curious, UserBenchmark is a pretty good resource to compare different drives - here's the 900P up against Samsung 980 Pro, both best-in-class. https://ssd.userbenchmark.com/Compare/Samsung-980-Pro-NVMe-PCIe-M2-1TB-vs-Intel-900P-Optane-NVMe-PCIe-280GB/m1302577vsm315555 https://ssd.userbenchmark.com/Compare/Samsung-980-Pro-NVMe-P...

cm2187 6y ago

But does that make a noticeable difference? I think a consumer SSD like the samsung 970 EVO PLUS has over 500k IOPS. Would you shave more than milliseconds from the boot time by going higher?

jiggawatts 6y ago
IOPS is measuring throughput for parallel operations. Optane SSDs have extremely low latency, which is critical for sequential operations. Most operating systems have long phases of largely single-threaded synchronous I/O, or nearly so. They certainly can't maintain queue depth required to even approach 500K IOPS.
- Dylan16807 6y ago
  At queue depth 1 the 980 pro delivers 20k reads per second / 45k writes per second. That still sounds like a huge amount. Optane is faster, but is it enough to matter outside some extremely demanding niches?
  - exmadscientist 6y ago
    The other reason Optane excels is latency. This is why single-threaded use cases, like system boot, love it. IOPS do not tell the whole story here. Consider disk A, at 1M IOPS, against disk B, at 1k IOPS. Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. Which disk is higher performing? You might say they're both capable of the task, so just get the cheaper one. This is wrong. Assume, for sake of a simple argument, that disk A can respond in 1us to these requests, and disk B in 1ms. That is, they'd be able to hit "max IOPS" on these requests, if only you have enough. Then disk A services the whole thing in 500us per second, or 0.05% used, and disk B in 500ms per second, or 50% used. So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. Either point of view can be correct, at different times or for different systems. Ignorance that both points of view exist is how things like bufferbloat are everywhere now.
    - Dylan16807 6y ago
      > The other reason Optane excels is latency. It's not an "other reason". IO operations at queue depth 1 are a direct measurement of latency. Queue depth 1 is not about throughput at all. > Suppose the workload is light, at only 500 IOPS, which are "small" enough to service quickly. I will assume the workload is light in CPU too, since spending half the time waiting for the disk is apparently not an instant disqualification. > So disk A is in fact actually 1000x faster, if you're waiting on its result. Which, for single-threaded operations, you often are. If you're instead optimizing for throughput, you don't care. If you're waiting on a bunch of results and disk A gets them to you significantly faster, it sounds like your initial premise of "only 500 IOPS" was wrong or misleading. If you have a response time you want to meet, then you should measure IOPS with a sliding window of that many milliseconds. If it's still 500 at most, then disk B will still be fine for meeting your target. If it turns out that "500" actually comes in bursts of 50 operations, and you want to be done in 10 milliseconds, then you should label your workload as "5k IOPS". Put in a drive that does at least 10k and you'll see all calculations done in under a frame; basically perfect latency. A drive a million times faster would have no practical difference.
  - imtringued 6y ago
    IOPS are actually meaningless assuming conventional filesystem APIs. In the context of those APIs, IOPS are just a different way of writing MB/s except you get to show off that your block sizes are smaller. Latency is a much bigger factor with conventional filesystem APIs. If you had fully async filesystem APIs that read offset x1 on file f1, offset x1 on file2, offset x100 on file f100 all at the same time then you would see linear scaling and the difference between Optane and NAND would grow smaller. However, that would require learning a new and much harder paradigm (you now have to be aware of parallelism inside SSDs). Optane is a simple upgrade that requires no software changes.
- deleted 6y ago
  [deleted]

daxfohl 6y ago

Genuinely curious, are disk writes a big part of booting? I'd imagine it to be mostly large sequential reads. Especially since operating systems have a long history of booting from HDD and still need reasonable support for that, I wouldn't expect the boot process has a whole lot of disk thrashing. I also can't think of a reason why it would need to.

exmadscientist 6y ago
Preface: My experience is primarily on Windows (required for, among others, crappy enterprise CAD software). Linux is probably different. Most developer workstations have had SSDs for many years. This applies to the Windows 10 developers, too. So they haven't paid any real attention to performance regressions on HDDs, since it still works. It's just slow. But it's a HDD so who cares? That means there's now a lot of I/O going on before and after login, more than would have ever been tolerated in the "old days". Yes, initial boot is just streaming reads, that's easy. But once the OS gets going, it starts logging things... writing. Or finishing up the dregs of the last system update. And NAND SSDs do not handle that mix well at all. They do extremely well with all-reads and very well with all-writes, but a mix will bog them down (block granularity? I don't know why). Optane doesn't do that. It doesn't care about reads versus writes, or small writes between larger reads. It just goes, and it responds fast no matter what the mix. It's not really the sort of thing where you notice "wow, this is so fast". It's the sort of thing where you notice randomly sometime that "hey, I've never actually had to wait for this disk". It's harder to notice the absence of pain compared to the pain itself. With Optane, something else is always the bottleneck. It's not a huge effect, but it was pretty easy to justify the smaller price premium for Optane a few years ago when it was introduced. I fully expected it would carve out a bigger niche for itself at the top of the performance market -- because it really is the top, ahead of anything NAND -- but that hasn't happened. More Intel failings, I guess.
- daxfohl 6y ago
  https://www.tomshardware.com/reviews/fastest-windows-10-boot-time,5810.html https://www.tomshardware.com/reviews/fastest-windows-10-boot... I should have just googled it. So 4 seconds for optane and 5 for Samsung. I'm surprised, but it looks legit. Though I wonder now if pcie4 changes things since the article was published.

tomnipotent 6y ago

For anyone curious, UserBenchmark is a pretty good resource to compare different drives - here's the 900P up against Samsung 980 Pro, both best-in-class. https://ssd.userbenchmark.com/Compare/Samsung-980-Pro-NVMe-PCIe-M2-1TB-vs-Intel-900P-Optane-NVMe-PCIe-280GB/m1302577vsm315555 https://ssd.userbenchmark.com/Compare/Samsung-980-Pro-NVMe-P...