5 ms·

What was the situation like before SSDs? Were the fastest hard drives at the time (15k rpm?) faster than the fastest CPUs?

by aloer 7y ago

insulanus 7y ago
Let's take a look at the throughput of a server hard disk vs. a typical server CPU: https://serverfault.com/questions/190451/what-is-the-throughput-of-15k-rpm-sas-drive https://serverfault.com/questions/190451/what-is-the-through... https://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/6 https://www.anandtech.com/show/9185/intel-xeon-d-review-perf... Now, that 45.8 GB/sec can't usually be fully utilized by the application, but it's a lot higher than the 200MB/sec of the server hard drive. There's also the complication of RAID, max memory fetch rate of a single core, etc. And the fact that the database has to do some processing besides moving data around. But it seems to me that server CPU bandwidth of the last generation is significantly higher than spinning disks.
blihp 7y ago
Individually no, collectively (i.e. storage arrays) yes.
- loeg 7y ago
  Sort of. Intel's server parts top out at on the order of 48 PCIe 3 lanes (Skylake-SP). Those parts have 6-channel DDR4 memory (rated to DDR4-2666), giving a theoretical DRAM throughput limit of about 128 GB/s. Meanwhile, a PCIe 3.0 lane has a theoretical throughput limit of about 985MB/s; 48 lanes makes 47 GB/s. Both are theoretical numbers and in practice will be lower, and you're right that it doesn't give you a lot of free CPU to do any kind of processing on the data, but it seems like the CPU can keep up (as long as Intel continues to have an anemic number of PCIe lanes and until PCIe 4 comes along). (FWIW, on a dual-socket Skylake-SP, Anandtech only achieved 122 GB/s (of theoretical 256 GB/s DRAM maximum). But Intel claims the dual-socket can reach 199 GB/s with an AVX-512 workload.[1] Scaling both of these numbers down by a factor of two gives a very crude estimate for single-socket DRAM bandwidth on a conventional workload (61 GB/s) and AVX-512 (99 GB/s).) Meanwhile, AMD's Zen server parts have more than double the PCIe lanes but only a few more memory channels, so I expect you're correct there — huge/fast PCIe arrays could be bottlenecked on DRAM on that platform. [1]: https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/12 https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...
  - blihp 7y ago
    The question was about spinning rust drives ('before SSDs') and the corresponding CPUs that were available at the same time.

What was the situation like before SSDs? Were the fastest hard drives at the time (15k rpm?) faster than the fastest CPUs?

by aloer 7y ago

What was the situation like before SSDs? Were the fastest hard drives at the time (15k rpm?) faster than the fastest CPUs?

insulanus 7y ago

Let's take a look at the throughput of a server hard disk vs. a typical server CPU: https://serverfault.com/questions/190451/what-is-the-throughput-of-15k-rpm-sas-drive https://serverfault.com/questions/190451/what-is-the-through... https://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/6 https://www.anandtech.com/show/9185/intel-xeon-d-review-perf... Now, that 45.8 GB/sec can't usually be fully utilized by the application, but it's a lot higher than the 200MB/sec of the server hard drive. There's also the complication of RAID, max memory fetch rate of a single core, etc. And the fact that the database has to do some processing besides moving data around. But it seems to me that server CPU bandwidth of the last generation is significantly higher than spinning disks.

blihp 7y ago

Individually no, collectively (i.e. storage arrays) yes.

loeg 7y ago
Sort of. Intel's server parts top out at on the order of 48 PCIe 3 lanes (Skylake-SP). Those parts have 6-channel DDR4 memory (rated to DDR4-2666), giving a theoretical DRAM throughput limit of about 128 GB/s. Meanwhile, a PCIe 3.0 lane has a theoretical throughput limit of about 985MB/s; 48 lanes makes 47 GB/s. Both are theoretical numbers and in practice will be lower, and you're right that it doesn't give you a lot of free CPU to do any kind of processing on the data, but it seems like the CPU can keep up (as long as Intel continues to have an anemic number of PCIe lanes and until PCIe 4 comes along). (FWIW, on a dual-socket Skylake-SP, Anandtech only achieved 122 GB/s (of theoretical 256 GB/s DRAM maximum). But Intel claims the dual-socket can reach 199 GB/s with an AVX-512 workload.[1] Scaling both of these numbers down by a factor of two gives a very crude estimate for single-socket DRAM bandwidth on a conventional workload (61 GB/s) and AVX-512 (99 GB/s).) Meanwhile, AMD's Zen server parts have more than double the PCIe lanes but only a few more memory channels, so I expect you're correct there — huge/fast PCIe arrays could be bottlenecked on DRAM on that platform. [1]: https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/12 https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...
- blihp 7y ago
  The question was about spinning rust drives ('before SSDs') and the corresponding CPUs that were available at the same time.