4 ms·
I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower tha
by malfist 3y ago
I keep hearing that, but that's simply not true. SSDs are fast, but they're several orders of magnitude slower than RAM, which is orders of magnitude slower than CPU Cache.
Samsung 990 Pro 2TB has a latency of 40 μs
DDR4-2133 with a CAS 15 has a latency of 14 nano seconds.
DDR4 latency is 0.035% of one of the fastest SSDs, or to put it another way, DDR4 is 2,857x faster than an SSD.
L1 cache is typically accessible in 4 clock cycles, in 4.8 ghz cpu like the i7-10700, L1 cache latency is sub 1ns.
- avg_dev 3y agopretty cool comparisons. quite some differences there. tangent, I remember reading some post called something like "Latency numbers every programmer should know" and being slightly ashamed when I could not internalize it.
- malfist 3y agoOh don't feel bad. I had to look up every one of those numbers
- nzgrover 3y agoYou might enjoy Grace Hopper's leacture which includes this snippet: https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706 https://youtu.be/ZR0ujwlvbkQ?si=vjEQHIGmffjqfHBN&t=2706
- darzu 3y agoprobably this one: https://gist.github.com/hellerbarde/2843375 https://gist.github.com/hellerbarde/2843375
- LeifCarrotson 3y agoI wonder how many people have built failed businesses that never had enough customer data to exceed the DDR4 in the average developer laptop, and never had so many simultaneous queries it couldn't be handled by a single core running SQLite, but built the software architecture on a distributed cloud system just in case it eventually scaled to hundreds of terabytes and billions of simultaneous queries.
- Repulsion9513 3y agoA LOT... especially here.
- malfist 3y agoI totally hear you about that. I work for FAANG, and I'm working on a service that has to be capable of sending 1.6m text messages in less than 10 minutes. The amount of complexity the architecture has because of those constraints is insane. When I worked at my previous job, management kept asking for that scale of designs for less than 1/1000 of the throughput and I was constantly pushing back. There's real costs to building for more scale than you need. It's not as simple as just tweaking a few things. To me there's a couple of big breakpoints in scale: * When you can run on a single server * When you need to run on a single server, but with HA redundancies * When you have to scale beyond a single server * When you have to adapt your scale to deal with the limits of a distributed system, i.e. designing for DyanmoDB's partition limits. Each step in that chain add irrevocable complexity, adds to OE, adds to cost to run and cost to build. Be sure you have to take those steps before you decide too.
- disqard 3y agoI'm trying to guess what "OE" stands for... over engineering? operating expenditure? I'd love to know what you meant :)
- kuschku 3y agoMaybe I'm misunderstanding something, but that's about 2700 a second. Or about 3Mbps. Even a very unoptimized application running on a dev laptop can serve 1Gbps nowadays without issues. So what are the constraints that demand a complex architecture?
- BackBlast 3y agoYou're missing the purpose of the cache. At least for this argument it's mostly for network responses. HDD was 10ms, which was noticeable for cached network request that needs to go back out on the wire. This was also bottle necked by IOPS, after 100-150 IOPS you were done. You could do a bit better with raid, but not the 2-3 orders of magnitude you really needed to be an effective cache. So it just couldn't work as a serious cache, the next step up was RAM. This is the operational environment which redis and such memory caches evolved. 40 us latency is fine for caching. Even the high load 500-600us latency is fine for the network request cache purpose. You can buy individual drives with > 1 million read IOPS. Plenty for a good cache. HDD couldn't fit the bill for the above reasons. RAM is faster, no question, but the lower latency of the RAM over the SSD isn't really helping performance here as the network latency is dominating. Rails conference 2023 has a talk that mentions this. They moved from a memory based cache system to an SSD based cache system. The Redis RAM based system latency was 0.8ms and the SSD based system was 1.2ms for some known system. Which is fine. It saves you a couple of orders of magnitude on cost and you can do much much larger and more aggressive caching with the extra space. Often times these RAM caching servers are a network hop away anyway, or at least a loopback TCP request. Making the question of comparing SSD latency to RAM totally irrelevant.
- jltsiren 3y agoRAM is not as fast in practice as the specs claim, because there is a lot of overhead in accessing it. I did some latency benchmarking on my M2 Max MBP when I got it last year. As long as the working set fits in L1 cache, read latency is ~2 ns. Then it starts increasing slowly, reaching ~10 ns at 10 MiB. Then there is a rapid rise to ~100 ns at 100 MiB, followed by slow growth until ~10 GiB. Then the latency starts increasing rapidly again, reaching ~330 ns at 64 GiB.
- Sweepi 3y agoTo be pedantic because its fun: The random access Latency for L1 is indeed ~4 cycles, but RAM is more like 70ns+ (so you are .5 order of magnitude off, how dare you?). These are my notes from an Anandtech article: RAM: 5950x: 79 ns (400 cycles) 3950x: 86 ns (400 cycles) 10900k: 71 ns Apple M1: 97 ns Apple M1 Max: 111 ns Apple A15P: 105 ns Apple A15E: 141 ns Apple A14P: 101 ns Apple A14E: 201 ns 888 X1: 114 ns L1 Cache: 5950x: 4 cycles (0.8 ns @ 5 GHz) Apple M1: 3 cycles Apple M1 Max: 3 cycles