4 ms·
Disk can be faster than memory
- gus_massa 9y agoIs the slow memory actual memory or it is fake memory paged to disk?
- mpweiher 9y agoYup, as the article says, access patterns matter. A lot. Sequential is a lot faster than random. My MacBook Pro does 2GB/s sequential to/from its SSD. That's 2 bytes every nanosecond. And that's a laptop. I have to admit I am not sure what the fastest DRAM access times are these days, but I don't think much faster than 20ns. If you're only getting a byte per random access, that's around a factor 40 faster for sequential disk vs. random memory.
- jwilliams 9y agoWorth noting the last reference run is on an AWS t2.micro system -- which is a "burstable" CPU instance. For something IO-driven (sequential reads) that probably works fine. When you don't have CPU the IO can buffer. The largest file was 33MB, which I presume EBS handles pretty easily. For something CPU-Memory-driven (memory reads) this bursting could hurt a lot more. Particularly if the CPU decides to page. Plus I wonder if these burstable CPUs throw out the TLB all the time. In this case a reused disk buffer may well be a lot better. For the Macbook Air 2012 - You might see that the SSD has a buffer that is bigger than the files being processed. Certainly will be pretty fast. Particularly on repeat runs. Neither diminishes the overall point of the article - which is you should benchmark your own scenarios.
- robot-dreams 9y agoThanks, I didn't know about "burstable" instances or TLBs before. It's interesting that CPU throttling could affect you differently depending on whether you're mostly reading from disk or memory.
- jwilliams 9y agoVery much me just musing on the results! Thanks for sharing. Would be interesting to see the same on a much bigger data set - and different hardware.