6 ms·
The Memory Wall: Past, Present, and Future of DRAM
- deater 2y agoif Sally Mckee, who coined the term "Memory Wall", had a nickel for each time it gets mentioned, she'd have a lot of nickels
- deleted 2y ago[deleted]
- didgetmaster 2y agoI think that 'memory wall' is completely different from 'memory hole', but I forget how.
- deleted 2y ago[deleted]
- magicalhippo 2y agoBack when I was young, which isn't that far ago, one tried to put as much pre-computed stuff into memory as it was much faster than the CPU. Lookup tables left, right and center. These days you can do thousands of calculations waiting for a few bytes of memory. And not only is the speed difference getting worse, but memory sizes aren't keeping up. Guess we're not far away from compressing stuff before putting it in memory is something you'd want to do most of the time. LZ4 decompression[1] is already just a factor of a few away from memcpy speed. [1]: https://github.com/lz4/lz4 https://github.com/lz4/lz4
- microtonal 2y agoOn the latter point, macOS had had compression memory for a long time by now and some Linux distributions also use it out of the box (I don’t know anything about Windows).
- wongogue 2y agoIt also has Compressed Memory since a version of Windows 10. You can see it in Task Manager.
- hinkley 2y agoOne of the time series databases streams compressed blocks in memory when doing searches, doing distinct blocks per core. For some scenarios it's faster to do a table scan than keep extra indexes hot.
- vardump 2y agoI think the grandparent was talking about transparent memory bus data compression. CPU would fetch some bag of bytes and decompress to cache.
- DaoVeles 2y agoEven in the 90's I recall decompressing ZIP files on a 486 being limited by the HDD speed. It felt like even then we would head towards compressed memory systems once it could be compressed quickly enough.
- Dylan16807 2y agoThat's compression for currently inactive pages, not nearly as aggressive as compressing as a matter of course.
- saagarjha 2y agoThis is basically intended to be fast swap rather than something to be applied to active pages.
- jarbus 2y agoI think quantization for large language models already do something like this - they compress the parameters in memory and then decompress when performing the forward passes
- hinkley 2y agoI was re-reminded of this when watching a performance analysis video that may or may not have been posted here (sometimes I get things here, or reddit, but sometimes the real story is in the related videos). It doesn't take a very big lookup table for it to be faster to rerun the calculations. Especially when you throw multiprocessing in. We need better benchmarking tools that load up competing workloads in the background so you can tell how your optimization really works in production instead of in your little toy universe in the benchmark.
- DaoVeles 2y agoReminded of Kaze Emanuars work on optimising N64 software. (https://www.youtube.com/channel/UCuvSqzfO_LV_QzHdmEj84SQ https://www.youtube.com/channel/UCuvSqzfO_LV_QzHdmEj84SQ) The RAM was so terrible that essentially you try to keep the processors running in cache for as long as possible. RAM access is painful. There is a performance profiling tool built into F3DEX3 that now shows that approximately 70% of the time the system is idle while running Zelda OoT. It is just waiting for memory transfers. The folks at SGI/RAMBUS cut corners a little too hard building that system. But turns out this kind of performance profile is just prep for were we are heading apparently.
- enva2712 2y agofixed function compression on-die would be interesting if applied to this
- hulitu 2y agoWhat happened to SRAM ?
- Tuna-Fish 2y agoStill in wide use in CPU caches. It does not scale well to smaller sizes, and inherently requires more transistors than DRAM, which means it's not cost-effective for main memory.
- DaoVeles 2y agoI forgot were I saw it but it was an analysis of cache to die size and how it has effectively stopped scaling. Larger cache just means larger chips now. It sort of makes sense because SRAM cannot have the same transient nature of a calculations transistor. It has to hold the state for longer than one 3 billionth of a second. So it has to be a little more robust. This is my intuitive take so it could be completely wrong.
- Tuna-Fish 2y agoThe densest kind of SRAM has stopped scaling. However, it's instructive to compare the size of the L3 on AMD Zen4 and Zen5 dies. There is still at least some scaling to be had for SRAM that's built to be fast. (Of course, this is still very irrelevant for comparison to DRAM; the price difference is huge.)
- Lightkey 2y agoProbably on Asianometry: https://www.youtube.com/watch?v=2G4_RZo41Zw https://www.youtube.com/watch?v=2G4_RZo41Zw
- Dylan16807 2y agoIt's huge compared to DRAM. It's worth making a few megabytes of SRAM per core, but not a lot more than that.
- ksec 2y agoI have been stating this for at least 12+ years now. On HN and in many places. And it is still not common knowledge. Hopefully this article get spread and reposted many times in the future. DRAM isn't getting (much) cheaper. And it will continue to be the case in next 10 years. Considering there is nothing on the roadmap to suggest any breakthrough. People old enough may remember in the late 00s and even up to mid 10s, there are words like we will get 16GB computer as baseline, or I could work with 64GB soon. Reality is that DRAM price has fluctuate lately within the same range in the past ~13 years. It was only in 2023 the price floor broke through the $2/GB barrier. Unless we somehow found a way that could magically shrink the capacitor by a substantial amount. Or we change the way we do programming.
- DaoVeles 2y agoIt was the latest console generation that merely doubled the RAM from the previous generation that showed this in action. That plus silly things like Macbooks with 8GB base line for over a decade.
- 0cf8612b2e1e 2y agoOn the other hand, my home workstation has 128GB of memory, for something like $300. That is so much capacity 99% of the time I no longer have to even consider doing something stupid on a data processing task, it just works. It is too early for me to imagine what capabilities are unlocked if I personally had say 1TB of memory, but large RAM capacities seem only required for mammoth database servers.
- sevensor 2y agoCompute in memory is exciting, but it’s going to be hard to pull off. Transistors go underneath capacitors, which means they have to survive the subsequent fabrication of the capacitors without thermal processing disrupting their carefully engineered dopant profiles. It’s not just conservatism that keeps on die logic simple. Bonding, as suggested by the article, is probably the only option, but that brings some absolutely hilarious alignment issues along with it. I’m sure it’s coming, but it won’t be cheap.
- imtringued 2y agoSK Hynix has already developed PIM and nobody cares. https://news.skhynix.com/sk-hynix-develops-pim-next-generation-ai-accelerator/ https://news.skhynix.com/sk-hynix-develops-pim-next-generati...
- imtringued 2y agoI'm curious if it is possible to design an AI accelerator built around ROM that is programmable via maskless lithography. The idea being that 90% of the chip is being produced using conventional mask based semiconductor technology, but the data is written through electron beam lithography. Given a regular ROM structure on the silicon wafer, it would be possible to take advantage of parallel beams and design an electron beam lithography machine specifically for ROM to reduce the cost of programming. The final frontier would be to build a "wafer scale engine"-esque single wafer chip and we would have essentially reached the limits of what is achievable through clever design of silicon based semiconductors.
- rayiner 2y agoCan someone explain like I’m five why DRAM latency basically hasn’t improved in decades? I remember 60ns FPM SIMMs in our 486 back in the 1990s. Doesn’t seem any faster now.
- deleted 2y ago[deleted]
- baybal2 2y ago[dead]