4 ms·
Sally McKee, who coined the term "the memory wall", has died
- akkartik 5mo agoMy dissertation was on the memory wall, and I never heard of her :/ RIP
- AnimalMuppet 5mo agoCould you (or someone else in the know) give us a brief overview of the current state of the memory wall issue?
- dirtbagskier 5mo ago[dead]
- akkartik 5mo agoOh my knowledge is woefully out of date. But I believe the memory wall is a fact of life for the most part. Like many others, I nibbled around the edges of the constraint at massive cost in increased complexity. Outside of very specific exceptions the cure tends to be worse than the disease.
- Veserv 5mo agoHigh bandwidth memory (HBM) can deliver TB/s of memory bandwidth and has completely shattered the memory wall for individual cores/compute elements. The only way for compute to keep up is going wide and parallel as seen in GPUs. Despite this, massively increased memory bandwidth does not translate to material performance improvements on non-parallel compute tasks because few tasks are actually memory bandwidth bound, instead being memory latency bound. The best known general solutions for improving memory latency are per-compute element memory caches. Unfortunately, this increases the complexity and size of your compute elements forcing you to reduce the number of compute elements, but a large number of compute elements is the only way to saturate HBM memory bandwidth. To keep up the best known techniques are either algorithmically batch which allows you to go wide using vector/batch instructions or you go the GPU route with memory latency-hiding parallelism.
- vlovich123 5mo agoWell…. The reason there’s such a big mismatch is the memory controller. Something like 80-90% of the energy is spent moving data in and out because of the complex addressing. If you move compute into the RAM and instead shuttle instructions in and out, you might get a huge speed up. The challenge is when an instruction references some data over there - that may end up eliminating all the advantage. But people I believe are trying to commercialize this concept.
- zozbot234 5mo ago> If you move compute into the RAM and instead shuttle instructions in and out, you might get a huge speed up. Isn't that just a per-compute cache/local memory? You're proposing a scaled-up variety of NUMA where every compute core has its local memory and going outside that will cost you more.
- vlovich123 5mo agoCorrect, you can think of this like NUMA or a distributed system where you have compute colocated with storage. It’s a special purpose accelerator for very specific problems that have been optimized to take advantage of such an architecture. It’s also not my proposal. The industry is exploring ways to cut down the energy requirements to do AI - 80-90% of the memory consumption is just moving memory back and forth across the memory controller. It has to read a row from a bank into a row buffer, access the specific cell being requested and then shuttle it over the bus to the compute and then write the data back to the cells. The current idea is to maybe do the processing on the entire row buffer but you could imagine scaling that up to do it at the bank level. The challenge is manufacturing complexity since DRAM is made different, heat from the ALU, etc. [1] https://semiconductor.samsung.com/news-events/tech-blog/hbm-pim-cutting-edge-memory-technology-to-accelerate-next-generation-ai/ https://semiconductor.samsung.com/news-events/tech-blog/hbm-...
- DespairYeMighty 5mo agoShe was a CS PhD and somewhat itinerant professor with a long career who wrote a prominent CS paper about computer memory, Hitting the Memory Wall: Implications of the Obvious https://dl.acm.org/doi/10.1145/216585.216588 https://dl.acm.org/doi/10.1145/216585.216588 on her obituary page, you will see a prominent "Memory Wall" link that is NOT a reference to her paper, but a place for sharing your thoughts about her life
- deater 5mo agoyou wouldn't believe how many people cite that paper as "Wulf et al." when that's practically more characters than saying "Wulf and McKee" I notice these things a bit more as she was my PhD thesis advisor
- marricks 5mo agoThere's only two authors! That's so rude!
- bjourne 5mo agoWhy? For all the automatic academic score tracking systems it doesn't matter one bit if it is Wulf et al. or Wulf and McKee.
- john_strinlai 5mo agoits about respect, not about academic score tracking systems
- mattkrause 5mo agoThe automated ones don't care, but it absolutely matters for the informal credit assignment process that actually runs academia. I really wish we had a better way to "name" papers. Big clinical trials often have an acronym (often hilariously forced: "CXCessoR4"). That takes the emphasis off (one) lead author but it's implausibly hard to make up one for every research paper.
- deater 5mo agoThere are probably so many stories out there of interesting things she did. A few are breifly referenced at her old website here: https://web.archive.org/web/20060116130917/http://www.csl.cornell.edu/~sam/personal.html https://web.archive.org/web/20060116130917/http://www.csl.co...
- dyauspitr 5mo agoI’m never heard of that term.
- northes 5mo agoThanks for your contribution, then.
- fao_ 5mo agoDamn, three years younger than one of my parents. A real shame. Call your loved ones :(