5 ms·
They're doing matrix operations in the Dram itself? That sounds insane and also fascinating
by Bolwin 1y ago
They're doing matrix operations in the Dram itself? That sounds insane and also fascinating
- summarity 1y agoGetting LLM inference running on any thing is going to be the next “it runs Doom”
- nkurz 1y agoYup, and incredibly they are able to do this on standard RAM by "intentionally violating the timing parameters": Processing-Using-DRAM (PUD) leverages the inherent analog operational characteristics of DRAM to enable highly parallel bit-serial computations directly within memory arrays. Prior research has demonstrated that commercial off-the-shelf DRAM can achieve PUD functionality without hardware modifications by intentionally violating the timing parameters. These studies have established two fundamental PUD operations: RowCopy and majority-of-X (MAJX) (Fig. 1). The RowCopy operation facilitates data movement between different rows within a subarray by issuing a PRE command followed immediately by an ACT command before bitline precharging completes, enabling data transfer through the bitlines. This operation affects all cells along a row simultaneously, making it approximately 100 times faster than processor-mediated data movement. The MAJX operation performs a majority vote among X cells sharing the same bitline that are activated simultaneously, implemented in commercial DRAM by issuing ACT, PRE, and ACT commands in rapid succession without delays. This allows concurrent activation of 2∼32 rows. MAJX enables bit-serial computations that leverage the parallelism of subarrays with 65,536 columns, serving as the fundamental computational unit for PUD.
- nayuki 1y agoThis kind of low-level protocol manipulation of DRAM has some similarities to rowhammer attacks.
- gwern 1y agoCan it be used to covertly run computations invisible to the OS or CPU?
- nsteel 1y agoThis research requires a custom memory controller that's doing "weird" things, the CPU isn't really getting involved here. It's very different compared to row hammer in my opinion. If you have a custom memory controller then I think all bets are off.
- wtallis 1y agoOnly to the same extent that any other co-processor add-in card can do stuff that's not observable by the CPU. Your CPU's RAM is managed by the CPU's memory controller hardware, and that memory controller does not give software the ability to issue individual DRAM commands like precharge. This research uses a memory controller implemented on a FPGA, talking to its own pool of RAM.
- elcritch 1y agoI hope Micron or another commercial player builds a product on this!
- tamlin 1y agoSamsung and SK-Hynix have had specs and papers for a few years already for HBM and GDDR. e.g. * https://www.servethehome.com/sk-hynix-ai-memory-at-hot-chips-2023/ https://www.servethehome.com/sk-hynix-ai-memory-at-hot-chips... * https://www.servethehome.com/samsung-processing-in-memory-technology-at-hot-chips-2023/ https://www.servethehome.com/samsung-processing-in-memory-te... Not sure anyone has started using it in production.