4 ms·
> To address these challenges, we highlight four architecture research opportunities: High Bandwidth Flash for 10X memory capacity with HBM-like bandwidth; Proc
by jauntywundrkind 9mo ago
> To address these challenges, we highlight four architecture research opportunities: High Bandwidth Flash for 10X memory capacity with HBM-like bandwidth; Processing-Near-Memory and 3D memory-logic stacking for high memory bandwidth; and low-latency interconnect to speedup communication.
High Bandwidth Flash (HBF) got submitted 6 hours ago! It's a great article, fantastic coverage of a wide section of the rapidly moving industry. https://news.ycombinator.com/item?id=46700384 https://news.ycombinator.com/item?id=46700384 https://blocksandfiles.com/2026/01/19/a-window-into-hbf-progress/ https://blocksandfiles.com/2026/01/19/a-window-into-hbf-prog...
HBF is about having many dozens or hundreds of channels of flash memory. The idea of having Processing Near HBF, spread out, perhaps in mixed 3d design, would be not at all surprising to me. One of the main challenges for HBF is building improved vias, improved stacking, and if that tech advanced the idea of more mixed NAND and compute layers rather than just NAND stacks perhaps opens up too.
This is all really exciting possible next steps.
- amelius 9mo agoWhy is persistence such a big thing here? Non-flash memory just needs a tiny bit of power to keep its data. I don't see the revolutionary usecase.
- Gracana 9mo agoDensity is the key here, not persistence.
- amelius 9mo agoThanks! This explains it. Now I'm wondering how you deal with the limited number of write cycles of Flash memory. Or maybe that is not an issue in some applications?
- mrob 9mo agoDuring inference, most of the memory is read only.
- amelius 9mo agoSounds fair. That's not the kind of machine I'd want as a development system though. And usually development systems are beefier than production systems. So curious how they'd solve that.
- Gracana 9mo agoYeah, it is quite specialized for inference. It's unlikely that you'd see this stuff outside of hardware specifically for that. Development systems for AI inference tend to be smaller by necessity. A DGX Spark, Station, a single B300 node... you'd work on something like that before deploying to a larger cluster. There's just nothing bigger than what you'd actually deploy to.
- transpute 9mo agoHBF, like expensive HBM, is targeted at AI data centers. The KAIST professor discussed an HBF unit having a capacity of 512 GB and a 1.638 TBps bandwidth. PCIe x8 GPU bandwidth is about 32GBbps, so HBF could be 50x PCIe bandwidth.