2 ms·

It seems we could use a new kind of memory that streams the weight data in, like GDDR in reverse.

• 0x457 1mo ago
  High Bandwidth Flash? https://www.sandisk.com/company/newsroom/blogs/2025/scaling-beyond-the-wall-inside-sandisks-high-bandwidth-flash-for-ai https://www.sandisk.com/company/newsroom/blogs/2025/scaling-...
  • carloslfu 1mo ago
    interesting!
  • rzzzt 1mo ago
    Optane? (Too soon?)
    • 0x457 1mo ago
      Optane was targeting the latency gap, while HBF targets bandwidth. Given how LLMs work, HBF is perfect for offloading.
• carloslfu 1mo ago
  yes! I guess future hardware designs will have something like that!