2 ms·

Move compute closer to memory.

• zozbot234 4y ago
  You mean moving memory closer to compute. And that's happening already in lots of places, even though it brings increased complexity due to having to account for NUMA.
  • visarga 4y ago
    I am thinking long term, as neural nets scale up, we'll have to move compute into memory. The other way around is problematic, small caches don't work well with neural nets. In fact the fastest Transformer (FlashTransformer) is based on principled usage of the SRAM cache because that's the bottleneck.
• TillE 4y ago
  Bigger L1/L2 caches is definitely another "easy" thing with substantial benefits, but doing it cheaply is hard.
  • sroussey 4y ago
    Those caches are SRAM which has not improved much the last few generations.
  • wmf 4y ago
    That's not easy at all because larger caches are slower. A higher hit rate with worse latency can give worse performance. We may see more levels of cache to better optimize the tradeoffs.