3 ms·

Sounds like the awesome architecture for transformers would be colocation of memory and compute.

• Joker_vD 2y ago
  Yes, that's why we generally run them on GPUs.
  • moffkalast 2y ago
    GPUs that pull a kilowatt when running yes. This might actually work on an FPGA if the addition doesn't take too many clock cycles compared to matmuls which were too slow.
  • phkahler 2y ago
    That's why we need a row of ALUs in RAM chips. Read a row of DRAM and use it in a vector operation. With the speed of row reading, the ALU could take many cycles per operation to limit area.
    • namibj 2y ago
      The big problem is that DRAM is extremely secretive about their processes, and they largely don't do that well for logic.
  • api 2y ago
    GPUs are better but I'm thinking of even tighter coupling, like an integrated architecture.