3 ms·
It depends on your precise definition of ASIC. The FPGA thing here would be analogous to an MSIC where m = model. It's clearly different to build a chip for a
by danielmarkbruce 2y ago
It depends on your precise definition of ASIC. The FPGA thing here would be analogous to an MSIC where m = model.
It's clearly different to build a chip for a specific model than what a TPU is.
Maybe we'll start seeing MSICs soon.
- YetAnotherNick 2y agoLLMs and many other models spend 99% of the FLOPs in matrix multiplication. And TPU initially had just single operation i.e. multiply matrix. Even if the MSIC is 100x better than GPU in other operations, it would just be 1% faster overall.
- danielmarkbruce 2y agoYou can still optimize various layers of memory for a specific model, make it all 8 bit or 4 bit or whatever you want, maybe burn in a specific activation function, all kinds of stuff. No chance you'd only get 1% speedup on a chip designed for a specific model.