This doesn't seem to match my experience with ML and GPUs/ASICs. TPU is the main ML ASIC in use. A major goal of the original TPU design seems to be reducing t

This doesn't seem to match my experience with ML and GPUs/ASICs. TPU is the main ML ASIC in use. A major goal of the original TPU design seems to be reducing the number of memory accesses. The other top-end ML device is NVIDIA's GPUs with Tensor Cores. Both of those chips are designed around fast matrix multiplication, which right now seems to be the most important operation in deep learning - see how RNNs have started to fall out of favor to CNN-based networks with attention heads. The TPU is not faster than NVIDIA's GPUs, but it is cheaper. Right now the future seems to be cheaper ML devices designed to be horizontally scalable. From the CPU perspective, it appears that the major ML effort is related to vectorizing instructions via advanced instruction sets. Everyone who creates silicon is focused very heavily on using smaller and smaller numeric types - float16 is standard and there is work being done for even smaller int based work. I haven't seen any analog-based ML devices in use. Can you share an example? Is there even a way to approximate the results of a matmul using analog devices? It's impossible to guess how much more speed we can get with current approaches, but everything from silicon to networking stack to libraries to network architectures are in their infancy so I would expect dramatic improvements in performance on a regular basis (but not as regular as other areas of software because silicon development is slow)