> You have to wonder why they started with inference and not the other way round. At least from what I know of ASICs (which is admittedly pretty high-level), t

> You have to wonder why they started with inference and not the other way round. At least from what I know of ASICs (which is admittedly pretty high-level), this direction is the natural way I'd expect it to go, because the inference problem is much closer in scale to the kind of problem ASICs are already widely used for and good at. Fast inference can be done with a chip of a size similar to what you see in existing ASICs, like specialized DSP chips. Speeding up training is a lot more speculative. You could try scaling up to massively powerful individual chips that can each replace an NVidia GPU (but with lower cost / lower power budget), or improve distributed training so you can effectively use a much larger number of much weaker customized chips. And there are more significant memory and memory-bandwidth issues with training, so it's less of the pure-compute type of problem that ASICs are more often clear wins for. All of which is probably possible with enough effort, but less straightforward, especially to end up with something that beats GPUs on relevant metrics.