Sure, but my point was that specialization is basically a one-time trick. Once you build a 5nm (or whatever they call the process node, as the numbers are basic

Sure, but my point was that specialization is basically a one-time trick. Once you build a 5nm (or whatever they call the process node, as the numbers are basically marketing these days) TPU, GPU, CPU, what have you, that's kind of it. You could spend a ton of effort on clever architecture and physical design, and maybe get a 2nd generation with a ~10-20% performance boost (but maybe not even better power efficiency, just more power). If you haven't maxed out your area/power budget, you might be able to scale up a bit more with a bigger, power hungrier die or MCM/interposer, but GPUs have basically already played that card. So each 'problem' that's worth it might get a specialized ASIC or two, but all of that could very well run its course in the next 5-10 years, and things look kind of bleak for performance gains after that.