> Or perhaps hardware will catch up before. I feel like that's been the pretty consistent lesson in computing over the past decades. New technologies start ou

> Or perhaps hardware will catch up before. I feel like that's been the pretty consistent lesson in computing over the past decades. New technologies start out as expensive, exotic, and specialized and become cheap and commonplace over time. The more business value the technology provides, the faster it will happen as well I think. The models will certainly get better (faster to train, less data needed, smaller param counts, etc.) too, though, just like compilers and software have evolved hugely alongside hardware.