You can already run stuff like it on mobiles, for example alpaca.cpp with Llama models run fine on the CPU on any device that has a few gigs of RAM. Getting to

You can already run stuff like it on mobiles, for example alpaca.cpp with Llama models run fine on the CPU on any device that has a few gigs of RAM. Getting to even larger models using weight quantization, distilling etc it's not 5 years, it's 5 weeks or maybe 5 months at most.. Btw transformers are really simple and the optimisations making them run fast on CPUs have come a long way. I don't know the M1/M2 benchmarks for this, but many CPUs for edge devices have NN accelerators on the silicon that can run this, it's the pairing of the accelerator and gigs of RAM that is the key.