What happens when training an LLM (or even a small portion of an LLM, like a domain-local update) takes milliseconds instead of weeks, and costs a millionth of

What happens when training an LLM (or even a small portion of an LLM, like a domain-local update) takes milliseconds instead of weeks, and costs a millionth of a penny instead of hundreds of millions of dollars? What happens when e.g. our smartphones can perform TRAINING hundreds of times per minute? Isn’t that the true gateway to human-like AGI? Seems to me that we might be there in under 50 years…