I'd love to take a language model, load it up, and train it on everything I write in online learning mode. Does one need some massive hardware to do online lea

I'd love to take a language model, load it up, and train it on everything I write in online learning mode. Does one need some massive hardware to do online learning with these models instead of just running the distilled final models?