TPUs have been used for training since a long time. (PS: we are startup trying to make TPUs more accessible, if you wanna fine-tune Llama3 on TPU check out htt

TPUs have been used for training since a long time. (PS: we are startup trying to make TPUs more accessible, if you wanna fine-tune Llama3 on TPU check out https://github.com/felafax/felafax https://github.com/felafax/felafax)