I think they are saying train (or at least fine-tune) on a CPU. This can work in some cases (years ago I certainly did it for CNNs - was slow but you are fine

I think they are saying train (or at least fine-tune) on a CPU. This can work in some cases (years ago I certainly did it for CNNs - was slow but you are fine tuning so anything is an improvement) but I don't know how viable it would be on a transformer.