I have trained transformers on a 4090 (not language models). Here’s a few notes. You can try out pretty much all GPUs on a cloud provider these days. Do it. V

I have trained transformers on a 4090 (not language models). Here’s a few notes. You can try out pretty much all GPUs on a cloud provider these days. Do it. VRAM is important for maxing out your batch size. It might make your training go faster, but other hardware matters too. How much having more VRAM speeds things up also depends on your training code. If your next batch isn’t ready by the time one is finished training, fix that first. Coil whine is noticeable on my machine. I can hear when the model is training/next batch is loading. Don’t bother with the founder’s edition.