3 ms·
You cannot train any GPT-2 models with an AMD GPU. Nvidia's CUDA is still the de facto toolkit. Either use Colab (free), or a preemptible GPU instance on GCE w
by minimaxir 7y ago
You cannot train any GPT-2 models with an AMD GPU. Nvidia's CUDA is still the de facto toolkit.
Either use Colab (free), or a preemptible GPU instance on GCE w/ the Deep Learning VM image (relatively cheap). Using consumer GPUs is a recipe for frustration.
- Tenoke 7y ago>You cannot train any GPT-2 models with an AMD GPU. It seems like you can. I know of at least one person who has finetunned 1.5b on a 16GB AMD. I think u/sillysaurusx had some part in it, but apparently translating the code from CUDA was fairly easy.
- gwern 7y agoThere are also several people on Twitter who have mentioned training it on AMD GPUs.
- FeepingCreature 7y agoWorks fine on AMD. Grab a Tensorflow-ROCM image and go to town.