3 ms·
I'm the lead researcher on the Middlebury Institute project looking at fine-tuning the bigger models, and I got OOM on 745M and 1.5B originally. I had to get an
by newhaus1994 7y ago
I'm the lead researcher on the Middlebury Institute project looking at fine-tuning the bigger models, and I got OOM on 745M and 1.5B originally. I had to get an Azure instance with 24GB VRAM to handle it (using nshepperd's codebase). It works, but takes a while (~500 epochs takes 12 hours on a 100k word training dataset).
- gwern 7y agoOuch! So 11GB is nowhere close to being enough, then. I wonder if even switching to FP16 will be adequate?
- newhaus1994 7y agoMight be able to get 745M down to work on a single GPU. I'm definitely not using all 24GB, so fp16 might be able to get it down enough.
- sdan 7y agoHow would you use fp16 to get it to work on a single GPU? And if you did, what GPU should you use?