3 ms·
Llama 7B is quite dumb. Using the 13B you'd get significantly better results, and you can train a qlora on a single 3090 (I think even less is possible but not
by cypress66 3y ago
Llama 7B is quite dumb. Using the 13B you'd get significantly better results, and you can train a qlora on a single 3090 (I think even less is possible but not sure)
- deleted 3y ago[deleted]
- advaith08 3y agooh yeah definitely. Do you know how I can get access to one for cheap though? I burnt through $150 just on this exercise with a P100 on GCP
- brucethemoose2 3y agovast.ai Yeah, GCP GPU prices are terrible. $150 for a short time on a P100 is highway robbery. TPUs are better, but still kinda pricey.
- stavros 3y agoI'd love to see an update for 13B, and I can vouch that vast.ai prices are very good.
- cypress66 3y agoOoof. I'd expect this to cost like 5 bucks on runpod using a single 3090. I use axolotl for training, I didn't check your notebook but axolotl likely comes with more optimized defaults for speed and vram than what you're doing.