4 ms·
> Enough to fine tune even LLama-2, albeit with a batch size of 1 :) At FP16? I think you need much less (~2 48GB cards) to finetune 70B with increasing level
by brucethemoose2 3y ago
> Enough to fine tune even LLama-2, albeit with a batch size of 1 :)
At FP16? I think you need much less (~2 48GB cards) to finetune 70B with increasing levels of optimization.
I think you can even do it on a single card with QLORA.
- buildbot 3y agoI have not been able to get it working with 2 4090s yet, but I think that's because the library I choose to use (axolotl) does not support model parallel at all.
- brucethemoose2 3y agoI wish Facebook would release the raw 34B model... Or better yet, that Mistral would release one.