3 ms·
A 48GB Nvidia GPU for LLaMa 65B. Maybe 2x24GB? Less for LLaMA 33B, which is still very good. Use a repo that supports QLORA, like https://github.com/OpenAcces
by brucethemoose2 3y ago
A 48GB Nvidia GPU for LLaMa 65B. Maybe 2x24GB?
Less for LLaMA 33B, which is still very good.
Use a repo that supports QLORA, like https://github.com/OpenAccess-AI-Collective/axolotl https://github.com/OpenAccess-AI-Collective/axolotl