3 ms·
The LLaMA 13B and 30B models that Facebook leaked a few days ago have already been successfully spun up on home-grade GPUs using 8-bit and 4-bit quantization, a
by int_19h 4y ago
The LLaMA 13B and 30B models that Facebook leaked a few days ago have already been successfully spun up on home-grade GPUs using 8-bit and 4-bit quantization, and they work surprisingly well:
https://rentry.org/llama-tard-v2 https://rentry.org/llama-tard-v2
30B is not quite as good as ChatGPT yet, but that's mainly a function of fine-tuning quality prompts, and the model is comparable in principle. There are already scripts for fine-tuning, so I expect plenty of pre-tuned chat models to show up very soon now.