3 ms·
Another great option is https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui The 7B model will run without cha
by eulers_secret 4y ago
Another great option is https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui
The 7B model will run without changes on a 3080. The 13B 4-bit model also runs on a 3080.
This Reddit post has the instructions I followed: https://old.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_install_llama_8bit_and_4bit/ https://old.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_...