3 ms·
I can run the 30b 4bit model on my m2 air that has 24gb of ram.
by nickthegreek 4y ago
I can run the 30b 4bit model on my m2 air that has 24gb of ram.
- jpp4e 4y agoHi nickthegreek! Could you tell me how you did that? Did you use FastChat or something else? Which model to download? What command to run? Thank you!!!
- nickthegreek 4y agohttps://huggingface.co/Pi3141/alpaca-lora-30B-ggml https://huggingface.co/Pi3141/alpaca-lora-30B-ggml I believe I’m using alpaca.cpp with a command: ./chat -m <bin filename>