3 ms·
ggml-model-f16.bin and ggml-model-q4_0.bin those are the output of convert-pth-to-ggml.py and quantize respectively I had to cancel 30B as I needed to use the
by muyuu 4y ago
ggml-model-f16.bin and ggml-model-q4_0.bin
those are the output of convert-pth-to-ggml.py and quantize respectively
I had to cancel 30B as I needed to use the computer after some 12 hours, now I have to fix the ext4 filesystem of the drive where I was doing it, fun times for the weekend
guess I'll settle for 13B, I was using 7B but the results are pretty lousy compared to GPT4all's Lora, let alone GPT3.5-turbo or better
I'll give a shot to quantising 13B, I'm on 16GB of RAM locally
- dekhn 4y agoYeah, the first time I ran the 30B model, it crashed my machine and I had to reinstall from scratch (linux).