Happened to me as well. Apparently, you can just run: python3.10 convert-pth-to-ggml.py models/7B 1 ./quantize ./models/7B/ggml-model-f16.bin ./models/7B

Happened to me as well. Apparently, you can just run: python3.10 convert-pth-to-ggml.py models/7B 1 ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2 And then play with: ./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128 -p "..."