4 ms·
> A major benefit (for me) is that I can build it with "make" and know that it just works. So, as someone who has never gotten around to doing this, and who al
by vidarh 3y ago
> A major benefit (for me) is that I can build it with "make" and know that it just works.
So, as someone who has never gotten around to doing this, and who also likes not having to deal with the Python tools, it's not quite that simple. Steps I had to take for talk-llama after cloing whisper.cpp:
* apt install libssdl2-dev (Linux; other steps elsewhere)
* make talk-llama from the root of whisper.cpp, not from
* ./download-ggml-model.sh small.en from the models directory
* Tried to run it with the command line in the README, have it
seg fault after failing to open ../llama.cpp/models/llama-13b/ggml-model-q4_0.gguf, cloning llama.cpp, finding the file is not in the repo.
* Searching through the readme for how to find the models, and finding I need to go searching elsewhere because no urls were listed.
* Having to install a bunch of Python dependencies to quantize the models...
This is still far from "build and run". Though I will fully believe that a lot of the Python-based ML projects are worse.
- markdog12 3y agoGot stuck on the model same as you did. No idea what model to use, not interested in fighting with Python to convert the models. I was able to get llama.cpp itself to work, though, including image analysis.
- francisduvivier 3y agoI also had the same issue, in my case it was because I was trying to use a llama 2 model. When trying with codellama https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/tree/main https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/tree/main, which is based on the first llama, it works.
- francisduvivier 3y agoCorrection: looks like it has to do with the quantization rather: 8bit quantization works while less does not not seem to work. Other working model example (no conversion needed): https://huggingface.co/TheBloke//Yarn-Mistral-7B-64k-GGUF/yarn-mistral-7b-64k.Q8_0.gguf https://huggingface.co/TheBloke//Yarn-Mistral-7B-64k-GGUF/ya...