4 ms·
For Windows: 1. Binary build https://github.com/jaykrell/llama.cpp/releases/tag/1 https://github.com/jaykrell/llama.cpp/releases/tag/1 2. Quantized model (7B/
by buzzier 4y ago
For Windows:
1. Binary build https://github.com/jaykrell/llama.cpp/releases/tag/1 https://github.com/jaykrell/llama.cpp/releases/tag/1
2. Quantized model (7B/13B/30B) https://mega.nz/folder/UjAUES6Z#bGhKkyiZX3eRrn9HcxVVfA https://mega.nz/folder/UjAUES6Z#bGhKkyiZX3eRrn9HcxVVfA
3. main.exe -m ggml-model-q4_0.bin -t 8 -n 128
- tough 4y agoI actually am installing in windows via WSL/Ubuntu fwiw
- placebo 4y agoThanks. Initial test: main.exe -m ggml-model-q4_0.bin -t 8 -n 128 -p "The Drake equation is nonsense because" The Drake equation is nonsense because it takes parameters that can only be known AFTER the conclusion is reached. It would be like saying "I'm going to prove a theorem by starting from the conclusion, then making up the proof. The Drake equation uses the existence of extraterrestrial intelligence as the conclusion and then making up the parameters. It is nonsense.
- GordonS 4y agoNice, main.exe seems to work just fine with the 7B quantized model - generates a token every 400ms on an AMD Ryzen 5 2600! But, quantize.exe doesn't seem to work - any valid command (such as below) pauses for a split second, then returns with no output? $ quantize.exe ggml-model-f16.bin ggml-model-q4_0.bin 2
- GordonS 4y agoIn case this helps anyone else: I built it myself on Windows with CMake, and then everything just works.
- starik36 4y agoDo you mind sharing the binaries?
- GordonS 4y agoSure! https://filetransfer.io/data-package/8hxKAiaH#link https://filetransfer.io/data-package/8hxKAiaH#link I wasn't sure where to upload them, and that link is only good for 50 downloads. Can put them somewhere else if you know a better location that doesn't require signup.
- starik36 4y agoThank you. llama.exe is basically main.exe? I actually learned how to compile this code via CMake/VS2019. It's sure a whole lot more complicated then it was 25 years ago when I was writing C.