4 ms·
Looks great! Does it work on Windows please?
by GordonS 4y ago
Looks great! Does it work on Windows please?
- dragonwriter 4y agoIf it makes common unix-ish assumptions like “Python 3 executables have a ‘3’ appended to their name”, which other comments here seem to suggest it does, it won’t, even if you have the required version of python installed.
- GordonS 4y agoSo, I actually got it working on Windows, pretty easily! The provided `main.exe` binary worked as-is, but `quantize.exe` did not - I built myself with CMake, and `quantize.exe` started working too.
- volaski 4y agoCurious too. Let me know if you try it out. Technically I think it should work.
- buzzier 4y agoFor Windows: 1. Binary build https://github.com/jaykrell/llama.cpp/releases/tag/1 https://github.com/jaykrell/llama.cpp/releases/tag/1 2. Quantized model (7B/13B/30B) https://mega.nz/folder/UjAUES6Z#bGhKkyiZX3eRrn9HcxVVfA https://mega.nz/folder/UjAUES6Z#bGhKkyiZX3eRrn9HcxVVfA 3. main.exe -m ggml-model-q4_0.bin -t 8 -n 128
- tough 4y agoI actually am installing in windows via WSL/Ubuntu fwiw
- placebo 4y agoThanks. Initial test: main.exe -m ggml-model-q4_0.bin -t 8 -n 128 -p "The Drake equation is nonsense because" The Drake equation is nonsense because it takes parameters that can only be known AFTER the conclusion is reached. It would be like saying "I'm going to prove a theorem by starting from the conclusion, then making up the proof. The Drake equation uses the existence of extraterrestrial intelligence as the conclusion and then making up the parameters. It is nonsense.
- GordonS 4y agoNice, main.exe seems to work just fine with the 7B quantized model - generates a token every 400ms on an AMD Ryzen 5 2600! But, quantize.exe doesn't seem to work - any valid command (such as below) pauses for a split second, then returns with no output? $ quantize.exe ggml-model-f16.bin ggml-model-q4_0.bin 2
- GordonS 4y agoIn case this helps anyone else: I built it myself on Windows with CMake, and then everything just works.
- starik36 4y agoDo you mind sharing the binaries?
- GordonS 4y agoSure! https://filetransfer.io/data-package/8hxKAiaH#link https://filetransfer.io/data-package/8hxKAiaH#link I wasn't sure where to upload them, and that link is only good for 50 downloads. Can put them somewhere else if you know a better location that doesn't require signup.
- starik36 4y agoThank you. llama.exe is basically main.exe? I actually learned how to compile this code via CMake/VS2019. It's sure a whole lot more complicated then it was 25 years ago when I was writing C.
- GordonS 4y agoYes, llama.exe is actually the name the project produces - the other poster must have renamed it to main.exe. I just did `scoop install cmake`, then built from the command line, was a doddle!
- starik36 4y agoI tried it, doesn't work. Trying the sibling post from @buzzier.