6 ms·
I have tried out Llamafile and I think it is bloody great. The simplicity of it is commendable. One issue I hope they overcome for Windows however, is being ab
by supposemaybe 3y ago
I have tried out Llamafile and I think it is bloody great. The simplicity of it is commendable.
One issue I hope they overcome for Windows however, is being able to run the executable when it’s more than 4GB (a Windows limitation) as all the powerful models are WAY in excess of that. I believe they will figure out a nice workaround in time for Windows users.
- fbdab103 3y agoI thought Windows could already run the llamafile executable separately and call whatever sized model blob?
- supposemaybe 3y agoI just had a look at their Github, and true enough: “External weights are particularly useful for Windows users because they enable you to work around Windows' 4GB executable file size limit. For Windows users, here's an example for the Mistral LLM: curl -L -o llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/0.6/llamafile-0.6 https://github.com/Mozilla-Ocho/llamafile/releases/download/... curl -L -o mistral.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGU... ./llamafile.exe -m mistral.gguf -ngl 9999
- yjftsjthsd-h 3y agoI mean, devolving to 1 executable and 1 data file is less slick, but it doesn't seem so terrible IMHO.