2 ms·
You're right. The limit is 4 gibibytes. Astonishingly enough, the llava-v1.5-7b-q4-server.llamafile is 0xfe1c0ed4 bytes in size, which is just 30MB shy of that
by jart 3y ago
You're right. The limit is 4 gibibytes. Astonishingly enough, the llava-v1.5-7b-q4-server.llamafile is 0xfe1c0ed4 bytes in size, which is just 30MB shy of that limit. https://github.com/Mozilla-Ocho/llamafile/commit/81c6ad3251f1fafd77a579d599f8646d7606df47 https://github.com/Mozilla-Ocho/llamafile/commit/81c6ad3251f...
- throwaway743 3y agoNot at my windows machine to test this out right now, but wondering what you mean by having to store the weights in a separate file for wizardcoder, as a result of the 4gb executable limit. How does one go about this? Thank you!
- jart 3y agoYou'd do something like this on PowerShell: curl -Lo llamafile.exe https://github.com/Mozilla-Ocho/llamafile/releases/download/0.1/llamafile-server-0.1 curl -Lo wizard.gguf https://huggingface.co/TheBloke/WizardCoder-Python-13B-V1.0-GGUF/resolve/main/wizardcoder-python-13b-v1.0.Q4_K_M.gguf .\llamafile.exe -m wizard.gguf
- throwaway743 3y agoAwesome! Thank you so much