3 ms·
I thought llamafile was supposed to be the solution to "too hard to work with"? https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llama
by ranger_danger 2y ago
I thought llamafile was supposed to be the solution to "too hard to work with"?
https://github.com/Mozilla-Ocho/llamafile https://github.com/Mozilla-Ocho/llamafile
- archerx 2y agoLlamafile is great and love it. I run all my models using it and it’s super portable, I have tested it on windows and linux, on a powerful PC and SBC. It worked great without too my issues. It takes about a month for the features from llama.cpp to trickle in. Also figuring the best mix of context length size to vram size to desired speed takes a while before it gets intuitive.
- rzzzt 2y agoI thought it's "docker model" (and OCI artifacts).
- dust42 2y agollamafile is a multiplatform executable that wraps the model and a slightly modified version of llama.cpp. IIRC funded by Moz.