3 ms·
What’s the relationship between llamafile and llama.cpp these days? IIRC, it was originally llama.cpp compiled with Cosmopolitan Libc. I’ve read more recently o
by skavi 3y ago
What’s the relationship between llamafile and llama.cpp these days? IIRC, it was originally llama.cpp compiled with Cosmopolitan Libc. I’ve read more recently of optimizations in llamafile with comparisons against llama.cpp. Are those pushed upstream? Have the projects diverged?
- sfink 2y agoFrom https://hacks.mozilla.org/2024/04/llamafiles-progress-four-months-in/ https://hacks.mozilla.org/2024/04/llamafiles-progress-four-m... it sounds like it's a very active "feeder fork", with jart doing a massive amount of cutting-edge work on CPU inference and slashing through the driver thicket, then upstreaming (some of?) it to llama.cpp. (I don't know enough to know if all of her work applies to the upstream.) I don't follow the space enough to know how much work is going into llama.cpp, but from my naive outsider view, it sounds like llamafile is the place to be for "mere mortal" LLM usage. That's where they (ok, mostly she) care about those sorts of use cases.