3 ms·

I think llama.cpp will die soon because the only models you can run with it are derivatives of a model that Facebook never intended to be publicly released, whi

I think llama.cpp will die soon because the only models you can run with it are derivatives of a model that Facebook never intended to be publicly released, which means all serious usage of it is in a legal limbo at best and just illegal at worst. Even if you get a model that's clean and donated to the world, the quality is still not going to be competitive with the hosted models. And yes I've played with it. It was/is exciting. I can see use cases for it. However none are achievable because the models are (a) not good enough and (b) too legally risky to use.

BrutalCoding 4y ago
(A) is very use case depending. Even with some of the bad smaller models now, I can see devs making use of them to enhance their app (e.g. local search, summaries, sentiments, translations) (B) llama.cpp supports gpt4all, which states that its working on fixing your concern. This is from their README: Roadmap Short Term - Train a GPT4All model based on GPTJ to alleviate llama distribution issues.