4 ms·
In addition to paid subscriptions, you might want to consider running an LLM locally. One of a number of projects enabling this approach is "continue" - https:/
by mypalmike 3y ago
In addition to paid subscriptions, you might want to consider running an LLM locally. One of a number of projects enabling this approach is "continue" - https://github.com/continuedev/continue https://github.com/continuedev/continue
- geepytee 3y agoThe issue with local IMO is that you'll going to have to compromise on quality. And I really don't want to waste my time talking with inferior models when better models are widely available.
- SOLAR_FIELDS 3y agoYeah, at the time of writing this comment it feels like if you want to easily convince me that your local LLM is good enough, just do a side by side comparison of prompts with your LLM vs GPT4. It doesn’t even have to be better. It just has to be kind of in the same ballpark
- fzzzy 3y agoI'd like to try this, do you have some prompts you like to use for evaluation?
- singularity2001 3y agopeople who think that their local LLM's are in the same ballpark as GPT or Claude are delusional
- 3abiton 3y agoHaven't found any good local model yet that rivals or come close to gpt-4
- fzzzy 3y agoHave you tried miqu q5
- 3abiton 3y agoI have dabbled with the quant version (exl), still not the same. Which quant/version are you using? There are so many that the output shifts dramatically!
- fzzzy 3y agoI'm using gguf q5 and I find the output degrades going to even q4. Mistral has said they still intend to open newer bigger models, so I'm really looking forward to that.