4 ms·
What is the latest in conversational models that allow GPT3 like (or close) performance w.r.t running things locally?
by cced 3y ago
What is the latest in conversational models that allow GPT3 like (or close) performance w.r.t running things locally?
- modernpink 3y agoGPT3 is dated so many open source models are competitive with it, but Vicuna 13b is supposed to be competitive with GPT4
- speedgoose 3y agoAgainst GPT3.5 perhaps the gaps aren’t too big for your use cases, but I wouldn’t say it’s in the GPT4 league. It looks close in the benchmarks but the difference in quality feels (to me) huge in practice. The others models are simply a lot worse.
- modernpink 3y agoInteresting. Have you tried StableVicuna?
- speedgoose 3y agoNo, is it worth a try? I didn’t see a lot of hype about it so I didn’t try it.
- noman-land 3y agoApparently Vicuna 13B is quite good according to Google's own leaked docs. https://twitter.com/jelleprins/status/1654197282311491592 https://twitter.com/jelleprins/status/1654197282311491592
- amelius 3y agoIs there a way to always stay up to date with the latest and best performing models? Perhaps it's me but I find it difficult to navigate HuggingFace and find models sorted by benchmark.
- noman-land 3y agoHonestly, I just read hackernews :).
- amelius 3y agoHN posts are not always in chronological order.
- noman-land 3y agoI didn't say it was the best way, just the way I'm doing it right now :).
- nickthegreek 3y agoI check r/LocalLlama
- space_fountain 3y agoThat's according to this (https://lmsys.org/blog/2023-03-30-vicuna/ https://lmsys.org/blog/2023-03-30-vicuna/) promotional blog post and just cited by the google memo right? Which isn't really even a doc, just a memo that was circulating inside google. I also find it strange they don't contrast gpt4 and gpt3.5
- int_19h 3y agoThis assessment is based largely on GPT-4 evaluation of the output. In actual use, Vicuna-13B isn't even as good as GPT-3.5, although I do have high hopes for 30B if and when they decide to make that available (or someone else trains it, since the dataset is out). And don't forget that all the LLaMA-based models only have 2K context size. It's good enough for random chat, but you quickly bump into it for any sort of complicated task solving or writing code. Increasing this to 4K - like GPT-3.5 has - would require significantly more RAM for the same model size.