3 ms·
Which open model comes closest to GPT-3.5 in your production workload, if you don't mind me asking?
by akiselev 4y ago
Which open model comes closest to GPT-3.5 in your production workload, if you don't mind me asking?
- bugglebeetle 4y agoNone of them really, because I use complex prompts with task breakdowns that no other models beside OpenAI’s seem capable of processing. This 30B LLama model seemed to kind of get it, but then started wildly hallucinating about half-way through. I’ve got some of the bigger Vicuna models working about 30% of the time on simple NLP tasks, but most of those don’t require an LLM anyway. They might perform better if you fine-tune them for whatever particular job, but that kind of defeats the purpose. The advantage of LLMs is supposed to be their generalized capabilities.
- xbmcuser 4y agoI think most people don't realise that OpenAi biggest advantage is the billions of queries it was asked those signals are what they used to optimise it. So I think very hard for a local model to reach similar capability.
- pmontra 4y agoI wonder how they use those queries for training. Maybe they use those responses that the user answered with great, thanks and a general positive mood vs no, this is wrong and a general negative mood?
- j45 4y agoThis thread made me wonder if there is an AI Chatbot testing/comparison framework as of yet.