3 ms·
If you're comparing to ChatGPT performance then Vicuna 13B would be a best comparison point for something Llama-based.
by HarHarVeryFunny 4y ago
If you're comparing to ChatGPT performance then Vicuna 13B would be a best comparison point for something Llama-based.
- bugglebeetle 4y agoVicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff. Try giving both somewhat large, task-based prompts with steps and see what happens.
- dragonwriter 4y ago> Vicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff. Until you connect it to external resources, I tend to think of anything you do with “brain-in-a-jar” isolated ChatGPT as gimmicky conversational stuff.
- goatlover 4y agoChatGPT is still going to be way more capable when you use it's API to connect to external resources.
- HarHarVeryFunny 4y agoMaybe I should have phrased that better! I didn't mean that Vicuna was comparable to ChatGPT, just that it's the best Llama-based comparison you can make (since it's at least been conversationally trained).
- levesque 4y agoIsn't ChatGPT a 165B parameter model?
- sanxiyn 4y agoNo. OpenAI haven't disclosed parameter count of GPT-3.5 or GPT-4, which are models used by ChatGPT. You may be thinking of GPT-3, which is indeed a 175B parameter model.
- levesque 4y agoAh, interesting. Thought GPT-3.5 had the same structure as GPT-3, for some reason. GPT-4 would obviously be different.
- MacsHeadroom 4y agoGPT-3.5 is likely a finetuned Curie 13B using output from the full size GPT-3 175B.