3 ms·
They don't benchmark against the OpenAI models at all for some reason. In my experience so far even GPT-3.5 was so much better than any OSS models I tried. Met
by crop_rotation 3y ago
They don't benchmark against the OpenAI models at all for some reason. In my experience so far even GPT-3.5 was so much better than any OSS models I tried.
Meta must be missing having a cloud business at this point. AI would have fit perfectly with that. Since they ran Whatsapp for so many years kinda without interfering too much, they could have also tried a somewhat independent cloud unit.
- whimsicalism 3y agoYou don't benchmark foundation model against RLHF model, results aren't very useful.
- moffkalast 3y agoThis does seem to be a RLHF model, not a base model. Unless 'supervised fine-tuning' and 'human preference' mean something else.
- whimsicalism 3y agoAh I see there is also a llama-2-chat model.
- gloryjulio 3y agoWith the meta chaotic internal culture, it's hard to handle the cloud as a business. They would be even worse than google cloud
- supermdguy 3y agoLooks like it comes in just under GPT-3.5 (based on page 7 in the GPT-4 report https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf)
- weird-eye-issue 3y agoThat is unrelated. Stop spreading misinformation. It is for the old version and not this new one
- madisonmay 3y agoSee figure-2
- alibero 3y agoCheck out figures 1 & 2 in the Llama-2 paper :) They benchmark against ChatGPT for helpfulness and harmfulness https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/ https://ai.meta.com/research/publications/llama-2-open-found...