4 ms·

If I am reading the table correctly they are claiming it is better than all models but 3.5-Sonnet Is anyone with X premium able to confirm the vibe check -- Is

by jatins 2y ago

If I am reading the table correctly they are claiming it is better than all models but 3.5-Sonnet Is anyone with X premium able to confirm the vibe check -- Is the model actually good or another case of training on benchmarks?

bekantan 2y ago
You can try it yourself on https://chat.lmsys.org https://chat.lmsys.org (sus-column-r model)
- deleted 2y ago
  [deleted]
floam 2y ago
In the intro text they described it better than Claude 3.5 Sonnet and GPT-4 Turbo (which isn’t OpenAI’s current model.)
opdahl 2y ago
My favorite part of that table is how they put 3.5-Sonnet all the way to the right of the table making it harder to compare.
modeless 2y ago
I don't think you are reading the table correctly. On LMSYS it's better than all models except the latest Gemini 1.5 Pro and GPT-4o. But there is a detailed benchmark table and different models win different benchmarks. So results are mixed, but the real takeaway is this is a competitive model that is good enough to be worth using today. It puts xAI significantly above their previous position, and up near the top of the field with OpenAI, Anthropic, Google, and Meta. And their new H100 cluster should allow them to keep up with the next wave of releases, whenever that starts.
saberience 2y ago
As with everything promised by Musk, I'll believe it when I see it and use it myself and compare it to Claude 3.5. Right now I'm not really a believer in Grok and I doubt it will be worth using.
eltoxo 2y ago
I don't personally see how an individual can judge this at this point unless it is a huge leap. More importantly, if the model is not a huge leap at this point I just don't care if it is as good as the very limited models we already have because I am not impressed by any of these anymore. Anything less than a 3.5 to 4 jump from here is just not going to vibe for me.

by jatins 2y ago

bekantan 2y ago

You can try it yourself on https://chat.lmsys.org https://chat.lmsys.org (sus-column-r model)

deleted 2y ago
[deleted]

floam 2y ago

In the intro text they described it better than Claude 3.5 Sonnet and GPT-4 Turbo (which isn’t OpenAI’s current model.)

opdahl 2y ago

My favorite part of that table is how they put 3.5-Sonnet all the way to the right of the table making it harder to compare.

modeless 2y ago

I don't think you are reading the table correctly. On LMSYS it's better than all models except the latest Gemini 1.5 Pro and GPT-4o. But there is a detailed benchmark table and different models win different benchmarks. So results are mixed, but the real takeaway is this is a competitive model that is good enough to be worth using today. It puts xAI significantly above their previous position, and up near the top of the field with OpenAI, Anthropic, Google, and Meta. And their new H100 cluster should allow them to keep up with the next wave of releases, whenever that starts.

saberience 2y ago

As with everything promised by Musk, I'll believe it when I see it and use it myself and compare it to Claude 3.5. Right now I'm not really a believer in Grok and I doubt it will be worth using.

eltoxo 2y ago

I don't personally see how an individual can judge this at this point unless it is a huge leap. More importantly, if the model is not a huge leap at this point I just don't care if it is as good as the very limited models we already have because I am not impressed by any of these anymore. Anything less than a 3.5 to 4 jump from here is just not going to vibe for me.