4 ms·
Seems like it is indeed the new SOTA model, with significantly better scores than o3, Gemini, and Claude in Humanity's Last Exam, GPQA, AIME25, HMMT25, USAMO 20
by modeless 1y ago
Seems like it is indeed the new SOTA model, with significantly better scores than o3, Gemini, and Claude in Humanity's Last Exam, GPQA, AIME25, HMMT25, USAMO 2025, LiveCodeBench, and ARC-AGI 1 and 2.
Specialized coding model coming "in a few weeks". I notice they didn't talk about coding performance very much today.
- esafak 1y agoI wish the coding models were available in coding agents. Haven't seem them anywhere.
- vincent_s 1y agoGrok 4 is now available in Cursor.
- markdog12 1y agoInteresting, I have the latest update and I don't see it in the models list.
- lexarflash8g 1y agoYou have to go to the settings and view more models and select it from the drop-down list.
- apparent 1y agoI had to go to add more models, and then it was available. So far, it is able to do some things that other models were not previously able to do.
- dmix 1y agoI just tried it, it was very slow like Gemini. But I really liked the few responses it gave me, highly technical language. Not the flowery stuff you find in ChatGPT or Gemini, but much more verbose and thorough than Claude.
- theshrike79 1y agoI like that Grok doesn't kiss my ass like Gemini and ChatGPT keep doing with their "excellent idea!" -crap.
- justarobert 1y agoPlenty like Aider and Cline can connect to pretty much any model with an API.
- mhoad 1y ago[flagged]
- Workaccount2 1y agoWell we have GPT-5 and Gemini 3 in the wings so it wouldn't be surprising if it is SOTA for a few days.
- monkeydust 1y agoyup this will probably trigger the next wave of releases, someone had to go first.
- sailingparrot 1y agoxAI, with OAI just a few weeks before, were the first to get a cluster up of a sufficient size to train a GPT-5 like model. xAI released this as fast as they could, it hasn't been sitting on shelf for month, and neither has GPT-5.
- deleted 1y ago[deleted]
- vessenes 1y agoAgreed. I noticed a quick flyby of a bad “reasoning smell” in the baseball World Series simulation, though - it looks like it pulled some numbers from polymarket, reasoned a long time, and then came back with the polymarket number for the Dodgers but presented as its own. It was a really fast run through, so I may be wrong, but it reminds me that it’s useful to have skeptics on the safety teams of these frontier models. That said, these are HUGE improvements. Providing we don’t have benchmark contamination, this should be a very popular daily driver. On coding - 256k context is the only real bit of bad news. I would guess their v7 model will have longer context, especially if it’s better at video. Either way, I’m looking forward to trying it.
- dbagr 1y agoEither they overtook other LLMs by simply using more compute (which is reasonable to think as they have a lot of GPUs) or I'm willing to bet there is benchmark contamination. I don't think their engineering team came up with any better techniques than used in training other LLMs, and Elon has a history of making deceptive announcements.
- z7 1y agoHow do you explain Grok 4 achieving new SOTA on ARC-AGI-2, nearly doubling the previous commercial SOTA? https://x.com/arcprize/status/1943168950763950555 https://x.com/arcprize/status/1943168950763950555
- saberience 1y agoThey could still have trained the model in such a way as to focus on benchmarks, e.g. training on more examples of ARC style questions. What I've noticed when testing previous versions of Grok, on paper they were better at benchmarks, but when I used it the responses were always worse than Sonnet and Gemini even though Grok had higher benchmark scores. Occasionally I test Grok to see if it could become my daily driver but it's never produced better answers than Claude or Gemini for me, regardless of what their marketing shows.
- zamalek 1y ago> Seems like it is indeed the new SOTA model, with significantly better scores than o3 It has been demonstrated for quite some time that censoring models results in drastically reduced scores. Sure, maybe prevent it from telling somehow how to build a bomb, but we've seen Grok 3 routinely side with progressive views despite having access to the worst of humanity (and its sponsor).
- fdsjgfklsfd 1y agoWait, are you implying that Grok 3 is "censored" because it aligns with "progressive" views?
- strangefellow 1y agoI think they're implying that Grok is smarter because it's less censored, and then separately noting that it still tends to be fairly progressive despite the lack of censorship (when it's not larping as Hitler) even though it was presumably trained on the worst humanity has to offer. Man, that sentence would have been incomprehensible just a couple years ago.
- zamalek 1y agoThat's what I was going for.
- Rover222 1y agoAs has been the case in almost all US social media companies until the last year. They were all heavily biased and censored towards left-leaning views.
- Squarex 1y agoEven if one does not have a positive view of Elon Musk, the catching up of Grok to the big three (Google, OpenAI, Anthropic) is incredible. They are now at the same level aproximately.