6 ms·
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answe
by iamcoder18 1mo ago
I've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration.
> In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster.
This is actually insane.
Hopefully the release ultrafast of Terra and Luna too.
- piyh 1mo agoFeels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.
- sscaryterry 1mo agoYep, https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law
- beng-nl 1mo agoI wish, but I don’t think this qualifies as single thread performance. Cerebras is whole wafer computation though right? So you get the result sooner, but the same amount of computation has occurred, just with huge parallelism.
- wrsh07 1mo agoSeems like they will do Sol first while capacity constrained? I can't imagine the margins they'll be charging
- sscaryterry 1mo agoI think this is what they need to actually have a margin :)
- zozbot234 1mo agoAnswering 2,500 independent questions is an embarrassingly parallel workload, all it needs is scale out. It would be more meaningful to know how much time was required for a single complete answer to a difficult HLE question.
- desmondl 1mo agoThe animation in the article shows GPT-5.6 Sol Ultrafast solving a question at 3 secs vs Fable's 27 secs. The "11 hour vs 78 hours" animation implied that they were solved in sequence and not in parallel
- whimsicalism 1mo agoi assume they meant serially....
- jcims 1mo ago>Answering 2,500 independent questions is an embarrassingly parallel workload So is calculating the total time required to answer all of the questions.
- baq 1mo agoAmadahl’s law dictates it’s approximately always better (as in, more efficient computationally) to have one super fast thing than many slower things doing the same job in parallel.
- moralestapia 1mo agoSuch a deep and smart insight. But why is that important if they're measuring time?
- moralestapia 1mo agoAnd just like that, GP disappears forever ...
- xyzsparetimexyz 1mo agoYes. but for embarrassingly parallel tasks it barely matters.
- sixtyj 1mo agoOutput from Cerebras with GPT model is 750 tokens per second. Don’t blink. (Chatjimmy has 14,200 TPS.)
- tomrod 1mo agoChatJimmy is a much smaller model and, AFAIK, has no reasoning capability. Absolutely insane raw speed, like a supercar, while Sol is more like a freight truck.
- sixtyj 1mo agoAt such output speed, I wouldn’t expect reasoning. (But I didn’t know it, thanks.) 700 TPS with reasoning is awesome and it speeds things up. Cerebras as public traded company is worth keeping an eye what they produce.
- msdz 1mo ago> At such output speed, I wouldn’t expect reasoning. As the sibling comment to yours mentioned, if they had a reasoning model “hardware-ified” onto a custom chip (as is their plan for IIRC this or next year, a new ASIC), it’d output fast decode speeds for the regular output as well as reasoning sections. Both would be ≈equally fast.
- senordevnyc 1mo agoYeah, I thought reasoning was literally just chain of thought in the output token stream, with the model itself adding delimiters to indicate what part of the output is internal reasoning, and what part is an answer to the user. Is that wrong?
- beering 1mo agoYou are right, reasoning is unrelated to tokens per second.
- 1mo ago
- rvz 1mo agoBeen waiting since Cerebras-GPT. [0] [0] https://news.ycombinator.com/item?id=35490837 https://news.ycombinator.com/item?id=35490837
- christoff12 1mo agoAn irrational gripe of mine is how GPT uses 7× instead of 7x. I recognize that the former is the multiplication symbol, but I don't think it should be used that way.
- Daneel_ 1mo agoIf anything, writing "7x" is actually just a poor emulation of writing "7×". Saying something is "done at 7x speed" should be read as "done at seven times speed" not as "done at seven x speed". So using the 'times' (multiplication) symbol is the better form in my opinion; it just happens to be significantly easier to type "x" instead, which is how we got here.
- TacticalCoder 1mo agoBut speech evolves. And we all pronounce "ten ex", not "ten times". Actual speech usage very clearly pronounce the letter 'x'. So it's 10x. And no need for Unicode codepoints.
- Daneel_ 1mo agoI fully agree that speech evolves, but maybe this is just in the US? I've never really heard anyone say it here in Australia, it's usually "times", although admittedly almost no one uses the phrase/format here.
- christoff12 1mo agoAh, yes. My gripe is certainly US American flavored.
- shussjjs 1mo agoA number plus an “x” is used beyond Trumpistan. I have never in my life heard anyone say “ten ex”.
- _flux 1mo ago
- andai 1mo agoI'm finding Luna suprisingly adequate for my work. I slept on it due to the benchmarks, but it's very fast and even on low reasoning I'm finding it more than adequate for "menial" work. (The speed is crucial for "interactive" work -- if a model is fast enough it goes from "async" to "real time", subjectively, which is a huge difference.) In fact, I'd say it's overqualified for the kind of work I'm doing, because it spends >half the time verifying trivial changes (and the verification isn't as helpful as you'd expect, even with bigger models). Maybe I can prompt it to be less aggressive about that (the new GPT models do it even without prompting). Anyway, Ultrafast Luna would be amazing, though I strongly doubt they can offer Cerebras at anything approaching the current prices. Now we wait for Moore's Law? :)
- epolanski 1mo agoI feel odd to use these models, because it feels like a faster model doesn't feel that much faster if it spends reading files, making edits and running checks. It feels too situational.
- andai 1mo agoWhich one of those takes time?
- phoghed 1mo agoLuna has impressive results on DeepSWE. I mostly split work between Luna and Sol. If something seems simple enough I always try it with Luna first.
- shepherdjerred 1mo agoLuna is incredible considering the cost. It’s very good for routine work.
- trollbridge 1mo agoLuna and Terra are great, but so are DeepSeek Flash, MiMo, Qwen 3.6, and even Gemma for some things. MiMo Pro has had UltraSpeed for a while.
- 1mo ago
- aetherspawn 1mo agoI discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency. Dollar for tokens, Sol and Fable are the same price. However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task. We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day. And, due to less guardrails, Sol also performed better, and we lost less tokens due to guardrails shutting down sessions (I feel like it’s illegal to take $50 of someone’s token money and then shut down a session with guardrails before they get an answer, and yet Anthropic do it to us constantly… either take our money and commit, or trigger the guardrails immediately)
- minraws 1mo agoWait isn't Fable like 2x more expensive if we compare under 272k tokens
- ChadMoran 1mo agoThe comment you're replying to almost feels like it was written by a bot or am I crazy?
- w4yai 1mo agoI agree. Weird to use <“> and <”> characters. Maybe written on phone, but certainly not on keyboard.
- gunalx 1mo agoCommon on non us keebs though.
- jrflo 1mo agoOther languages use different characters for quotes, if anything that's an indication that's not written by a LLM because it's not favoring the standard English character. https://en.wikipedia.org/wiki/Quotation_mark#Specific_language_features https://en.wikipedia.org/wiki/Quotation_mark#Specific_langua...
- krzyk 1mo agoIsn't ultrafast just making hundreds of subagents?
- anentropic 1mo agoNo, that's Claude "ultracode"