XCSme

15 ms·

391.
by XCSme 7mo ago
Well, in that case, the difference is quite minimal between 5 mini and 5.4 mini 5.4 mini seems to be a lot more wild/unstable, but with this instability it gets the right answer more often. https://aibenchy.com/compare&
392.
by XCSme 7mo ago
It's odd, that on many benchmarks, including mine[0], Nano does better than Mini. 5.4 mini seems to struggle with consistency, and even with temperature 0 sometimes gives the correct response, sometimes a wrong one... [0]: https:&#x2F
393.
by XCSme 7mo ago
Plus the ads on cable TV, I don't think there is any way to remove them
394.
by XCSme 7mo ago
In my tests it's worst with adding extra formatting or output: https://aibenchy.com/compare/anthropic-claude-opus-4-6-mediu... For example, sometimes it outputs in markdown, without being asked to (e.g. "**13
395.
by XCSme 7mo ago
To be honest, I had this "issue" too. I upgraded to a new model (gpt-4o-mini to grok-4.1-fast), suddenly all my workflows were broken. I was like "this new model is shit!", then I looked into my prompts and realized the
396.
by XCSme 7mo ago
Claude is quite bad at following instructions compared to other SOTA models. As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."
397.
Grok 4.20 brings minimal improvements over Grok-4.1-fast(aibenchy.com)
2 points by XCSme 7mo ago|1 comments
398.
by XCSme 7mo ago
It is also 10x more expensive, but also 4-5x faster. The multi-agent version is 100x more expensive, 4-5x faster, but for basic tasks (without tool calling) gives a lot of wrong answers.
399.
by XCSme 7mo ago
Blockchain!
400.
by XCSme 7mo ago
I don't know, it looks like the neurons are triggering quite randomly. Also they didn't fully explain the reward mechanism.
401.
by XCSme 7mo ago
My partner (31yo) died of cancer on Christmas day last year. Since then, it is hard, but also beautiful once you realize life is too short to be upset about things or postpone doing what makes you happy. I reach out to people more now, and
402.
by XCSme 7mo ago
Started on making my own AI model benchmarks and leaderboard[0], after I tested MiniMax M2.5, which was supposedly good based on standard benchmarks, but peformed really poorly in practice and burned through hundreds of thousands of reasoni
403.
by XCSme 7mo ago
I used their free tier. The account creation process was really confusing, and they kept turning off my instance because usage was not high enough. It seemed quited oudated/confusing to use last time I tried it a few years ago.
404.
by XCSme 7mo ago
So, to give an example of my idea, as some people asked why is there a "reasoning model" when I said non-reasoning. The "reasoning model" is just an instant model trained to output text that looks like reasoning. user: &
405.
Why Not Boost?(twitter.com)
1 points by XCSme 7mo ago|1 comments
406.
Show HN: AI Benchy – AI benchmarks and comparisons(aibenchy.com)
1 points by XCSme 8mo ago|0 comments
407.
by XCSme 8mo ago
I actually made it, so I'm not sure if it has credibility, but the tests are simply various (quite simple) questions, and models are just tested on it. I am also surprised Gemini 3 Flash does so well (note that only the MEDIUM reasonin
408.
by XCSme 8mo ago
Looking ok, but nothing special: https://aibenchy.com/model/openai-gpt-5-4-medium/
409.
by XCSme 8mo ago
What I'm most confused, is why call it both GPT-5.3 Instant and gpt-5.3-chat?
410.
by XCSme 8mo ago
Seems to be quite similar to 5.3-codex, but somehow almost 2x more expensive: https://aibenchy.com/compare/openai-gpt-5-4-medium/openai-gp...
411.
by XCSme 8mo ago
Google kept promoting the "speed" of the model, so I guess it will be useful for some close to real-time use-cases, maybe live chat/support (?)
412.
by XCSme 8mo ago
https://aibenchy.com/compare/google-gemini-3-1-flash-lite-pr...
413.
by XCSme 8mo ago
Yup, MiniMax M-2.5 is a standout in that aspect. It's $/token is very low, because it reasons forever (fun fact, that's also the reason why it's #1 on OpenRouter, because it simply burns through tokens, and OpenRouter ra
414.
by XCSme 8mo ago
Surprised they have a lot of text/numbers but no charts.
415.
by XCSme 8mo ago
It also seems to do same or slightly worse than gpt-5.2-chat on my benchmarks. No wonder they didn't show any benchmarks in their blog post (?) https://aibenchy.com/compare/openai-gpt-5-2-chat-none/openai...
416.
by XCSme 8mo ago
Unless you set Gemini 3.1 Flash-lite to HIGH, then it uses a crazy amount of tokens for reasoning (and also leads to a worse result, maybe it's bugged on high?)
417.
by XCSme 8mo ago
Gemini 3.1 Lite with no reasoning does better than Gpt-5.3 with no reasoning? https://aibenchy.com/compare/google-gemini-3-1-flash-lite-pr...
418.
by XCSme 8mo ago
> 3.1 Flash-Lite (reasoning) (reasoning) doesn't say much. Is it low/med/high reasoning? I ran my own benchmarks, and 3.1 Flash-Lite on high costs A LOT: https://aibenchy.com/compare/google-gemini-3-1
419.
by XCSme 8mo ago
They said it's available in the API too, in the blog post. EDIT: > GPT‑5.3 Instant is available starting today to all users in ChatGPT, as well as to developers in the API as ‘gpt-5.3-chat-latest.’ Updates to Thinking and Pro will f
420.
by XCSme 8mo ago
Yeah, but that's why many blockers block it, right?

More ›