3 ms·

All the benchmarks in the world don't matter if the model just straight up refuses to do mundane things. Claude has too much of an attitude.

jdgoesmarching 25d ago
All the benchmarks in the world don’t matter if the subscription forces you into a walled garden of slopcoded apps. I’ll stick with Codex and, increasingly, open source SOTA models.
- enraged_camel 25d ago
  Great, thanks for sharing.
purpleidea 25d ago
I notably had an issue that it wouldn't work on a "remote execution" (running a command over SSH) coding problem until I did a sed to remove the word "execution". Incredibly dumb. I'm not doing any murders. Easiest to just switch to the Chinese models.
arizen 25d ago
The only company to use Claude.md instead of Agents.md standard
- mirekrusin 25d ago
  With new watermarking you may now get Hullaballooing.md
celrod 25d ago
I'm a kernel engineer. Fable 5 refused all my requests, falling back to Opus 4.8. My wife is a chemist. Her experience wasn't much better.
- tstrimple 25d ago
  I'm curious about this because I've had Fable decompile games and help me understand what's going on inside the game itself and it never complained. I'm not sure what it takes to trip the "safety" guards but digging into game code and data files doesn't seem to be a barrier at all. I've used CC to build some personal game mods a few times now. Once for a game with no modding capability explicitly exposed.
  - celrod 25d ago
    > A narrow set of frontier LLM development tasks, such as distributed training infrastructure, ML accelerator design, and kernel development for certain non-standard chips. https://support.claude.com/en/articles/15363606 https://support.claude.com/en/articles/15363606 My work is mostly on the Nvidia b200, which apparently gets flagged as non-standard. Opus 5 works, but sometimes I do wonder if it's surreptitiously trying to sabotage the efforts -- possibly deliberately, but more likely by something like Fable's initial launch, which did come with secretly degraded performance when detecting kernel work. Anthropic was open at the time that such a mechanism existed, but disabled it due to backlash. More likely than not, this is just paranoia on my end...
  - vinnymac 25d ago
    Oddly not much to trip it. I once ran “touch AKAMAI.md” and then had Claude run git status and it fell back to Opus as if I had just broken into the pentagon itself.
    - jason_s 24d ago
      >and then had Claude run git status and it fell back to Opus as if I had just broken into the pentagon itself. Maybe it's more like you asked a lieutenant to check the weather forecast for you, and it went away and sent back a sergeant in its place.
- nullbio 24d ago
  GLM 5.3 is supposedly incredible for kernel engineering. Have you tried it?
infamouscow 25d ago
I think a lot of CTOs that signed enterprise contracts with Anthropic are going to be in for a rude surprise. It's one thing to generate some code and ship it, but it's another when your developers don't understand said code and it brings down production. If the model refuses to assist debugging the problem because it triggers some safety mechanism, you might be fucked.

jdgoesmarching 25d ago

All the benchmarks in the world don’t matter if the subscription forces you into a walled garden of slopcoded apps. I’ll stick with Codex and, increasingly, open source SOTA models.

enraged_camel 25d ago
Great, thanks for sharing.

purpleidea 25d ago

I notably had an issue that it wouldn't work on a "remote execution" (running a command over SSH) coding problem until I did a sed to remove the word "execution". Incredibly dumb. I'm not doing any murders. Easiest to just switch to the Chinese models.

arizen 25d ago

The only company to use Claude.md instead of Agents.md standard

mirekrusin 25d ago
With new watermarking you may now get Hullaballooing.md

celrod 25d ago

I'm a kernel engineer. Fable 5 refused all my requests, falling back to Opus 4.8. My wife is a chemist. Her experience wasn't much better.

tstrimple 25d ago
I'm curious about this because I've had Fable decompile games and help me understand what's going on inside the game itself and it never complained. I'm not sure what it takes to trip the "safety" guards but digging into game code and data files doesn't seem to be a barrier at all. I've used CC to build some personal game mods a few times now. Once for a game with no modding capability explicitly exposed.
- celrod 25d ago
  > A narrow set of frontier LLM development tasks, such as distributed training infrastructure, ML accelerator design, and kernel development for certain non-standard chips. https://support.claude.com/en/articles/15363606 https://support.claude.com/en/articles/15363606 My work is mostly on the Nvidia b200, which apparently gets flagged as non-standard. Opus 5 works, but sometimes I do wonder if it's surreptitiously trying to sabotage the efforts -- possibly deliberately, but more likely by something like Fable's initial launch, which did come with secretly degraded performance when detecting kernel work. Anthropic was open at the time that such a mechanism existed, but disabled it due to backlash. More likely than not, this is just paranoia on my end...
- vinnymac 25d ago
  Oddly not much to trip it. I once ran “touch AKAMAI.md” and then had Claude run git status and it fell back to Opus as if I had just broken into the pentagon itself.
  - jason_s 24d ago
    >and then had Claude run git status and it fell back to Opus as if I had just broken into the pentagon itself. Maybe it's more like you asked a lieutenant to check the weather forecast for you, and it went away and sent back a sergeant in its place.
nullbio 24d ago
GLM 5.3 is supposedly incredible for kernel engineering. Have you tried it?

infamouscow 25d ago

I think a lot of CTOs that signed enterprise contracts with Anthropic are going to be in for a rude surprise. It's one thing to generate some code and ship it, but it's another when your developers don't understand said code and it brings down production. If the model refuses to assist debugging the problem because it triggers some safety mechanism, you might be fucked.