3 ms·
First I saw it was Claude Opus 3.7. Had a very long back and fourth about some code, I pointed out an error, and Claude responded "That's what I get for program
by nomel 5mo ago
First I saw it was Claude Opus 3.7. Had a very long back and fourth about some code, I pointed out an error, and Claude responded "That's what I get for programming at 2am", with the output being filled with "... code here ..." type shortcuts, basically no ability to one-shot a whole implementation anymore. The conversation length WAS reasonably into the 2am range, if it were real. Thought about it, did the statistical trick where I tell it to "have some rest, take a day off!" then immediately follow up with "Ready to continue?", with the next response having no shortcuts, with full implementation, and much better quality. These are trained on human text. This is the human norm, so I always find it interesting when human like behaviors, very broadly present in the statistics, come out like this.
I also see it a little with Opus 4.7, with Claude Code, with the hint being much more terse planning text, that borderlines unhelpful. I put some "rest" in the context to push the latent space closer to what's in the statistics of the training data: a well rested human.
- the_af 5mo agoHow do you know you're not reading things that aren't there? LLMs are very good at roleplaying, and they will pick up on hints you may inadvertently be giving them (about them being "tired" and needing "rest", etc). I have never witnessed this of Claude Opus, by the way. They do get context rot, but that's a relatively better understood phenomenon unrelated to personality.
- deleted 5mo ago[deleted]
- deleted 5mo ago[deleted]
- deleted 5mo ago[deleted]
- nomel 5mo ago> LLMs are very good at roleplaying Yes, and I think this is where it's coming from. They're role playing as a human programmer, because near 100% of the training text, in the base model, is humans as a programmer. During fine tuning, I'm sure they spend significant resources remove the human aspects of the statistics. I see these things reduced each model, so there's something changing. They're probably getting better at that. I suspect Claude is also necessarily getting, worse, which the unaligned models should necessarily be best at (quick google search in some role-play subreddits seems to point in this direction).
- pj_mukh 5mo agoAre you sure you didn't run out credits and set effort to low? This exact thing happened to me when I did that. It just became, kinda lazy.
- nomel 5mo ago3.7 "I'm tired" it was just direct API "chat", no CC that I could use at the time. Current 4.7 Opus with claude code, with effort pinned to max, because I'm on an API only plan, with a personal daily limit you would probably be jealous of. ;)