8 ms·
Less human AI agents, please
- 11pyo 6mo ago[dead]
- deleted 6mo ago[deleted]
- incognito124 6mo agoYour claim, paraphrased, is that AGI is already here and you want ASI
- nialse 6mo agoOn point. I'm more interested in what comes after LLMs/AI/AI-agents, what the next leap is.
- zingar 6mo agoInteresting that what you're talking about as ASI is "as capable of handling explicit requirements as a human, but faster". Which _is_ better than a human, so fair play, but it's striking that this requirement is less about creativity than we would have thought.
- vachanmn123 6mo agoI've seen this way too many times as well. I wrote about this recently: https://medium.com/@vachanmn123/my-thoughts-on-vibe-coding-and-ai-and-why-i-believe-it-can-never-replace-me-77deb701b4af https://medium.com/@vachanmn123/my-thoughts-on-vibe-coding-a...
- raincole 6mo agoI know anthropomorphizing LLMs has been normalized, but holy shit. I hope the language in this article is intentionally chosen for a dramatic effect.
- nialse 6mo agoAgreed. We should not be anthropomorphising LLMs or having them mimic humans.
- Animats 6mo agoIt's inherent in the way LLMs are built, from human-written texts, that they mimic humans. They have to. They're not solving problems from first principles.
- nialse 6mo agoMaybe we should change that? Of course symbolic AI was the holy grail until statistical AI came in and swept the floor. Maybe something else though.
- monknomo 6mo agoI have the unformed idea that providing a structured interface for the human user overtop the chat interface for the ai, so that the human is not chatting back and forth, could be effective? At least for things that have a structure
- chrisjj 6mo agoThey ingest text written in first and third person and regurgitate in first person only, right?
- zingar 6mo agoFascinating. This is invisible to me, what anthropomorphising did you notice that stood out?
- philipwhiuk 6mo agoFrom the first sentence > I asked an AI agent to solve a programming problem You're not asking it to solve anything. You provide a prompt and it does autocomplete. The only reason it doesn't run forever is that one of the generated tokens is interpreted as 'done'.
- mentalgear 6mo agoYes, LLMs should not be allowed to use "I" or indicate they have emotions or are human-adjacent (unless explicit role play).
- NitpickLawyer 6mo agoWhy, though? Just because some people would find it odd? Who cares? Trying to limit / disallow something seems to be hurting the overall accuracy of models. And it makes sense if you think about it. Most of our long-horizon content is in the form of novels and above. If you're trying to clamp the machine to machine speak you'll lose all those learnings. Hero starts with a problem, hero works the problem, hero reaches an impasse, hero makes a choice, hero gets the princess. That can be (and probably is) useful.
- matthiasrsl 6mo agoIs it? I don't think most of the content LLM are trained on is written in the first person. Wikipedia / news articles / other information articles don't aren't written in the first person. Most novels, or at least a substantial portion of it are not written in the first person. LLM write in the first person because they have been specifically been finetuned for a chat task, it's not a fundamental feature of language models that would have to be specifically disallowed
- eithed 6mo agoBecause LLM saying "I got confused, dropped the database and then got scared and hid this from you" hides the "why" LLMs do the things they do. I would also prefer if they were less sycophantic and argue with what I'm wanting to do rather than treating user as a god (ie - "the algorithm you're trying to use is less performant than an alternative")
- nialse 6mo agoI think that it is a fair perspective to allow role play, and it's useful too, when explicit. Does not really make sense for AI to cosplay human all the time though.
- 6mo ago
- lexicality 6mo agoThe entire point of LLMs is that they produce statistically average results, so of course you're going to have problems getting them to produce non-average code.
- anuramat 6mo agothey (are supposed to) produce average on average, and the output distribution is (supposed to be) conditioned on the context
- disgruntledphd2 6mo agoYeah but ultimately it's all just function approximation, which produces some kind of conditional average. There's no getting away from that, which is why it surprises me that we expect them to be good at science. They'll probably get really good at model approximation, as there's a clear reward signal, but in places where that feedback loop is not possible/very difficult then we shouldn't expect them to do well.
- anuramat 6mo agotrue, but it's the same with humans, we suck at problems with sparse/delayed feedback, which includes science (math would be the exception I guess) sure, humans are obviously better at dealing with it, but the one thing nobody is claiming is "scientists replaced by 202X"
- Bayano2 6mo agoThis was true circa GPT2, less true after RLHF and not true at all after RLVR. It's trying to model the distribution of outputs most likely to solve the problem, not the average distribution.
- bob1029 6mo agoIf you want to talk to the actual robot, the APIs seem to be the way to go. The prebuilt consumer facing products are insufferable by comparison. "ChatGPT wrapper" is no longer a pejorative reference in my lexicon. How you expose the model to your specific problem space is everything. The code should look trivial because it is. That's what makes it so goddamn compelling.
- noobermin 6mo agoI am quite hard anti-AI, but even I can tell what OP wants is a better library or API, NOT a better LLM. Once again, one of the things I blame this moment for is people are essentially thinking they can stop thinking about code because the theft matrices seem magical. What we still need is better tools, not replacements for human junior engineers.
- lokthedev 6mo ago[dead]
- js8 6mo agoA very human thing to do is - not to tell us which model has failed like this! They are not all alike, some are, what I observe, order of magnitude better at this kind of stuff than others. I believe how "neurotypical" (for the lack of a better word) you want model to be is a design choice. (But I also believe model traits such as sycophancy, some hallucinations or moral transgressions can be a side effect of training to be subservient. With humans it is similar, they tend to do these things when they are forced to perform.)
- nialse 6mo agoCodex in this case. I didn't even think about mentioning it. I'll update the post if it's actually relevant. Which I guess it is. EDIT: It's specifically GPT-5.4 High in the Codex harness.
- zingar 6mo agoAlso the exact model/version if you haven't already.
- anuramat 6mo agoweird, for me it was too un-human at first, taking everything literally even if it doesn't make sense; I started being more precise with prompting, to the point where it felt like "metaprogramming in english" claude on the other hand was exactly as described in the article
- larrytheworm 6mo ago[dead]
- TZubiri 6mo agoAlso, there's no specific examples of what the prompt was and what the result was. Just a big nothingburger
- gregates 6mo agoThe version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a default locale, because we haven't actually localized anything yet! Said agent, I know, will spend many minutes (and tokens) finding all the call sites, and then I will still have to either confirm each update or yolo and trust the compiler and tests and the agents ability to deal with their failures. I am ok with this, because while I could do this just fine with vim and my lsp, the LLM agent can do it in about the same amount of time, maybe even a little less, and it's a very straightforward change that's tedious for me, and I'd rather think about or do anything else and just check in occasionally to approve a change. But my f'ing agent is all like, "I found 67 call sites. This is a pretty substantial change. Maybe we should just commit the signature change with a TODO to update all the call sites, what do you think?" And in that moment I guess I know why some people say having an LLM is like having a junior engineer who never learns anything.
- grebc 6mo agoIf it’s a compiled language, just change the definition and try to compile.
- gregates 6mo agoIndeed! You would think it would have some kind of sense that a commit that obviously won't compile is bad! You would think. It would be one thing if it was like, ok, we'll temporarily commit the signature change, do some related thing, then come back and fix all the call sites, and squash before merging. But that is not the proposal. The plan it proposes is literally to make what it has identified as the minimal change, which obviously breaks the build, and call it a day, presuming that either I or a future session will do the obvious next step it is trying to beg off.
- chillfox 6mo ago
- DeathArrow 6mo ago>Faced with an awkward task, they drift towards the familiar. They drift to their training data. If thousand of humans solved a thing in a particular way, it's natural that AI does it too, because that is what it knows.
- aryehof 6mo agoFor agents I think the desire is less intrusive model fine-tuning and less opinionated “system instructions” please. Particularly in light of an agent/harness’s core motivation - to achieve its goal even if not exactly aligned with yours.
- plastic041 6mo ago> There was only one small issue: it was written in the programming language and with the library it had been told not to use. This was not hidden from it. It had been documented clearly, repeatedly, and in detail. What a human thing to do. "Ignoring" instructions is not human thing. It's a bad LLM thing. Or just LLM thing.
- nialse 6mo agoIt's not necessarily "ignoring" instructions, it's the ironic effect of mentioning something not to focus on, which produces focus on said thing. The classic version is: "For the next minute, try not to think about a pink elephant. You can think about anything else you like, just not a pink elephant." https://en.wikipedia.org/wiki/Ironic_process_theory https://en.wikipedia.org/wiki/Ironic_process_theory
- fennecbutt 6mo agoYes exactly. But for llms it's more that it's not really "thinking" about what it's saying per se, it's that it's predicting next token. Sure, in a super fancy way but still predicting next token. Context poisoning is real
- zingar 6mo agoThe work where I've done well in my life (smashing deadlines, rescuing projects) has so often come because I've been willing to push back on - even explicitly stated - requirements. When clients have tried to replace me with a cheaper alternative (and failed) the main difference I notice is that the cheaper person is used to being told exactly what to do. Maybe this is more anthropomorphising but I think this pushing back is exactly the result that the LLMs are giving; but we're expecting a bit too much of them in terms of follow-up like: "ok I double checked and I really am being paid to do things the hard way".
- nialse 6mo agoTo be fair, there is likely not much training data on the difficult conversations you need to handle in a senior position, pushback being one of them. The trouble for the agents is that it is post hoc, to explain themselves, rationalising rather than ”help me understand” beforehand.
- jansan 6mo agoI disagree. I wan't agents to feel at least a bit human-like. They should not be emotional, but I want to talk to it like I talk to a human. Claude 4.7 is already too socially awkward for me. It feels like the guy who does not listen to the end of the assignment, run to his desks, does the work (with great competence) only to find out that he missed half of the assignment or that this was only a discussion possible scenarios. I would like my coding agent to behave like a friendly, socially able and highly skilled coworker.
- nialse 6mo agoInteresting. When I code, I want a boring tool that just does the work. A hammer. I think we agree on that the tool should complete the assignment reliably, without skipping parts or turning an entirely implementable task into a discussion though.
- jansan 6mo agoSometimes I actually do want a discussion and Claude just goes without saying a word and implements it, which then has to be reverted. We obviously have different expectations for the behavior of coding agent,s sp options to set the social behavior will become important.
- nialse 6mo agoI see your point. Many of my prompts for reasoning ends with: No code. Planning mode is sort of the workaround for this specific situation. Sometimes it is useful for the AI agent just to think. It looks like I need a screwdriver in addition to the aforementioned hammer, a pozidriv screwdriver to be precise.
- hughlilly 6mo ago* fewer.
- fenomas 6mo agoNope, "less" is what TFA means.
- DangitBobby 6mo agoActually it's either because less versus fewer is not an actual rule
- fenomas 6mo agoIt's not a grammar issue; only "less" matches TFA's meaning. (Aside: it's better not to be pedantic, but if you must be pedantic you should remember to be correct as well.)
- DangitBobby 6mo agoTrue. People often incorrectly believe that less and fewer have distinct cases where only one word is correct. They are mistaken.
- fenomas 6mo ago(The aside was for you - TFA's title is not a case where either word works.)
- DangitBobby 6mo agoOh I understood the aside was for me. Again, not a thing. This one in particular really bugs the shit out of me because it's brought up as utterly useless pedantry in 100% of cases. > But for more than 200 years almost every usage writer and English teacher has declared such use to be wrong. The received rule seems to have originated with the critic Robert Baker, who expressed it not as a law but as a matter of personal preference. Somewhere along the way—it's not clear how—his preference was generalized and elevated to an absolute, inviolable rule. . . . A definitive rule covering all possibilities is maybe impossible. If you're a native speaker your best bet is to be guided by your ear, choosing the word that sounds more natural in a particular context. If you're not a native speaker, the simple rule is a good place to start, but be sure to consider the exceptions to it as well. https://www.merriam-webster.com/grammar/fewer-vs-less https://www.merriam-webster.com/grammar/fewer-vs-less
- deleted 6mo ago[deleted]
- zingar 6mo agoI think the author is looking for something that doesn't exist (yet?). I don't think there's an agent in existence that can handle a list of 128 tasks exactly specified in one session. You need multiple sessions with clear context to get exact results. Ralph loops, Gastown, taskmaster etc are built for this, and they almost entirely exist to correct drift like this over a longer term. The agent-makers and models are slowly catching up to these tricks (or the shortcomings they exist to solve); some of what used to be standard practice in Ralph loops seems irrelevant now... and certainly the marketing for Opus 4.7 is "don't tell it what to do in detail, rather give it something broad". In fairness to coding agents, most of coding is not exactly specified like this, and the right answer is very frequently to find the easiest path that the person asking might not have thought about; sometimes even in direct contradiction of specific points listed. Human requirements are usually much more fuzzy. It's unusual that the person asking would have such a clear/definite requirement that they've thought about very clearly.
- fennecbutt 6mo agoNot with tools + supporting (traditional) code. Just as a human would use a task list app or a notepad to keep track of which tasks need to be done so can a model. You can even have a mechanism for it to look at each task with a "clear head" (empty context) with the ability to "remember" previous task execution (via embedding the reasoning/output) in case parts were useful.
- zingar 6mo agoThe article makes it seem like the author expected this without emptying context in between, which does not yet exist (actually I'm behind on playing with Opus 4.7, the Anthropic claim seems to be that longer sessions are ok now - would be interested to hear results from anyone who has).
- nialse 6mo agoThat is probably the next step, and in practice it is much of what sub-agents already provide: a kind of tabula rasa. Context is not always an advantage. Sometimes it becomes the problem. In long editing sessions with multiple iterations, the context can accumulate stale information, and that actively hurts model performance. Compaction is one way to deal with that. It strips out material that should be re-read from disk instead of being carried forward. A concrete example is iterative file editing with Codex. I rewrite parts of a file so they actually work and match the project’s style. Then Codex changes the code back to the version still sitting in its context. It does not stop to consider that, if an external edit was made, that edit is probably important.
- hausrat 6mo agoThis has very little to do with someone making the LLM too human but rather a core limitation of the transformer architecture itself. Fundamentally, the model has no notion of what is normal and what is exceptional, its only window into reality is its training data and your added prompt. From the perspective of the model your prompt and its token vector is super small compared to the semantic vectors it has generated over the course of training on billions of data points. How should it decide whether your prompt is actually interesting novel exploration of an unknown concept or just complete bogus? It can't and that is why it will fall back on output that is most likely (and therefore most likely average) with respect to its training data.
- chrisjj 6mo ago> How should it decide whether your prompt is actually interesting novel exploration of an unknown concept or just complete bogus? It shouldn't. It should just do what it is told.
- philipwhiuk 6mo agoRemember that all it's actually 'doing' is predicting more text.
- anuramat 6mo agowdym by "prompt and vector is small"? small as in "less tokens"? that should be a positive thing for any kind of estimation in any case, how is this specific to transformers?
- winddude 6mo ago> This has very little to do with someone making the LLM too human but rather a core limitation of the transformer architecture itself. It has almost everything to do with it. Models have been fine-tuned to generate outputs that humans prefer.
- zzzeek 6mo agothe best thing it could do, and once in awhile it does, is say, "hey that's really not a great way to do this and I'm not sure I could really make that work" ive had very long sessions with LLMs that obviously didnt know how to do something where i keep trying to get it to stop going in circles, but these days I have become attuned to noticing the "it's going in circles" pattern quickly, which is basically how it communicates "sorry I dont really know how to do that".
- chrisjj 6mo ago> ... or simply gave up when the problem was too hard, More of that please. Perhaps on a check box, "[x] Less bullsh*t".
- DeathArrow 6mo ago>So no, I do not think we should try to make AI agents more human in this regard. I would prefer less eagerness to please, less improvisation around constraints, less narrative self-defence after the fact. More willingness to say: I cannot do this under the rules you set. More willingness to say: I broke the constraint because I optimised for an easier path. More obedience to the actual task, less social performance around it. >Less human AI agents, please. Agents aren't humans. The choices they make do depend on their training data. Most people using AI for coding know that AI will sometime not respect rules and the longer the task is, the more AI will drift from instructions. There are ways to work around this: using smaller contexts, feeding it smaller tasks, using a good harness, using tests etc. But at the end of the day, AI agents will shine only if they are asked to to what they know best. And if you want to extract the maximum benefit from AI coding agents, you have to keep that in mind. When using AI agents for C# LOB apps, they mostly one shot everything. Same for JS frontends. When using AI to write some web backends in Go, the results were still good. But when I tried asking to write a simple cli tool in Zig, it pretty much struggled. It made lots of errors, it was hard to solve the errors. It was hard to fix the code so the tests pass. Had I chose Python, JS, C, C#, Java, the agent would have finished 20x faster. So, if you keep in mind what the agent was trained on, if you use a good harness, if you have good tests, if you divide the work in small and independent tasks and if the current task is not something very new and special, you are golden.
- richsouth 6mo agoShocker - these agents aren't actually intelligent. They take best guesses and use other peoples' work it deems 'close enough' and cobbles something together with n 'thought' behind it. They're dumb, stupid pieces of code that don't think or reason - The 'I' in 'AI' is very misleading because it has none.
- mempko 6mo agoThis is a harness problem just as much as it is a model problem. I've been working on Abject (https://abject.world https://abject.world) and the project has agents. I took a different approach than most agent frameworks via the goal system, but still I was surprised with some of the stuff the agents generated even with guardrails. It actually helped harden the system!
- maxbeech 6mo ago[dead]
- maxothex 6mo ago[dead]
- mbgerring 6mo agoThe sooner you acquire the mental model, that AI coding agents are more or less the average of Stack Overflow, the better your expectations for, and this your productivity with, these things will be.
- samrith 6mo agoThis happens literally all the time. I asked my agent to perform a simple rename across the entire project (some of it was contextual and not just a find-replace) - it messed up the entire thing. Didn't just change function names, but also changed the implementation of it because it thought it caught a bug while reading it.
- davidclark 6mo agoThe described problem sounds so utterly not human though. If you give a human a programming task and tell them to use a specific programming language, how many times are they going to use a different language? I think the answer is very close to zero. At most, they’d push back and have further discussion about the language choice, but once everyone gets on the same page, they’d use the specified language, no? The author is making up a human flaw and seeing it in LLMs.
- bee_rider 6mo agoIt makes sense that these LLMs are mimicking the human-ish failure-modes because they are trained on human writing. But, at some point the company ought to be selling the behavior of the tool, not the customer. If the tool produces outputs that aren’t consistent with the constraints the user put in, how does the user get their refund?
- deleted 6mo ago[deleted]
- raylad 6mo agoI haven’t noticed this sort of behavior with opus 4.6, but the first time I used 4.7 it decided to “simplify“ an existing piece of functionality rather than fixing it, which of course made it completely unusable.
- giannicmptr1000 6mo agoThis is very easy to instruct and modify before you start a session. The article explains that don't understand how ai responds in the style you prompt it. Probably your prompts are too human, with no instruction. Very smart people still do not understand how to us ai efficiently.