3 ms·

> The process of coding with an “agentic” LLM appears to be the process of carefully distilling all the worst parts of code review, and removing and discarding

> The process of coding with an “agentic” LLM appears to be the process of carefully distilling all the worst parts of code review, and removing and discarding all of its benefits. This has been my experience as well, especially since the only space I have to work with agents are on C++ projects where they flat-out spiral into an increasingly dire loop of creating memory leaks, identifying the memory leaks they created, and then creating more memory leaks to fix them. There are probably some fields or languages where these agents are more effective-I've had more luck with small tasks in JS and Python for sure. But I've burned a full work week trying and falling to get Claude to add some missing destructors to a working but leaky C++ project. At one point I let it run fully unattended on the repo in a VM for four hours with the goal of adding a destructor to a class. It added nearly 2k broken LOC that couldn't compile because its first fix added an undeclared destructor, and its response to that failure was to do the same thing to _every class in the project_. Every time saying "I now know what the problem is" as it then created a new problem. If LLMs could just lack confidence in themselves for even a moment and state that they don't know what the problem is but it's willing to throw spaghetti at the wall to find out, I could respect that more than it marching through with absolute confidence that its garbage code did anything but break the project.

bcrosby95 1y ago
I tried using them for a game server, written in C, as a test case and I came to the same conclusions. It did pretty well at making something that was buggy but worked, but increasingly continually fell apart around the 3k lines mark, which really isn't much code. It's not like these are esoteric languages, so I'm not sure why they (in my experience) have such a big problem with them. My current guess is that C and C++ aren't so mono domain. The languages its good at seems to be languages used predominantly in the web world, and most of the open code out there is for web. I'm curious if anyone has tried to use it for C# in Unity or a bespoke game in Rust and how well it does there.
- starkparker 1y ago
  I'm OK with an LLM not knowing how to do something. The part of working with it that I can't get past is its full-throated confidence in its actions. It'll break down, step by step, why its actions would solve the problem, then attempts to build it and can't recognize how or why its changes made the problem worse.
- crq-yml 1y ago
  I did spend a little while evaluating what GPT could output for a fairly intensive gaming task - implement a particular novel physical dynamics model. (Yes, there are newer tools now: I haven't used them and don't want to pay to use them.) And what it did was impressive in a certain sense in that it wrote up a large body of code with believable elements, but closer examination demonstrated that, of course, it wasn't actually going to function as-is, it made up some functions, it left others stubbed. I would never trust it to manage memory or other system resources correctly, because AFAIK nobody has implemented that kind of "simulate the program lifecycle" bookkeeping into the generated output. There's a domain that I do find some application for it in, which is "configure ffmpeg". That and tasks within game engines that are similar in nature, where the problem to be solved involves API boilerplate, do benefit. What also works, to some degree, is to ask it to generate code that follows a certain routine spec, like a state machine. Again, very boilerplate-driven stuff, and I expect an 80% solution and am pleasantly surprised if it gets everything. And it is very willing to let me "define a DSL" on the fly and then act as a stochastic compiler, generating output patterns that are largely in expectation with the input. That is something I know I could explore further. And I'm OK with this degree of utility. I think I can refine my use of it, and the tools themselves can also be refined. But there's a ceiling on it as a directly impactful thing, where the full story will play out over time as a communication tech, not a "generate code" tech, with the potential to better address Conway's law development issues.
deburo 1y ago
It seems quite obvious as surely the amount of C++, C, etc. is less, in the public domain, than the millions of lines of JS, C#, Java, Python & others that are available on github, gitlab & others. So for those users, you might still be finding LLMs lacking. It's also understandable that you might get more errors when generating C++ than say C#, since it's harder to use. It reminds me that I recently changed companies in part because at my previous job, I didn't have as many opportunities to work with LLMs (doing the things they are good at) as my current one.
- supriyo-biswas 1y ago
  > JS, C#, Java, Python Not a comment on your primary points, but most LLMs out there also suck at enterprise Java projects (Spring boot, etc.) unless it's the most mundane things such as writing simple controllers.
- deleted 1y ago
  [deleted]
- deleted 1y ago
  [deleted]
groby_b 1y ago
> add some missing destructors to a working but leaky C++ project. That's amazing, given C++ provides default destructors. They don't "miss" unless you or the AI omitted them deliberately. So I'll guess you meant custom destructors that clean up manual resource management. Which likely means you ran into the classic failure mode of AI, you underspecified constraints. It's C++. Humans or AI, as soon as there's more than one, be prepared to discuss ownership semantics in absolutely nauseating detail. You don't fix broken ownership semantics by telling a junior "try to get it to compile" and walking away for a couple of hours, either. And LLMs are interns, juniors at best. That means for a solo C++ project, yeah, you're probably better off not using them. The core issue being that they're incapable of actually learning, not the lack of skills per se - if you're willing to spec the hell out of everything every single time, they still do OK. It's soul-deadening, but it works. > If LLMs could just lack confidence in themselves for even a moment and state that they don't know what the problem is but it's willing to throw spaghetti at the wall to find out, I could respect that more than it marching through with absolute confidence that its garbage code did anything but break the project. As I said, juniors. This is a pretty common failure mode for fresh grads, too.
- solarwindy 1y ago
  Juniors do learn though, by contrast (as you said yourself). So these models are, at best, perpetual juniors. Except, they make bizarre mistakes that human juniors would not, and which are hard for even experienced developers to spot, because while it becomes possible with experience to preempt the kinds of mistakes that stem from the flawed or incomplete mental models of human juniors, these models do not themselves have a model of the computation underlying the code they produce—which in my experience makes the whole process of working with them endlessly frustrating.
  - groby_b 1y ago
    > Except, they make bizarre mistakes that human juniors would not, You underestimate juniors - Stack Overflow has done marvels to expand the capacity for bizarre mistakes :) > these models do not themselves have a model of the computation underlying the code they produce—which in my experience makes the whole process of working with them endlessly frustrating. I think that's... up for debate. They are able to do work that requires at least a very convincing facsimile of understanding. (My favorite benchmark is a couple of 100-500 line pieces of hand-obfuscated C++ with eldritch template horrors, and "what does this do" elicits a perfectly good explanation - and the code is definitely not in the corpus) What they don't have is an ability to infer constraints you didn't explicitly or implicitly spell out. And only limited capability to ask clarifying question. Therein lies the rub - each interaction requires amounts of detail that you only have at the very beginning of a work relationship with a junior. You _can_ spell it out, and they do much better. It's just extremely tedious. They also are horrible at correcting mistakes through ongoing conversation. If your LLM goes off into error land, just give up, reformulate the entire problem, and try again. In essence, you write a PRD and DD, plus a task breakdown, every single time. If you work on something that justifies this overhead - great time savers. If you work on something that can be done sloppily - great time savers. But a lot of code is in between those two lane markers.

bcrosby95 1y ago

I tried using them for a game server, written in C, as a test case and I came to the same conclusions. It did pretty well at making something that was buggy but worked, but increasingly continually fell apart around the 3k lines mark, which really isn't much code. It's not like these are esoteric languages, so I'm not sure why they (in my experience) have such a big problem with them. My current guess is that C and C++ aren't so mono domain. The languages its good at seems to be languages used predominantly in the web world, and most of the open code out there is for web. I'm curious if anyone has tried to use it for C# in Unity or a bespoke game in Rust and how well it does there.

starkparker 1y ago
I'm OK with an LLM not knowing how to do something. The part of working with it that I can't get past is its full-throated confidence in its actions. It'll break down, step by step, why its actions would solve the problem, then attempts to build it and can't recognize how or why its changes made the problem worse.
crq-yml 1y ago
I did spend a little while evaluating what GPT could output for a fairly intensive gaming task - implement a particular novel physical dynamics model. (Yes, there are newer tools now: I haven't used them and don't want to pay to use them.) And what it did was impressive in a certain sense in that it wrote up a large body of code with believable elements, but closer examination demonstrated that, of course, it wasn't actually going to function as-is, it made up some functions, it left others stubbed. I would never trust it to manage memory or other system resources correctly, because AFAIK nobody has implemented that kind of "simulate the program lifecycle" bookkeeping into the generated output. There's a domain that I do find some application for it in, which is "configure ffmpeg". That and tasks within game engines that are similar in nature, where the problem to be solved involves API boilerplate, do benefit. What also works, to some degree, is to ask it to generate code that follows a certain routine spec, like a state machine. Again, very boilerplate-driven stuff, and I expect an 80% solution and am pleasantly surprised if it gets everything. And it is very willing to let me "define a DSL" on the fly and then act as a stochastic compiler, generating output patterns that are largely in expectation with the input. That is something I know I could explore further. And I'm OK with this degree of utility. I think I can refine my use of it, and the tools themselves can also be refined. But there's a ceiling on it as a directly impactful thing, where the full story will play out over time as a communication tech, not a "generate code" tech, with the potential to better address Conway's law development issues.

deburo 1y ago

It seems quite obvious as surely the amount of C++, C, etc. is less, in the public domain, than the millions of lines of JS, C#, Java, Python & others that are available on github, gitlab & others. So for those users, you might still be finding LLMs lacking. It's also understandable that you might get more errors when generating C++ than say C#, since it's harder to use. It reminds me that I recently changed companies in part because at my previous job, I didn't have as many opportunities to work with LLMs (doing the things they are good at) as my current one.

supriyo-biswas 1y ago
> JS, C#, Java, Python Not a comment on your primary points, but most LLMs out there also suck at enterprise Java projects (Spring boot, etc.) unless it's the most mundane things such as writing simple controllers.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]

groby_b 1y ago

> add some missing destructors to a working but leaky C++ project. That's amazing, given C++ provides default destructors. They don't "miss" unless you or the AI omitted them deliberately. So I'll guess you meant custom destructors that clean up manual resource management. Which likely means you ran into the classic failure mode of AI, you underspecified constraints. It's C++. Humans or AI, as soon as there's more than one, be prepared to discuss ownership semantics in absolutely nauseating detail. You don't fix broken ownership semantics by telling a junior "try to get it to compile" and walking away for a couple of hours, either. And LLMs are interns, juniors at best. That means for a solo C++ project, yeah, you're probably better off not using them. The core issue being that they're incapable of actually learning, not the lack of skills per se - if you're willing to spec the hell out of everything every single time, they still do OK. It's soul-deadening, but it works. > If LLMs could just lack confidence in themselves for even a moment and state that they don't know what the problem is but it's willing to throw spaghetti at the wall to find out, I could respect that more than it marching through with absolute confidence that its garbage code did anything but break the project. As I said, juniors. This is a pretty common failure mode for fresh grads, too.

solarwindy 1y ago
Juniors do learn though, by contrast (as you said yourself). So these models are, at best, perpetual juniors. Except, they make bizarre mistakes that human juniors would not, and which are hard for even experienced developers to spot, because while it becomes possible with experience to preempt the kinds of mistakes that stem from the flawed or incomplete mental models of human juniors, these models do not themselves have a model of the computation underlying the code they produce—which in my experience makes the whole process of working with them endlessly frustrating.
- groby_b 1y ago
  > Except, they make bizarre mistakes that human juniors would not, You underestimate juniors - Stack Overflow has done marvels to expand the capacity for bizarre mistakes :) > these models do not themselves have a model of the computation underlying the code they produce—which in my experience makes the whole process of working with them endlessly frustrating. I think that's... up for debate. They are able to do work that requires at least a very convincing facsimile of understanding. (My favorite benchmark is a couple of 100-500 line pieces of hand-obfuscated C++ with eldritch template horrors, and "what does this do" elicits a perfectly good explanation - and the code is definitely not in the corpus) What they don't have is an ability to infer constraints you didn't explicitly or implicitly spell out. And only limited capability to ask clarifying question. Therein lies the rub - each interaction requires amounts of detail that you only have at the very beginning of a work relationship with a junior. You _can_ spell it out, and they do much better. It's just extremely tedious. They also are horrible at correcting mistakes through ongoing conversation. If your LLM goes off into error land, just give up, reformulate the entire problem, and try again. In essence, you write a PRD and DD, plus a task breakdown, every single time. If you work on something that justifies this overhead - great time savers. If you work on something that can be done sloppily - great time savers. But a lot of code is in between those two lane markers.