3 ms·

> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can

by Someone 7mo ago

> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex descriptions. For example, try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. > Sometimes the interpolated detail is wrong (and indeterministic), so, if reliable result is to be achieved Seems you agree they _cannot_ reliably generate (relatively small amounts of) working code from relatively terse descriptions

mike_hearn 7mo ago
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
- dxdm 7mo ago
  Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...". In fact, this is a huge chunk of the value a developer brings to the table.
  - gusmd 7mo ago
    And this is usually one of the defining traits of a senior engineer. They understand the tech and its limitations, and thus are able to look around corners, ask good questions, and, overall, provide quality product input.
    - danielam 7mo ago
      In other words, prudential judgement. Programs are a socially constructed artifact that help communicate and express a model (which is perpetually locked in people's heads with variance across engineers; divergence is addressed as the program develops). Determining what should or should not be done is a matter of not just domain knowledge, but practical reason, which is to say prudence, which is a virtue that can only be acquired by experience. It is an ability to apply universal principles to particular situations. This is why young devs, even when clever in some local sense, are worse at understanding the right moves to make in context. Code does not stand alone. It exists entirely in the service of something and is bound by constraints that are external to it.
  - bunderbunder 7mo ago
    This is very much my experience from working with outsourced development. Almost by design, they tend to lack domain expertise or an intimate understanding of the cultures and engineering values of the company they're contracted out to. This means that they will very quickly help you discover all the little details that seemed so obvious to you that you didn't even think to mention them, but were nonetheless critical to a successful implementation. The corollary to that is, the potential ROI of outsourcing is inversely proportional to how many of these little details your project has, and how important they are. So far I've found LLM coding to be kind of the same. For projects where those details are relatively unimportant, they can save me a bunch of effort. But I would not want to let an LLM build and maintain something like an API or database schema. Doing a good job of those requires too much knowledge of expected usage patterns working through design tradeoffs. And they tend to be incredibly expensive to change after deployment so it pays to take your time and get your hands dirty. I also kind of hate them for writing tests, for similar reasons. I know many people love them for it because writing tests isn't super happy fun times, but for my part I'm tired of dealing with LLM-generated test suites being so brittle that they actively hinder future development.
- MoreQARespect 7mo ago
  Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
  - pablobaz 7mo ago
    which bits of this do you think llm based agents can't do?
    - ModernMech 7mo ago
      The main thing they cannot do is be held accountable for any decisions, which makes them not trustworthy.
      - vbezhenar 7mo ago
        This is not correct. They can say "sorry" which makes them as accountable as ordinary developer.
        interstice 7mo ago
        I've found recent versions of Claude and codex to be reluctant in this regard. They will recognise the problem they created a few minutes ago but often behave as if someone else did it. In many ways that's true though, I suppose.
        bee_rider 7mo ago
        Does it do this for really cut and dry problems? I’ve noticed that ChatGPT will put a lot of effort into (retroactively) “discovering” a basically-valid alternative interpretation of something it said previously, if you object on good grounds. Like it’s trying to evade admitting that it made a mistake, but also find some say to satisfy your objection. Fair enough, if slightly annoying. But I have also caught it on straightforward matters of fact and it’ll apologize. Sometimes in an over the top fashion…
- ModernMech 7mo ago
  Specs are insufficient to guide human developer teams, so I don’t understand the comparison.
- adrian_b 7mo ago
  When LLMs generate an appropriate program from ambiguous requirements, they do this because the requirements happen to match something similar that has been done previously elsewhere. There is a huge amount of programming work that consists in reinventing the wheel, i.e. in redoing something very similar to programs that have been written thousands of times before. For this kind of work LLMs can greatly improve productivity, even if they are not much better than if you would be allowed to search, copy and paste from the programs on which the LLM has been trained. The advantage of an LLM is the automation of the search/copy/paste actions, and even more than this, the removal of the copyrights from the original programs. The copyright laws are what has resulted in huge amounts of superfluous programming work, which is necessary even when there are open-source solutions, but the employer of the programmer wants to "own the IP". On the other hand, for really novel applications, or for old applications where you want to obtain better performance than anyone has gotten before, providing an ambiguous prompt to an LLM will get you nowhere.
  - bluefirebrand 7mo ago
    > and even more than this, the removal of the copyrights from the original programs This seems really strange to me. Can you explain how this is different than just stealing code from other sources, or copying it wholly from open source repos?
- FuckButtons 7mo ago
  You can guide humans, but ultimately the reason senior software developers have been payed large sums of money is that even with specs mostly we have found it works better to have someone with good judgement actually doing the work, otherwise we would have just been using specifications. The question remains open if llm’s can show good judgement, often my experience with claude is that it doesn’t if the problem domain is non-trivial but it’s possible that won’t always be true.
mathgradthrow 7mo ago
> try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. this works well for me
jes5199 7mo ago
anything can be reliable if you have good tests

by Someone 7mo ago

mike_hearn 7mo ago

Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.

dxdm 7mo ago
Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...". In fact, this is a huge chunk of the value a developer brings to the table.
- gusmd 7mo ago
  And this is usually one of the defining traits of a senior engineer. They understand the tech and its limitations, and thus are able to look around corners, ask good questions, and, overall, provide quality product input.
  - danielam 7mo ago
    In other words, prudential judgement. Programs are a socially constructed artifact that help communicate and express a model (which is perpetually locked in people's heads with variance across engineers; divergence is addressed as the program develops). Determining what should or should not be done is a matter of not just domain knowledge, but practical reason, which is to say prudence, which is a virtue that can only be acquired by experience. It is an ability to apply universal principles to particular situations. This is why young devs, even when clever in some local sense, are worse at understanding the right moves to make in context. Code does not stand alone. It exists entirely in the service of something and is bound by constraints that are external to it.
- bunderbunder 7mo ago
  This is very much my experience from working with outsourced development. Almost by design, they tend to lack domain expertise or an intimate understanding of the cultures and engineering values of the company they're contracted out to. This means that they will very quickly help you discover all the little details that seemed so obvious to you that you didn't even think to mention them, but were nonetheless critical to a successful implementation. The corollary to that is, the potential ROI of outsourcing is inversely proportional to how many of these little details your project has, and how important they are. So far I've found LLM coding to be kind of the same. For projects where those details are relatively unimportant, they can save me a bunch of effort. But I would not want to let an LLM build and maintain something like an API or database schema. Doing a good job of those requires too much knowledge of expected usage patterns working through design tradeoffs. And they tend to be incredibly expensive to change after deployment so it pays to take your time and get your hands dirty. I also kind of hate them for writing tests, for similar reasons. I know many people love them for it because writing tests isn't super happy fun times, but for my part I'm tired of dealing with LLM-generated test suites being so brittle that they actively hinder future development.
MoreQARespect 7mo ago
Humans have the ability to retrospect, push back on a faulty spec, push back on an unclarified spec, do experiments, make judgement calls and build tools and processes to account for their own foibles.
- pablobaz 7mo ago
  which bits of this do you think llm based agents can't do?
  - ModernMech 7mo ago
    The main thing they cannot do is be held accountable for any decisions, which makes them not trustworthy.
    - vbezhenar 7mo ago
      This is not correct. They can say "sorry" which makes them as accountable as ordinary developer.
      - interstice 7mo ago
        I've found recent versions of Claude and codex to be reluctant in this regard. They will recognise the problem they created a few minutes ago but often behave as if someone else did it. In many ways that's true though, I suppose.
        bee_rider 7mo ago
        Does it do this for really cut and dry problems? I’ve noticed that ChatGPT will put a lot of effort into (retroactively) “discovering” a basically-valid alternative interpretation of something it said previously, if you object on good grounds. Like it’s trying to evade admitting that it made a mistake, but also find some say to satisfy your objection. Fair enough, if slightly annoying. But I have also caught it on straightforward matters of fact and it’ll apologize. Sometimes in an over the top fashion…
ModernMech 7mo ago
Specs are insufficient to guide human developer teams, so I don’t understand the comparison.
adrian_b 7mo ago
When LLMs generate an appropriate program from ambiguous requirements, they do this because the requirements happen to match something similar that has been done previously elsewhere. There is a huge amount of programming work that consists in reinventing the wheel, i.e. in redoing something very similar to programs that have been written thousands of times before. For this kind of work LLMs can greatly improve productivity, even if they are not much better than if you would be allowed to search, copy and paste from the programs on which the LLM has been trained. The advantage of an LLM is the automation of the search/copy/paste actions, and even more than this, the removal of the copyrights from the original programs. The copyright laws are what has resulted in huge amounts of superfluous programming work, which is necessary even when there are open-source solutions, but the employer of the programmer wants to "own the IP". On the other hand, for really novel applications, or for old applications where you want to obtain better performance than anyone has gotten before, providing an ambiguous prompt to an LLM will get you nowhere.
- bluefirebrand 7mo ago
  > and even more than this, the removal of the copyrights from the original programs This seems really strange to me. Can you explain how this is different than just stealing code from other sources, or copying it wholly from open source repos?
FuckButtons 7mo ago
You can guide humans, but ultimately the reason senior software developers have been payed large sums of money is that even with specs mostly we have found it works better to have someone with good judgement actually doing the work, otherwise we would have just been using specifications. The question remains open if llm’s can show good judgement, often my experience with claude is that it doesn’t if the problem domain is non-trivial but it’s possible that won’t always be true.

mathgradthrow 7mo ago

> try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. this works well for me

jes5199 7mo ago

anything can be reliable if you have good tests