4 ms·

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is no

> There is no world where you input a document lacking clarity and detail and get a coding agent to reliably fill in that missing clarity and detail That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. Furthermore, LLMs are the ultimate detail fillers, because they are language interpolation/extrapolation machines. And their popularity is precisely because they are usually very good at filling in details: LLMs use their vast knowledge to guess what detail to generate, so the result usually makes sense. This doesn't detract much from the main point of the article though. Sometimes the interpolated detail is wrong (and indeterministic), so, if reliable result is to be achieved, important details have to be constrained, and for that they have to be specified. And whereas we have decades of tools and culture for coding, we largely don't have that for extremely detailed specs (except maybe at NASA or similar places). We could figure it out in the future, but we haven't yet.

Someone 7mo ago
> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex descriptions. For example, try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. > Sometimes the interpolated detail is wrong (and indeterministic), so, if reliable result is to be achieved Seems you agree they _cannot_ reliably generate (relatively small amounts of) working code from relatively terse descriptions
- mike_hearn 7mo ago
  Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
  - dxdm 7mo ago
    Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...". In fact, this is a huge chunk of the value a developer brings to the table.
    - gusmd 7mo ago
      And this is usually one of the defining traits of a senior engineer. They understand the tech and its limitations, and thus are able to look around corners, ask good questions, and, overall, provide quality product input.
      - danielam 7mo ago
        In other words, prudential judgement. Programs are a socially constructed artifact that help communicate and express a model (which is perpetually locked in people's heads with variance across engineers; divergence is addressed as the program develops). Determining what should or should not be done is a matter of not just domain knowledge, but practical reason, which is to say prudence, which is a virtue that can only be acquired by experience. It is an ability to apply universal principles to particular situations. This is why young devs, even when clever in some local sense, are worse at understanding the right moves to make in context. Code does not stand alone. It exists entirely in the service of something and is bound by constraints that are external to it.
    - bunderbunder 7mo ago
      This is very much my experience from working with outsourced development. Almost by design, they tend to lack domain expertise or an intimate understanding of the cultures and engineering values of the company they're contracted out to. This means that they will very quickly help you discover all the little details that seemed so obvious to you that you didn't even think to mention them, but were nonetheless critical to a successful implementation. The corollary to that is, the potential ROI of outsourcing is inversely proportional to how many of these little details your project has, and how important they are. So far I've found LLM coding to be kind of the same. For projects where those details are relatively unimportant, they can save me a bunch of effort. But I would not want to let an LLM build and maintain something like an API or database schema. Doing a good job of those requires too much knowledge of expected usage patterns working through design tradeoffs. And they tend to be incredibly expensive to change after deployment so it pays to take your time and get your hands dirty. I also kind of hate them for writing tests, for similar reasons. I know many people love them for it because writing tests isn't super happy fun times, but for my part I'm tired of dealing with LLM-generated test suites being so brittle that they actively hinder future development.
- mathgradthrow 7mo ago
  > try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. this works well for me
- jes5199 7mo ago
  anything can be reliable if you have good tests
lmm 7mo ago
> LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. They can generate boilerplate, sure. Or they can expand out a known/named algorithm implementation, like pulling in a library. But neither of those is generating detail that wasn't there in the original (at most it pulls in the detail from somewhere in the training set).
- tibbe 7mo ago
  They do more than that. If you ask for ui with a button that button won't be upside down even if you didn't specify its orientation. Lots of the detail can be inferred from general human preferences, which are present in the LLMs' training data. This extends way beyond CS stuff like details of algorithm implementations.
  - skywhopper 7mo ago
    That’s exactly what they said. Details “elsewhere in its training set”.
  - zabzonk 7mo ago
    Isn't "not being upsidedown" just one of the default properties of a button in whatever GUI toolkit you are using? I'd be worried if an LLM _did_ start setting all the possible button properties.
    - MoreQARespect 7mo ago
      Putting LLMs on a pedestal is very much in vogue these days.
    - bluGill 7mo ago
      If you ask for increase and decrease buttons they will put the right icons on it (not words) and lay them out right.
      - zelphirkalt 7mo ago
skywhopper 7mo ago
“LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions” Only with well-known patterns that represent shared knowledge specified elsewhere. If the details they “fill in” each time differ in ways that change behavior, then the spec is deficient. If we “figure out” how to write such detailed specs in the future, as you suggest, then that becomes the “code”.
- mexicocitinluez 7mo ago
  Also, they're a bit more willing to make assumptions. After awhile, I think we all get a sense of not only the amount of micro-decisions you have to make will building stuff (even when you're intimate with the domain), but also the amount of assumptions you'll need to make about things you either don't know yet or haven't fully fleshed out. I'm painfully aware of the assumptions I'm making nowadays and that definitely changes the way I build things. And while I love these tools, their ability to not only make assumptions, but over-engineer those assumptions can have disastrous effects. I had Claude build me a zip code heat map given a data source and it did it spectacularly. Same with a route planner. But asking it build out medical procedure documentation configurations based off of a general plan DID NOT work as well as I had expected it would. Also, I asked Claude about what the cron expression I wrote would do, and it got it wrong (which is expected because Azure Web Jobs uses a non-standard form). But even after telling it that it was wrong, and giving it the documentation to rely on, it still doubled down on the wrong answer.
- ModernMech 7mo ago
  Right, when you tell it “draw me a renaissance woman” and it gives you a facsimile of the Mona Lisa, it’s not because it intelligently anticipated what you wanted — it’s just been trained thoroughly to make that association.
roysting 7mo ago
In get the sense that what you are responding to and even many comments to yours are expressing a kind of coping with the current dynamic, only exacerbated by the rather elitist and egoistic mentality that people in tech have had for a very long time now; i.e., they are falling…being pushed from Mt Olympus and there is A LOT of anxious rationalization going on. Not a mere 5 years ago even tech people were chortling down their upturned noses that people were complaining that their jobs were being “taken”, and now that the turns have tabled, there is a bunch of denial, anger, and grief going on, maybe even some depression as many of the recently unemployed realize the current state of things. It’s all easy to deride the inferiority of AI when you’re employed in a job doing things as you had been all your career, thinking you cannot be replaced… until you find yourself on the other side of the turn that has tabled.
- rdevilla 7mo ago
  I can't help but imagine that this is how some people felt about doctors once webmd came out. It's some nice rhetoric, but you're not actually saying much.
- otikik 7mo ago
  I use AI for my work every single day - and during some weekends too. Claude Code, with Opus. It is far from being able to reliably produce the code that we need for production. It produces code that looks ok most of the time, but I have seen it lose track of key details, misinterpret requisites and even ignore them sometimes - "on purpose", as in it writing something like "let's not do that requirement, it's not necessary". This kind of thing happens at least once per day to me, maybe more. I am not denying that it is useful, let me be clear. It is extremely convenient, especially for mechanical tasks. It has other advantages like quick exploration of other people's code, for example. If my employer didn't provide a corporate account for me, I would pay one from my own pocket. That said, I agree with OP and the author that it is not reliable when producing code from specs. It does things right, I would say often. That might be good enough for some fields/people. It's good enough for me, too. I however review every line it produces, because I've seen it miss, often, as well.
  - 7mo ago
deleted 7mo ago
[deleted]
hnthrow0287345 7mo ago
It means for most specs, you can just use an average solution, or a most-popular solution. I'm absolutely on board with that. We probably need less weird and outlier decisions in designs for something that is a boring ass business website.
agentultra 7mo ago
We do have such detailed specifications. But they are written in a language with a narrow interface. It’s a technique called, “program synthesis,” and you can find an example of such a language called, Synquid. It might be illuminating to see what a mathematically precise specification can and cannot do when it comes to generating programs. A major challenge in formal methods is proving that the program implements the specification faithfully, known as the specification gap. If you have a very high level and flexible specification language, such as TLA+, there is a lot of work to do to verify that the program you write meets the specification you wrote. For something like Synquid that is closer to the code there are more constraints on expressivity. The point is that spoken language is not sufficiently precise to define a program. Just because an LLM can fill in plausible details where sufficient detail is lacking doesn’t indicate that it’s solving the specification gap. If the program happens to implement the specification faithfully you got lucky. You still don’t actually know that’s true until you verify it. It’s different with a narrow interface though: you can be very precise and very abstract with a good mathematical system for expressing specifications. It’s a lot more work and requires more training to do than filling in a markdown file and trying to coax the algorithm into outputting what you want through prose and fiction.
- et1337 7mo ago
  I think the worst case is actually that the LLM faithfully implements your spec, but your spec was flawed. To the extent that you outsource the mechanical details to a machine trained to do exactly what you tell it, you destroy or at least hamper the feedback loop between fuzzy human thoughts and cold hard facts.
  - agentultra 7mo ago
    Unfortunately even formal specifications have this problem. Nothing can replace thinking. But sycophancy, I agree, is a problem. These tools are designed to be pleasing, to generate plausible output; but they cannot think critically about the tasks they're given. Nothing will save you from a bad specification. And there's no royal road to knowing how to write good ones.
abdulhaq 7mo ago
> Sometimes the interpolated detail is wrong you just (correctly) negated your own claim
cush 7mo ago
Exactly. Any developer working on any project will encounter a decision that wasn’t in the spec, where they use their judgement and taste to fill in gaps. The idea that only code can be a complete spec assumes the code perfectly matches the original intent - which we know it rarely does in a project of meaningful size.
observationist 7mo ago
ChatGPT 5.4 pro has surprised me several times; when asking a "can such and such be done" type question, intending to have a discussion about whether a thing can be done in principle, and what it might look like, it's actually produced a working example in response, in addition to answering the questions. Some of the missing pieces come from memory, knowing which topics I like to explore, some from the model itself, either baked in knowledge or what it picks up searching, but they can definitely take a vague, handwavy half baked idea and whip up a full app or game or whitepaper. Sometimes it's "exactly what I wanted!", other times it's "exactly the kind of thing I was talking about!" Semantics and context and nuance are part and parcel of LLM capabilities. Superhuman in some areas, definitely subhuman in others. AI is getting pretty competent and clever.
deleted 7mo ago
[deleted]
lowbloodsugar 7mo ago
Isn’t this no different than a compiler? I don’t specify which registers to use in C or Java, and yet the code runs! An awful lot of time has been put into the compiler to know which registers to use and how to juggle them. Is an LLM any different in its behavior (albeit different in how it was trained)? If not, then specs are just an even higher level programming language. I think the difference is that, with a C compiler, when it gets it wrong I’ll have some terrible performance impact, it when the LLM gets it wrong, it will do something nobody wanted, like delete someone’s account or debit one account without crediting another.
strix_varius 7mo ago
> the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions > Sometimes the interpolated detail is wrong (and indeterministic) ... You consider incorrect, non deterministic results to be "reliable"?
- tinodb 7mo ago
  Do you consider the implementation of such specs by another human to (always) be correct and deterministic? Heck, if I reimplement something I worked on a month ago it’s probably not going to be the exact same. Being non deterministic needn’t to be a problem, as long as it falls within certain boundaries and produces working results.

Someone 7mo ago

> That is not true, and the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. LLMs can generate (relatively small amounts of) working code from relatively terse descriptions, but I don’t think they can do so _reliably_. They’re more reliable the shorter the code fragment and the more common the code, but they do break down for complex descriptions. For example, try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. > Sometimes the interpolated detail is wrong (and indeterministic), so, if reliable result is to be achieved Seems you agree they _cannot_ reliably generate (relatively small amounts of) working code from relatively terse descriptions

mike_hearn 7mo ago
Neither can humans, but the industry has decades of experience with how to instruct and guide human developer teams using specs.
- dxdm 7mo ago
  Usually, you don't want your developers to be coding monkeys, for good results. You need the human developer in the loop to even define the spec, maybe contributing ideas, but at the very least asking questions about "what happens when..." and "have you thought about...". In fact, this is a huge chunk of the value a developer brings to the table.
  - gusmd 7mo ago
    And this is usually one of the defining traits of a senior engineer. They understand the tech and its limitations, and thus are able to look around corners, ask good questions, and, overall, provide quality product input.
    - danielam 7mo ago
      In other words, prudential judgement. Programs are a socially constructed artifact that help communicate and express a model (which is perpetually locked in people's heads with variance across engineers; divergence is addressed as the program develops). Determining what should or should not be done is a matter of not just domain knowledge, but practical reason, which is to say prudence, which is a virtue that can only be acquired by experience. It is an ability to apply universal principles to particular situations. This is why young devs, even when clever in some local sense, are worse at understanding the right moves to make in context. Code does not stand alone. It exists entirely in the service of something and is bound by constraints that are external to it.
  - bunderbunder 7mo ago
    This is very much my experience from working with outsourced development. Almost by design, they tend to lack domain expertise or an intimate understanding of the cultures and engineering values of the company they're contracted out to. This means that they will very quickly help you discover all the little details that seemed so obvious to you that you didn't even think to mention them, but were nonetheless critical to a successful implementation. The corollary to that is, the potential ROI of outsourcing is inversely proportional to how many of these little details your project has, and how important they are. So far I've found LLM coding to be kind of the same. For projects where those details are relatively unimportant, they can save me a bunch of effort. But I would not want to let an LLM build and maintain something like an API or database schema. Doing a good job of those requires too much knowledge of expected usage patterns working through design tradeoffs. And they tend to be incredibly expensive to change after deployment so it pays to take your time and get your hands dirty. I also kind of hate them for writing tests, for similar reasons. I know many people love them for it because writing tests isn't super happy fun times, but for my part I'm tired of dealing with LLM-generated test suites being so brittle that they actively hinder future development.
mathgradthrow 7mo ago
> try tweaking the description of a widely-known algorithm just a little bit and see how good the generated code follows the spec. this works well for me
jes5199 7mo ago
anything can be reliable if you have good tests

lmm 7mo ago

> LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions. Code is the detail being filled in. They can generate boilerplate, sure. Or they can expand out a known/named algorithm implementation, like pulling in a library. But neither of those is generating detail that wasn't there in the original (at most it pulls in the detail from somewhere in the training set).

tibbe 7mo ago
They do more than that. If you ask for ui with a button that button won't be upside down even if you didn't specify its orientation. Lots of the detail can be inferred from general human preferences, which are present in the LLMs' training data. This extends way beyond CS stuff like details of algorithm implementations.
- skywhopper 7mo ago
  That’s exactly what they said. Details “elsewhere in its training set”.
- zabzonk 7mo ago
  Isn't "not being upsidedown" just one of the default properties of a button in whatever GUI toolkit you are using? I'd be worried if an LLM _did_ start setting all the possible button properties.
  - MoreQARespect 7mo ago
    Putting LLMs on a pedestal is very much in vogue these days.
  - bluGill 7mo ago
    If you ask for increase and decrease buttons they will put the right icons on it (not words) and lay them out right.
    - zelphirkalt 7mo ago

skywhopper 7mo ago

“LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions” Only with well-known patterns that represent shared knowledge specified elsewhere. If the details they “fill in” each time differ in ways that change behavior, then the spec is deficient. If we “figure out” how to write such detailed specs in the future, as you suggest, then that becomes the “code”.

mexicocitinluez 7mo ago
Also, they're a bit more willing to make assumptions. After awhile, I think we all get a sense of not only the amount of micro-decisions you have to make will building stuff (even when you're intimate with the domain), but also the amount of assumptions you'll need to make about things you either don't know yet or haven't fully fleshed out. I'm painfully aware of the assumptions I'm making nowadays and that definitely changes the way I build things. And while I love these tools, their ability to not only make assumptions, but over-engineer those assumptions can have disastrous effects. I had Claude build me a zip code heat map given a data source and it did it spectacularly. Same with a route planner. But asking it build out medical procedure documentation configurations based off of a general plan DID NOT work as well as I had expected it would. Also, I asked Claude about what the cron expression I wrote would do, and it got it wrong (which is expected because Azure Web Jobs uses a non-standard form). But even after telling it that it was wrong, and giving it the documentation to rely on, it still doubled down on the wrong answer.
ModernMech 7mo ago
Right, when you tell it “draw me a renaissance woman” and it gives you a facsimile of the Mona Lisa, it’s not because it intelligently anticipated what you wanted — it’s just been trained thoroughly to make that association.

roysting 7mo ago

In get the sense that what you are responding to and even many comments to yours are expressing a kind of coping with the current dynamic, only exacerbated by the rather elitist and egoistic mentality that people in tech have had for a very long time now; i.e., they are falling…being pushed from Mt Olympus and there is A LOT of anxious rationalization going on. Not a mere 5 years ago even tech people were chortling down their upturned noses that people were complaining that their jobs were being “taken”, and now that the turns have tabled, there is a bunch of denial, anger, and grief going on, maybe even some depression as many of the recently unemployed realize the current state of things. It’s all easy to deride the inferiority of AI when you’re employed in a job doing things as you had been all your career, thinking you cannot be replaced… until you find yourself on the other side of the turn that has tabled.

rdevilla 7mo ago
I can't help but imagine that this is how some people felt about doctors once webmd came out. It's some nice rhetoric, but you're not actually saying much.
otikik 7mo ago
I use AI for my work every single day - and during some weekends too. Claude Code, with Opus. It is far from being able to reliably produce the code that we need for production. It produces code that looks ok most of the time, but I have seen it lose track of key details, misinterpret requisites and even ignore them sometimes - "on purpose", as in it writing something like "let's not do that requirement, it's not necessary". This kind of thing happens at least once per day to me, maybe more. I am not denying that it is useful, let me be clear. It is extremely convenient, especially for mechanical tasks. It has other advantages like quick exploration of other people's code, for example. If my employer didn't provide a corporate account for me, I would pay one from my own pocket. That said, I agree with OP and the author that it is not reliable when producing code from specs. It does things right, I would say often. That might be good enough for some fields/people. It's good enough for me, too. I however review every line it produces, because I've seen it miss, often, as well.
- 7mo ago

deleted 7mo ago

[deleted]

hnthrow0287345 7mo ago

It means for most specs, you can just use an average solution, or a most-popular solution. I'm absolutely on board with that. We probably need less weird and outlier decisions in designs for something that is a boring ass business website.

agentultra 7mo ago

We do have such detailed specifications. But they are written in a language with a narrow interface. It’s a technique called, “program synthesis,” and you can find an example of such a language called, Synquid. It might be illuminating to see what a mathematically precise specification can and cannot do when it comes to generating programs. A major challenge in formal methods is proving that the program implements the specification faithfully, known as the specification gap. If you have a very high level and flexible specification language, such as TLA+, there is a lot of work to do to verify that the program you write meets the specification you wrote. For something like Synquid that is closer to the code there are more constraints on expressivity. The point is that spoken language is not sufficiently precise to define a program. Just because an LLM can fill in plausible details where sufficient detail is lacking doesn’t indicate that it’s solving the specification gap. If the program happens to implement the specification faithfully you got lucky. You still don’t actually know that’s true until you verify it. It’s different with a narrow interface though: you can be very precise and very abstract with a good mathematical system for expressing specifications. It’s a lot more work and requires more training to do than filling in a markdown file and trying to coax the algorithm into outputting what you want through prose and fiction.

et1337 7mo ago
I think the worst case is actually that the LLM faithfully implements your spec, but your spec was flawed. To the extent that you outsource the mechanical details to a machine trained to do exactly what you tell it, you destroy or at least hamper the feedback loop between fuzzy human thoughts and cold hard facts.
- agentultra 7mo ago
  Unfortunately even formal specifications have this problem. Nothing can replace thinking. But sycophancy, I agree, is a problem. These tools are designed to be pleasing, to generate plausible output; but they cannot think critically about the tasks they're given. Nothing will save you from a bad specification. And there's no royal road to knowing how to write good ones.

abdulhaq 7mo ago

> Sometimes the interpolated detail is wrong you just (correctly) negated your own claim

cush 7mo ago

Exactly. Any developer working on any project will encounter a decision that wasn’t in the spec, where they use their judgement and taste to fill in gaps. The idea that only code can be a complete spec assumes the code perfectly matches the original intent - which we know it rarely does in a project of meaningful size.

observationist 7mo ago

ChatGPT 5.4 pro has surprised me several times; when asking a "can such and such be done" type question, intending to have a discussion about whether a thing can be done in principle, and what it might look like, it's actually produced a working example in response, in addition to answering the questions. Some of the missing pieces come from memory, knowing which topics I like to explore, some from the model itself, either baked in knowledge or what it picks up searching, but they can definitely take a vague, handwavy half baked idea and whip up a full app or game or whitepaper. Sometimes it's "exactly what I wanted!", other times it's "exactly the kind of thing I was talking about!" Semantics and context and nuance are part and parcel of LLM capabilities. Superhuman in some areas, definitely subhuman in others. AI is getting pretty competent and clever.

deleted 7mo ago

[deleted]

lowbloodsugar 7mo ago

Isn’t this no different than a compiler? I don’t specify which registers to use in C or Java, and yet the code runs! An awful lot of time has been put into the compiler to know which registers to use and how to juggle them. Is an LLM any different in its behavior (albeit different in how it was trained)? If not, then specs are just an even higher level programming language. I think the difference is that, with a C compiler, when it gets it wrong I’ll have some terrible performance impact, it when the LLM gets it wrong, it will do something nobody wanted, like delete someone’s account or debit one account without crediting another.

strix_varius 7mo ago

> the proof is that LLMs _can_ reliably generate (relatively small amounts of) working code from relatively terse descriptions > Sometimes the interpolated detail is wrong (and indeterministic) ... You consider incorrect, non deterministic results to be "reliable"?

tinodb 7mo ago
Do you consider the implementation of such specs by another human to (always) be correct and deterministic? Heck, if I reimplement something I worked on a month ago it’s probably not going to be the exact same. Being non deterministic needn’t to be a problem, as long as it falls within certain boundaries and produces working results.