4 ms·

Yes, say "Build a compiler" will require you to clean up stuff if you leave the agent for days, but not because of the LLM or the quality of the tool, but becau

Yes, say "Build a compiler" will require you to clean up stuff if you leave the agent for days, but not because of the LLM or the quality of the tool, but because you hardly specified anything, so of course it's gonna make assumptions you need to correct. If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says. Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.

n4r9 2mo ago
You're making assumptions; OP made no mention of how detailed their spec was.
- embedding-shape 2mo ago
  Indeed, you're right, I did do assumptions, based on what I could understand from it, and also based on my own experience using these tools too. May be their specification is very detailed and exactly as they wish it to be, yet the agent creates something completely else, then probably something else than the specification is wrong, and my comment is pointless :)
mccoyb 2mo ago
Bro, what the fuck do you think I’m doing? Do you think I don’t know about spec driven development? What is the most complicated thing you’ve built with LM agents? Have you done it with a single spec? How novel was it? This comment is so laughably “you’re holding it wrong” I can’t respond to you seriously. > If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things The set of software that has followed this process is measure zero.
- embedding-shape 2mo ago
  > Bro, what the fuck do you think I’m doing? Do you think I don’t know about spec driven development? Bro, I don't know you, and have no idea what you do, except for the parts you share in your comment, like talking to a LLM :) If you say "if you ask an agent to write a compiler" and don't expand on that, I'll take your word for it. I'm glad you got a laugh, and didn't take it so serious :) Not entirely sure why you got so defensive, but it's all good, again I don't know you, sorry if it felt like I called you stupid or something. At least some other people found it helpful I suppose, so in the end it sucks you're unable to get value from a process I've found working for me, hope you'll find something that works out for you in the end, regardless of what it is!
  - mccoyb 2mo ago
    I'm still interested in your claim: > If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says. Are you claiming that you reliably get good software out of agents using this process? My experiential claim is that I've followed this process for complex pieces of software (and, often, taking significantly longer than a day -- more like several weeks, as if I was writing an academic paper) and they still get it wrong ... even with an "academic paper level spec" Models: only the best, Fable 5 max, Sol 5.6 max, yada yada -- I'm not convinced this process works for "serious endeavors" (for instance: let's build a novel accelerator compiler or let's build a new incremental linker) where "serious" here means "something where I'll likely be spending multiple weeks working on it"
    - embedding-shape 2mo ago
      > Are you claiming that you reliably get good software out of agents using this process? Yeah, for sure, I tend to be really strict about the overall design, keeping things minimal and only the abstractions needed, nothing more/less. I don't know what "most complicated software" even is, I've only been writing software for 20 years or so, but mostly around startups, smaller companies and as a consultant/freelancer for companies of all sizes, so I won't even claim to know what "really complicated software" is. What I've done so far ranges from implementations of various ML models, native cross-platform applications, various types of games, simulation software and a bunch of tooling to help me with music production and VFX/animation/3D work. Some of those I guess I wouldn't even have approached without LLMs, and probably would have taken me years if I did attempt, but following that process I basically end up with one spec, maybe 2K-3K lines in total, including step-by-step how things have to be implemented, then fire off Codex + GPT-5.3-Sol max with "/goal" saying exactly how things should be implemented, verified and so on, and update the specification file as it goes along, and not stop until everything from the specification been confirmed to be implemented correctly. Longest been 4-5 days until the goal is completed. Always a lot of iterations afterwards of course, as I realize some things I put in the specification ultimately didn't made sense once it's implemented, so revise specification, figure out how to change, fire off "/goal", and repeat this until I have code that is better or at least equal to how I'd write it myself. It was a long time ago this process lead to something completely different being implemented, I usually get exactly what the specification says.

n4r9 2mo ago

You're making assumptions; OP made no mention of how detailed their spec was.

embedding-shape 2mo ago
Indeed, you're right, I did do assumptions, based on what I could understand from it, and also based on my own experience using these tools too. May be their specification is very detailed and exactly as they wish it to be, yet the agent creates something completely else, then probably something else than the specification is wrong, and my comment is pointless :)

mccoyb 2mo ago

Bro, what the fuck do you think I’m doing? Do you think I don’t know about spec driven development? What is the most complicated thing you’ve built with LM agents? Have you done it with a single spec? How novel was it? This comment is so laughably “you’re holding it wrong” I can’t respond to you seriously. > If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things The set of software that has followed this process is measure zero.

embedding-shape 2mo ago
> Bro, what the fuck do you think I’m doing? Do you think I don’t know about spec driven development? Bro, I don't know you, and have no idea what you do, except for the parts you share in your comment, like talking to a LLM :) If you say "if you ask an agent to write a compiler" and don't expand on that, I'll take your word for it. I'm glad you got a laugh, and didn't take it so serious :) Not entirely sure why you got so defensive, but it's all good, again I don't know you, sorry if it felt like I called you stupid or something. At least some other people found it helpful I suppose, so in the end it sucks you're unable to get value from a process I've found working for me, hope you'll find something that works out for you in the end, regardless of what it is!
- mccoyb 2mo ago
  I'm still interested in your claim: > If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says. Are you claiming that you reliably get good software out of agents using this process? My experiential claim is that I've followed this process for complex pieces of software (and, often, taking significantly longer than a day -- more like several weeks, as if I was writing an academic paper) and they still get it wrong ... even with an "academic paper level spec" Models: only the best, Fable 5 max, Sol 5.6 max, yada yada -- I'm not convinced this process works for "serious endeavors" (for instance: let's build a novel accelerator compiler or let's build a new incremental linker) where "serious" here means "something where I'll likely be spending multiple weeks working on it"
  - embedding-shape 2mo ago
    > Are you claiming that you reliably get good software out of agents using this process? Yeah, for sure, I tend to be really strict about the overall design, keeping things minimal and only the abstractions needed, nothing more/less. I don't know what "most complicated software" even is, I've only been writing software for 20 years or so, but mostly around startups, smaller companies and as a consultant/freelancer for companies of all sizes, so I won't even claim to know what "really complicated software" is. What I've done so far ranges from implementations of various ML models, native cross-platform applications, various types of games, simulation software and a bunch of tooling to help me with music production and VFX/animation/3D work. Some of those I guess I wouldn't even have approached without LLMs, and probably would have taken me years if I did attempt, but following that process I basically end up with one spec, maybe 2K-3K lines in total, including step-by-step how things have to be implemented, then fire off Codex + GPT-5.3-Sol max with "/goal" saying exactly how things should be implemented, verified and so on, and update the specification file as it goes along, and not stop until everything from the specification been confirmed to be implemented correctly. Longest been 4-5 days until the goal is completed. Always a lot of iterations afterwards of course, as I realize some things I put in the specification ultimately didn't made sense once it's implemented, so revise specification, figure out how to change, fire off "/goal", and repeat this until I have code that is better or at least equal to how I'd write it myself. It was a long time ago this process lead to something completely different being implemented, I usually get exactly what the specification says.