4 ms·

This analogy only works if you’re ready to commit _only prompts_ and have those prompts do a clean “compile” of your production runtime everytime you pull. Are

This analogy only works if you’re ready to commit _only prompts_ and have those prompts do a clean “compile” of your production runtime everytime you pull. Are you ready to do that?

jtrn 1mo ago
See my response to the other comment if you don't understand that analogy is not the same as "identical". And to state that "This analogy only works if ...." is just PATENTLY wrong. The analogy works fine if you say that it compares analogous situations. Like if we focus on some encumbrance complaining that "kids these days are too stupid because they don't understand the fundamentals like I do" or "These new tools that make it easier for stupid people, not smart people like me, to make stuff is dangerous because they don't know what they are doing". That's just a few of MANY analogous observations we can make for the two situations. But I guess you think that only the thing you care about is the only thing that exists. And in the end, everybody who complains like this is just going to be shown to be just as mistaken as all the people who complained that "people who don't code in assembly are dangerous!" And it's just marvelous to watch it play out slowly over the last couple of years. And we are just a couple of years in. I'm just making a note of everybody who is mistaken, as a study in denial and biased thinking. The end for all of this was obvious after Opus 4.6 hit. And it's just getting more and more obvious with each model release and harness improvement. This is a gold mine for studying flawed thinking.
Kim_Bruning 1mo ago
It turns out that this almost works. You'd probably want to commit explicit design documents btw, not every last prompt. FWIW, when people try this out in practice, they tend to commit the design docs alongside the generated tests and the generated code. Mutating the docs then leads to mutations in the code. Several friends and I have workflows that look a bit like this, but I know for sure mine isn't 100% strict(yet?), and ymmv. Everyone is still figuring out the best practices I think.
- jtrn 1mo ago
  Thank you for the first sensible response that actually engages with the point I made. And your answer is better than mine, and less angry.... :P The question from glouwbug was just taken for granted to be "no, and therefore the analogy fails." And if I understand your answer, it's basically: "no, but we're visibly closer every quarter, and here's what the intermediate state looks like, and we might even get there" The interesting thing in all of this to me is what must happen for the same spec to be deterministically certain to generate the same software. Could you delete the code, regenerate from docs alone, and trust the result? And obviously.... not yet. In practice, workflows drift, sometimes you patch the code directly because it's faster, and now code and docs have not been properly updated. But the entire flow and concept of: [Spec] -> [AI/Harness] -> [Finished software], and how we increasing determinism in that flow, is just immensely interesting to me.
  - glouwbug 1mo ago
    Right, but hasn't generating the perfect spec always been the _exact_ problem of software development, regardless of language / LLM interface?
    - jtrn 1mo ago
      Yes, I agree with this fully. But what’s interesting now is that the degree of determinism is increasing, as the community as a whole keeps refining the individual parts. The importance of spec becomes more obvious when the feedback loop speeds up, from spec to running software. That already made a huge difference in how many think about spec. There are multiple GitHub projects that are "spec only," where the goal is to spec it out in such a way that the software one wants is the inevitable result, if you just input the spec into a AI/harnes. And the AI gets better, and the harnesses get better. So at some point we probably live in a reality where we can say: "If you spec out the software you want in this specific way, and add in these guidelines in AGENTS.md, and use X AI with Y harness, you almost certainly get identical software out the other end". And yes "almost" is faaaaar from "always identical output". But the fact that we are even in the game of increasing determinism, in the Spec->AI->Software flow, is just mind-blowingly cool to me. That is a detour from my main point, though, that on one specific level of analysis (can we move up one level of abstraction and lose some detail understanding, but gain more in productivity), AI, compiler, software frameworks, are all examples of the answer being: Yes. And I do agree that we need to mitigate the damage that people with less experience can do because they don't know what pitfalls to avoid. But I would rather we focus on fixing that by improving the AI and harness, than the people that just keep saying that "AI is bad". In the same way I would rather make a tractor safer to use, not just complain that it's dangerous because someone drove it into the lake. Because the goal is not to make the perfect deterministic output from a compiler. that's just a step towards the real goal, which should, in my mind, be to help other people solve problems and do useful stuff. In the same way that the goal of the tractor is not to just plow the field, but to plow the field as fast and efficiently as possible so we can feed ourselves. Wall of text because this topic has been bothering me for a while now, and Im using this thread to sort out my own thinking on it.

This analogy only works if you’re ready to commit _only prompts_ and have those prompts do a clean “compile” of your production runtime everytime you pull. Are you ready to do that?

jtrn 1mo ago

See my response to the other comment if you don't understand that analogy is not the same as "identical". And to state that "This analogy only works if ...." is just PATENTLY wrong. The analogy works fine if you say that it compares analogous situations. Like if we focus on some encumbrance complaining that "kids these days are too stupid because they don't understand the fundamentals like I do" or "These new tools that make it easier for stupid people, not smart people like me, to make stuff is dangerous because they don't know what they are doing". That's just a few of MANY analogous observations we can make for the two situations. But I guess you think that only the thing you care about is the only thing that exists. And in the end, everybody who complains like this is just going to be shown to be just as mistaken as all the people who complained that "people who don't code in assembly are dangerous!" And it's just marvelous to watch it play out slowly over the last couple of years. And we are just a couple of years in. I'm just making a note of everybody who is mistaken, as a study in denial and biased thinking. The end for all of this was obvious after Opus 4.6 hit. And it's just getting more and more obvious with each model release and harness improvement. This is a gold mine for studying flawed thinking.

Kim_Bruning 1mo ago

It turns out that this almost works. You'd probably want to commit explicit design documents btw, not every last prompt. FWIW, when people try this out in practice, they tend to commit the design docs alongside the generated tests and the generated code. Mutating the docs then leads to mutations in the code. Several friends and I have workflows that look a bit like this, but I know for sure mine isn't 100% strict(yet?), and ymmv. Everyone is still figuring out the best practices I think.

jtrn 1mo ago
Thank you for the first sensible response that actually engages with the point I made. And your answer is better than mine, and less angry.... :P The question from glouwbug was just taken for granted to be "no, and therefore the analogy fails." And if I understand your answer, it's basically: "no, but we're visibly closer every quarter, and here's what the intermediate state looks like, and we might even get there" The interesting thing in all of this to me is what must happen for the same spec to be deterministically certain to generate the same software. Could you delete the code, regenerate from docs alone, and trust the result? And obviously.... not yet. In practice, workflows drift, sometimes you patch the code directly because it's faster, and now code and docs have not been properly updated. But the entire flow and concept of: [Spec] -> [AI/Harness] -> [Finished software], and how we increasing determinism in that flow, is just immensely interesting to me.
- glouwbug 1mo ago
  Right, but hasn't generating the perfect spec always been the _exact_ problem of software development, regardless of language / LLM interface?
  - jtrn 1mo ago
    Yes, I agree with this fully. But what’s interesting now is that the degree of determinism is increasing, as the community as a whole keeps refining the individual parts. The importance of spec becomes more obvious when the feedback loop speeds up, from spec to running software. That already made a huge difference in how many think about spec. There are multiple GitHub projects that are "spec only," where the goal is to spec it out in such a way that the software one wants is the inevitable result, if you just input the spec into a AI/harnes. And the AI gets better, and the harnesses get better. So at some point we probably live in a reality where we can say: "If you spec out the software you want in this specific way, and add in these guidelines in AGENTS.md, and use X AI with Y harness, you almost certainly get identical software out the other end". And yes "almost" is faaaaar from "always identical output". But the fact that we are even in the game of increasing determinism, in the Spec->AI->Software flow, is just mind-blowingly cool to me. That is a detour from my main point, though, that on one specific level of analysis (can we move up one level of abstraction and lose some detail understanding, but gain more in productivity), AI, compiler, software frameworks, are all examples of the answer being: Yes. And I do agree that we need to mitigate the damage that people with less experience can do because they don't know what pitfalls to avoid. But I would rather we focus on fixing that by improving the AI and harness, than the people that just keep saying that "AI is bad". In the same way I would rather make a tractor safer to use, not just complain that it's dangerous because someone drove it into the lake. Because the goal is not to make the perfect deterministic output from a compiler. that's just a step towards the real goal, which should, in my mind, be to help other people solve problems and do useful stuff. In the same way that the goal of the tractor is not to just plow the field, but to plow the field as fast and efficiently as possible so we can feed ourselves. Wall of text because this topic has been bothering me for a while now, and Im using this thread to sort out my own thinking on it.