3 ms·

This articles ignores that AI agents have intelligence which means that they can figure out unspecified parts of the spec on their own. There is a lot of the de

This articles ignores that AI agents have intelligence which means that they can figure out unspecified parts of the spec on their own. There is a lot of the design of software that I don't care about and I'm fine letting AI pick a reasonable approach.

deleted 8mo ago
[deleted]
ChimpWithHat 8mo ago
Exactly. The real speed up from AI will come when we can under specify a system and the AI uses its intelligence to make good choices on the parts we left out. If you have to spec something out with zero ambiguity you’re basically just coding in English. I suspect current ideas around formal/detailed spec driven development will be abandoned in a couple years when models are significantly better.
- macinjosh 8mo ago
  For now I would be happy if it just explored the problem space and identify the choices to be made and filtered down to the non-obvious and/or more opinionated ones. Bundle these together and ask the me all at once and then it is off to the races.
- majormajor 8mo ago
  This is humans have traditionally done with greenfield systems. No choices have been made yet, they're all cheap decisions. The difficulty has always arisen when the lines of code pile up AND users start requesting other things AND it is important not to break the "unintended behavior" parts of the system that arose from those initial decisions. It would take either a sea-change in how agents work (think absorbing the whole codebase in the context window and understanding it at the level required to anticipate any surprising edge case consequences of a change, instead of doing think-search-read-think-search-read loops) or several more orders of magnitude of speed (to exhaustively chase down the huge number of combinations of logic paths+state that systems end up playing with) to get around that problem. So yeah, hobby projects are a million times easier, as is bootstrapping larger projects. But for business works, deterministic behaviors and consistent specs are important.
- bigstrat2003 8mo ago
  > in a couple years when models are significantly better. They aren't significantly better now than a couple of years ago. So it doesn't seem likely they will be significantly better in a couple of years than they are now.
  - charcircuit 8mo ago
    A couple years ago we didn't even have thinking. AI could barely complete a line of code that you working on and now they are capable of long running tasks like building an entire C compiler.
abcde666777 8mo ago
These algorithms don't have intelligence, they just regurgitate human intelligence that was in their training data. That also goes the other way - they can't produce intelligence that wasn't represented in their training input.
- half-kh-hacker 8mo ago
  How does post-training via reinforcement learning factor in? Does every evaluated judgement count as 'the training data' ?
  - abcde666777 8mo ago
    I guess I'd place both within a broader umbrella: human generated input. So it still holds that they're regurgitating the decisions made by humans.
  - internet_points 8mo ago
    yes
- charcircuit 8mo ago
  Firstly, it doesn't really matter if they can produce novel designs or not. 99% of what is being done is not novel. It is manipulating data in ways computers have been manipulating data for decades. The design of what is implemented is also going to be derivative of what already exists in the world too. Being too novel makes for a bad product since users will not easily understand how to use it and adapt their existing knowledge of how other things work. Secondly, they are able to produce intelligence that wasn't represented in their training input. As a simple example take a look at chess AI. The top chess engines have more intelligence over the game of chess than the top humans. They have surpassed humans understanding of chess. Similar with LLMs. They train on synthetic data that other LLMs have made and are able to find ways to get better and better on their own. Humans learn off the knowledge of other humans and it compounds. The same thing applies to AI. It is able to generated information and try things and then later reference what it tried when doing something else.
TheRoque 8mo ago
Exactly, I find that type of article too dismissive. Like, we know we don't have to write the full syntax of a loop when we write the spec "find the object in the list", and we might even not write this spec because that part is obvious to any human (hence to an LLM too)
ozozozd 8mo ago
Here is an easy example: Say you and I both wrote the same spec that under-specifies the same parts. But we both expect different behavior, and trust that LLM will make the _reasonable_ choices. Hint: “The choice that I would have made.” Btw, by definition, when we under-specify we leave some decisions to the LLM unknowingly. And absent our looks or age as input, the LLM will make some _reasonable_ choices based on our spec. But will those choices be closer to yours or mine? Assuming it won’t be neither.
- charcircuit 8mo ago
  And I'll say it doesn't matter. If it goes with your approach then I'll say that the approach works too. And if goes with my approach I hope you would recognize that my approach also is good enough for what we need. As long as the software meets the requirements, its okay if it isn't implemented the exact way it would have been if it was done by hand.
  - bigfishrunning 8mo ago
    > As long as the software meets the requirements "Requirements" are a spec. If your requirements lack detail, you probably won't get what you expected -- vague "requirements" can be met without solving your problem in many cases.
    - charcircuit 8mo ago
      It's okay if you don't get what you expect as long as it works. That's the point. There's a ton of different valid permutations and letting a LLM pick a good one ends up working in practice.
hrmtst93837 8mo ago
[flagged]

deleted 8mo ago

[deleted]

ChimpWithHat 8mo ago

Exactly. The real speed up from AI will come when we can under specify a system and the AI uses its intelligence to make good choices on the parts we left out. If you have to spec something out with zero ambiguity you’re basically just coding in English. I suspect current ideas around formal/detailed spec driven development will be abandoned in a couple years when models are significantly better.

macinjosh 8mo ago
For now I would be happy if it just explored the problem space and identify the choices to be made and filtered down to the non-obvious and/or more opinionated ones. Bundle these together and ask the me all at once and then it is off to the races.
majormajor 8mo ago
This is humans have traditionally done with greenfield systems. No choices have been made yet, they're all cheap decisions. The difficulty has always arisen when the lines of code pile up AND users start requesting other things AND it is important not to break the "unintended behavior" parts of the system that arose from those initial decisions. It would take either a sea-change in how agents work (think absorbing the whole codebase in the context window and understanding it at the level required to anticipate any surprising edge case consequences of a change, instead of doing think-search-read-think-search-read loops) or several more orders of magnitude of speed (to exhaustively chase down the huge number of combinations of logic paths+state that systems end up playing with) to get around that problem. So yeah, hobby projects are a million times easier, as is bootstrapping larger projects. But for business works, deterministic behaviors and consistent specs are important.
bigstrat2003 8mo ago
> in a couple years when models are significantly better. They aren't significantly better now than a couple of years ago. So it doesn't seem likely they will be significantly better in a couple of years than they are now.
- charcircuit 8mo ago
  A couple years ago we didn't even have thinking. AI could barely complete a line of code that you working on and now they are capable of long running tasks like building an entire C compiler.

abcde666777 8mo ago

These algorithms don't have intelligence, they just regurgitate human intelligence that was in their training data. That also goes the other way - they can't produce intelligence that wasn't represented in their training input.

half-kh-hacker 8mo ago
How does post-training via reinforcement learning factor in? Does every evaluated judgement count as 'the training data' ?
- abcde666777 8mo ago
  I guess I'd place both within a broader umbrella: human generated input. So it still holds that they're regurgitating the decisions made by humans.
- internet_points 8mo ago
  yes
charcircuit 8mo ago
Firstly, it doesn't really matter if they can produce novel designs or not. 99% of what is being done is not novel. It is manipulating data in ways computers have been manipulating data for decades. The design of what is implemented is also going to be derivative of what already exists in the world too. Being too novel makes for a bad product since users will not easily understand how to use it and adapt their existing knowledge of how other things work. Secondly, they are able to produce intelligence that wasn't represented in their training input. As a simple example take a look at chess AI. The top chess engines have more intelligence over the game of chess than the top humans. They have surpassed humans understanding of chess. Similar with LLMs. They train on synthetic data that other LLMs have made and are able to find ways to get better and better on their own. Humans learn off the knowledge of other humans and it compounds. The same thing applies to AI. It is able to generated information and try things and then later reference what it tried when doing something else.

TheRoque 8mo ago

Exactly, I find that type of article too dismissive. Like, we know we don't have to write the full syntax of a loop when we write the spec "find the object in the list", and we might even not write this spec because that part is obvious to any human (hence to an LLM too)

ozozozd 8mo ago

Here is an easy example: Say you and I both wrote the same spec that under-specifies the same parts. But we both expect different behavior, and trust that LLM will make the _reasonable_ choices. Hint: “The choice that I would have made.” Btw, by definition, when we under-specify we leave some decisions to the LLM unknowingly. And absent our looks or age as input, the LLM will make some _reasonable_ choices based on our spec. But will those choices be closer to yours or mine? Assuming it won’t be neither.

charcircuit 8mo ago
And I'll say it doesn't matter. If it goes with your approach then I'll say that the approach works too. And if goes with my approach I hope you would recognize that my approach also is good enough for what we need. As long as the software meets the requirements, its okay if it isn't implemented the exact way it would have been if it was done by hand.
- bigfishrunning 8mo ago
  > As long as the software meets the requirements "Requirements" are a spec. If your requirements lack detail, you probably won't get what you expected -- vague "requirements" can be met without solving your problem in many cases.
  - charcircuit 8mo ago
    It's okay if you don't get what you expect as long as it works. That's the point. There's a ton of different valid permutations and letting a LLM pick a good one ends up working in practice.

hrmtst93837 8mo ago

[flagged]