3 ms·

On the one hand, it's impressive that they've apparently built a general-purpose LLM interface for this. On the other hand, even before considering LLM limitati

On the one hand, it's impressive that they've apparently built a general-purpose LLM interface for this. On the other hand, even before considering LLM limitations, trying to define app behavior of any complexity using plain language is a great way to get a buggy and inconsistent mess.

bufferoverflow 2y ago
Yes, even when we get designs that passed multiple rounds of user interviews, even after developers went over them and approved and estimated work, we still end up finding undefined behaviors while coding it. We would need AGI-level AI to code software from English and not produce buggy mess. On the other hand, current top LLMs are pretty great at writing chunks of code if you specify exactly what you want. I am personally impressed with Claude 3.5.
- jimnotgym 2y ago
  I would also find developers who could take the slightest vague term, misunderstand it in a 'technically correct' but to everyone else, outrageous way. To counter this I used to hold short meetings with them every single morning to pull them back into line! We also shortened every build into bite sized chunks so the humans could check they didn't go off-piste. We called this Agile. Poor communication between humans is seemingly insoluble. Perhaps a computer misunderstanding you in real time so you can correct is the best solution?
  - intelVISA 2y ago
    Comms is a Hard Problem, but specs should not be regularly ambiguous that they require daily course correct - huge velocity killer.
frereubu 2y ago
Exactly. Even when building websites for clients they often ask for mututally incompatible things within the same sentence, and there's often hidden complexity buried in the most innocuous-sounding sentences.
jimnotgym 2y ago
Indeed. This one reason why when humans describe what they want from their software, developers build something entirely different from what was requested. Perhaps this shortens the cycle time?

bufferoverflow 2y ago

Yes, even when we get designs that passed multiple rounds of user interviews, even after developers went over them and approved and estimated work, we still end up finding undefined behaviors while coding it. We would need AGI-level AI to code software from English and not produce buggy mess. On the other hand, current top LLMs are pretty great at writing chunks of code if you specify exactly what you want. I am personally impressed with Claude 3.5.

jimnotgym 2y ago
I would also find developers who could take the slightest vague term, misunderstand it in a 'technically correct' but to everyone else, outrageous way. To counter this I used to hold short meetings with them every single morning to pull them back into line! We also shortened every build into bite sized chunks so the humans could check they didn't go off-piste. We called this Agile. Poor communication between humans is seemingly insoluble. Perhaps a computer misunderstanding you in real time so you can correct is the best solution?
- intelVISA 2y ago
  Comms is a Hard Problem, but specs should not be regularly ambiguous that they require daily course correct - huge velocity killer.

frereubu 2y ago

Exactly. Even when building websites for clients they often ask for mututally incompatible things within the same sentence, and there's often hidden complexity buried in the most innocuous-sounding sentences.

jimnotgym 2y ago

Indeed. This one reason why when humans describe what they want from their software, developers build something entirely different from what was requested. Perhaps this shortens the cycle time?