4 ms·

Why do you need to review design decisions with a team anymore? I get the impression that Fable, when well directed, is better than maybe 80% of SWEs. Maybe mo

by luc_ 12d ago

Why do you need to review design decisions with a team anymore? I get the impression that Fable, when well directed, is better than maybe 80% of SWEs. Maybe more. [edit: Yes, I'm maybe baiting other users, but I want to know your honest opinions on this.]

mtlynch 12d ago
I find that LLMs are still worse than humans at limiting complexity, which is one of the most important outcomes of a design review. If I tell a senior SWE that I'm creating a Discourse-like discussion forum, and I want users to have three options for selecting an avatar: (1) import from Gravatar, (2) upload a JPG or SVG or PNG or GIF, or (3) let the user draw their avatar on a canvas, the LLM will happily go and design that and write a 5 KLOC implementation, whereas a good SWE would push back and say, "That's like 10x the complexity of just allowing JPGs. How about we simplify it to say that in v1, the only option is to upload a JPG." I've tried working with Fable/Sol and saying, "Look for features that we can simplify to reduce complexity," and they don't understand. They'll guess at features we can cut entirely, but they fail to see how to capture the essence of the feature without the complexity. I've noticed this a lot with Fable recently. Like I'll say, "Show an error message in the web UI if X fails," and Fable comes back with this like 800 LOC error message generator that has switch-cases and combines inputs from three different sources when all I wanted was something like, "Update failed: database is locked."
- Fripplebubby 12d ago
  I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? If it is the secondary outcomes that are the enemy, and AI actually effectively allows you to mitigate those outcomes (debatable! I debate this with myself all the time!), then maybe we should embrace the complexity (or the agent should on our behalf)
  - mtlynch 12d ago
    > I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? I agree, but I think we're still a long way away from being able to trust AI to manage all software complexity for us. For one, LLMs frequently get tripped up by their own complexity. But even if the complexity didn't make LLMs more error prone or expensive to run, you still often need a human in the loop to understand what the system does. I think of it kind of like compilers. Compilers do a good enough job that 99% of developers don't understand code at the bytecode or machine instruction level, but if we lost that last 1% of programmers who understand CPU instructions, we'd be in serious trouble.
lirolero 12d ago
[dead]
bigstrat2003 11d ago
LLMs are worse than any competent programmer. Just turning them loose is a terrible idea.

by luc_ 12d ago

mtlynch 12d ago

I find that LLMs are still worse than humans at limiting complexity, which is one of the most important outcomes of a design review. If I tell a senior SWE that I'm creating a Discourse-like discussion forum, and I want users to have three options for selecting an avatar: (1) import from Gravatar, (2) upload a JPG or SVG or PNG or GIF, or (3) let the user draw their avatar on a canvas, the LLM will happily go and design that and write a 5 KLOC implementation, whereas a good SWE would push back and say, "That's like 10x the complexity of just allowing JPGs. How about we simplify it to say that in v1, the only option is to upload a JPG." I've tried working with Fable/Sol and saying, "Look for features that we can simplify to reduce complexity," and they don't understand. They'll guess at features we can cut entirely, but they fail to see how to capture the essence of the feature without the complexity. I've noticed this a lot with Fable recently. Like I'll say, "Show an error message in the web UI if X fails," and Fable comes back with this like 800 LOC error message generator that has switch-cases and combines inputs from three different sources when all I wanted was something like, "Update failed: database is locked."

Fripplebubby 12d ago
I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? If it is the secondary outcomes that are the enemy, and AI actually effectively allows you to mitigate those outcomes (debatable! I debate this with myself all the time!), then maybe we should embrace the complexity (or the agent should on our behalf)
- mtlynch 12d ago
  > I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? I agree, but I think we're still a long way away from being able to trust AI to manage all software complexity for us. For one, LLMs frequently get tripped up by their own complexity. But even if the complexity didn't make LLMs more error prone or expensive to run, you still often need a human in the loop to understand what the system does. I think of it kind of like compilers. Compilers do a good enough job that 99% of developers don't understand code at the bytecode or machine instruction level, but if we lost that last 1% of programmers who understand CPU instructions, we'd be in serious trouble.

lirolero 12d ago

[dead]

bigstrat2003 11d ago

LLMs are worse than any competent programmer. Just turning them loose is a terrible idea.