3 ms·

I can agree with this sentiment. It does not matter how insanely good LLMs become, if you cannot assess it quickly enough. You will ALWAYS want a human to verif

I can agree with this sentiment. It does not matter how insanely good LLMs become, if you cannot assess it quickly enough. You will ALWAYS want a human to verify and validate, and test the software. There could be a ticking timebomb in there somewhere. Maybe the real skynet will kill us with ticking time bomb software bugs we blindly accepted.

thenanyu 1y ago
In most scenarios I can tell you if I like or dislike a feature much faster than it takes a developer to build it
- k__ 1y ago
  If it just came down to the "idea guy liking or disliking a feature" things would be quite easy...
  - thenanyu 1y ago
    why doesn't it? it doesn't have to be you or me personally, it could be a representative sample of our users
    - cestith 1y ago
      So if you wait to put together a representative sample of users and gather the data long enough for the numbers to matter, you’ve gated further changes. If you’ve gated further changes for a week, why does it matter that the feature change was done in an hour or a day?
      - thenanyu 1y ago
        Releasing it to users does not take a long time. Randomly select 5% of your user base and give them the feature. If your development process was mature, this would be a button you could push in your deployment env.
        cestith 1y ago
        Deployment isn’t the problem. If you’ve pushed a new version of a feature that five percent of users use per week and you’ve pushed it to five percent of the userbase how long do you need to wait to have reliable data on that before you push another experimental version?
ACCount37 1y ago
The threshold of supervision keeps rising - and it's going to keep rising. GPT-2 was barely capable of writing two lines of code. GPT-3.5 could write a simple code snippet, and be right more often than it was wrong. GPT-4 was a leap over that, enabling things like "vibe coding" for small simple projects, and GPT-5 is yet another advancement in the same direction. Each AI upgrade brings forth more capabilities - with every upgrade, the AI can go further before it needs supervision. I can totally see the amount of supervision an AI needs collapsing to zero within our lifetimes.
- gyrovagueGeist 1y ago
  In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. Because they generate so much code, that often passes initial tests, looks reasonable, and fails in nonhuman ways, in a pretty opinionated style tbh. I have less context (and need to spend much more effort and supervision time to get up to speed to learn) to fix, refactor, and integrate the solutions, than if I was only trusting short few line windows at a time.
  - warkdarrior 1y ago
    > I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. That is because you are trained in the old way to writing code: manual crafting of software line by line, slowly, deliberately, thoughtfully. New generations of developers will not use the same workflow as you, just like you do not use the same workflow as folks who programmed punch cards.
    - _se 1y ago
      No, it's because reading code is slower than writing it. The only way these tools can possibly be faster for non-trivial work is if you don't give a shit enough about the output to not even read it. And if you can do that and still achieve your goal, chances are your goal wasn't that difficult to begin with. That's why we're now consistently measuring individuals to be slower using these tools even though many of them feel faster.
IanCal 1y ago
That doesn’t require developer time though. Also that time is needed regardless, do you think it’s the majority of time related to releasing a feature?

thenanyu 1y ago

In most scenarios I can tell you if I like or dislike a feature much faster than it takes a developer to build it

k__ 1y ago
If it just came down to the "idea guy liking or disliking a feature" things would be quite easy...
- thenanyu 1y ago
  why doesn't it? it doesn't have to be you or me personally, it could be a representative sample of our users
  - cestith 1y ago
    So if you wait to put together a representative sample of users and gather the data long enough for the numbers to matter, you’ve gated further changes. If you’ve gated further changes for a week, why does it matter that the feature change was done in an hour or a day?
    - thenanyu 1y ago
      Releasing it to users does not take a long time. Randomly select 5% of your user base and give them the feature. If your development process was mature, this would be a button you could push in your deployment env.
      - cestith 1y ago
        Deployment isn’t the problem. If you’ve pushed a new version of a feature that five percent of users use per week and you’ve pushed it to five percent of the userbase how long do you need to wait to have reliable data on that before you push another experimental version?

ACCount37 1y ago

The threshold of supervision keeps rising - and it's going to keep rising. GPT-2 was barely capable of writing two lines of code. GPT-3.5 could write a simple code snippet, and be right more often than it was wrong. GPT-4 was a leap over that, enabling things like "vibe coding" for small simple projects, and GPT-5 is yet another advancement in the same direction. Each AI upgrade brings forth more capabilities - with every upgrade, the AI can go further before it needs supervision. I can totally see the amount of supervision an AI needs collapsing to zero within our lifetimes.

gyrovagueGeist 1y ago
In the middle term, I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. Because they generate so much code, that often passes initial tests, looks reasonable, and fails in nonhuman ways, in a pretty opinionated style tbh. I have less context (and need to spend much more effort and supervision time to get up to speed to learn) to fix, refactor, and integrate the solutions, than if I was only trusting short few line windows at a time.
- warkdarrior 1y ago
  > I almost feel less productive using modern GPT-5/Claude Sonnet 4 for software dev than prior models, precisely because they are more hands off and less supervised. That is because you are trained in the old way to writing code: manual crafting of software line by line, slowly, deliberately, thoughtfully. New generations of developers will not use the same workflow as you, just like you do not use the same workflow as folks who programmed punch cards.
  - _se 1y ago
    No, it's because reading code is slower than writing it. The only way these tools can possibly be faster for non-trivial work is if you don't give a shit enough about the output to not even read it. And if you can do that and still achieve your goal, chances are your goal wasn't that difficult to begin with. That's why we're now consistently measuring individuals to be slower using these tools even though many of them feel faster.

IanCal 1y ago

That doesn’t require developer time though. Also that time is needed regardless, do you think it’s the majority of time related to releasing a feature?