3 ms·

Yep, my strong feeling is that the net benefit of all of this will be zero. The time you have to spend holding the LLM hand is almost equal to how much time you

Yep, my strong feeling is that the net benefit of all of this will be zero. The time you have to spend holding the LLM hand is almost equal to how much time you would have spent writing it yourself. But then you've got yourself a codebase that you didn't write yourself, and we all know hunting bugs in someone else's code is way harder than code you had a part in designing/writing. People are honestly just drunk on this thing at this point. The sunken cost fallacy has people pushing on (ie. spending more time) when LLMs aren't getting it right. People are happy to trade convenience for everything else, just look at junk food where people trade in flavour and their health. And ultimately we are in a time when nobody is building for the future, it's all get rich quick schemes: squeeze then get out before anyone asks why the river ran dry. LLMs are like the perfect drug for our current society. Just look at how technology has helped us in the past decades. Instead of launching us towards some kind of Star Trek utopia, most people now just work more for less!

jama211 1y ago
Only when purely vibe coding. AI currently saves a LOT of time if you get it to generate boilerplate, diagnose bugs, or assist with sandboxed issues. The proof is in the pudding. The work I do takes me half as long as it used to and is just as high in quality, even though I manage and carefully curate the output.
- sarchertech 1y ago
  I use AI for most of those things. And I think it probably saves me a bit of time. But in that study that came out a few weeks ago where they actually looked at time saved, every single developer overestimated their time saved. To the point where even the ones who lost time thought they saved time. LLMs are very good at making you feel like you’re saving time even when you aren’t. That doesn’t mean they can’t be a net productivity benefit. But I’d be very very very surprised if you have real hard data to back up your feelings about your work taking you half as long and being equal quality.
  - Filligree 1y ago
    That study predates Claude Code though. I’m not surprised by the contents. I had the same feeling; I made some attempts at using LLMs for coding prior to CC, and with rare exceptions it never saved me any time. CC changed that situation hugely, at least in my subjective view. It’s of course possible that it’s not as good as I feel it is, but I would at least want a new study.
    - jimbokun 1y ago
      > That study predates Claude Code though. Is there a study demonstrating Claude Code improves productivity?
    - sarchertech 1y ago
      I don’t believe that CC is so much better than cursor using Claude models that it moves the needle enough to flip the results of that study. The key thing to look at is that even the participants that did objectively save time, overestimated time saved by a huge amount. But also you’re always likely to be at least one model ahead of any studies that come out.
  - jama211 1y ago
    I mean, I used to average 2 hours of intense work a day and now it’s 1 hour.
    - sarchertech 1y ago
      How are you tracking that? Are you keeping a log, or are you just guessing? Do you have a mostly objective definition of intense work or are you just basing it on how you feel? Is your situation at work otherwise exactly the same, or have you gotten into a better groove with your manager? Are you working on exactly the same thing? Have you leveled up with some more experience? Have you learned the domain better? Is your work objectively the same quality? Is it possible that you are producing less but it’s still far above the minimum so no one has noticed? Is your work good enough for now, but a year from now when someone tries to change it, it will be a lot harder for them? Based on the only real studies we have, humans grossly overestimate AI time savings. It’s highly likely you are too.
- globular-toast 1y ago
  I don't write much boilerplate anyway. I long ago figured out ways to not do that (I use a computer to do repetitive tasks for me). So when people talk about boilerplate I feel like they're only just catching up to me, not surpassing me. As for catching bugs, maybe, but I feel like it's pot luck. Sometimes it can find something, sometimes it's just complete rubbish. Sometimes worth giving it a spin but still not convinced it's saving that much. Then again I don't spend much time hunting down bugs in unfamiliar code bases.
  - jama211 1y ago
    Like any tool, it has use cases where it excels and use cases where it’s pretty useless. Unfamiliar code bases is a great example, if it’s able to find the bug it could do so almost instantly, as opposed to a human trying to read through the code base for ages. But for someone who is intimately familiar with a code base, they’ll probably solve the problem way faster, especially if it’s subtle. Also say if your job is taking image designs and building them in html/css, just feeding it an image getting it to dump you an html/css framework and then you just clean up the details of will save you a lot of time. But on the flip side if you need to make critically safe software where every line matters, you’ll be way faster on your own. People want to give a black and white “ai is bad” or “ai is great”, but the truth _as always_ is “it depends”. Humans aren’t very good at “it depends”.
utyop22 1y ago
My way of looking at this is simple. What are people doing this quote on quote, time that they have gained back? Working on new projects? Ok can you show me the impact on the financials (show me the money)? And then I usually get dead silence. And before someone mentions the layoffs - lmao get real. Its offshoring 2.0 so that the large firms can increase their internal equity to keep funding this effort. Most people are terrible at giving true informed opinions - they never dig deep enough to determine if what they are saying is proper.

jama211 1y ago

Only when purely vibe coding. AI currently saves a LOT of time if you get it to generate boilerplate, diagnose bugs, or assist with sandboxed issues. The proof is in the pudding. The work I do takes me half as long as it used to and is just as high in quality, even though I manage and carefully curate the output.

sarchertech 1y ago
I use AI for most of those things. And I think it probably saves me a bit of time. But in that study that came out a few weeks ago where they actually looked at time saved, every single developer overestimated their time saved. To the point where even the ones who lost time thought they saved time. LLMs are very good at making you feel like you’re saving time even when you aren’t. That doesn’t mean they can’t be a net productivity benefit. But I’d be very very very surprised if you have real hard data to back up your feelings about your work taking you half as long and being equal quality.
- Filligree 1y ago
  That study predates Claude Code though. I’m not surprised by the contents. I had the same feeling; I made some attempts at using LLMs for coding prior to CC, and with rare exceptions it never saved me any time. CC changed that situation hugely, at least in my subjective view. It’s of course possible that it’s not as good as I feel it is, but I would at least want a new study.
  - jimbokun 1y ago
    > That study predates Claude Code though. Is there a study demonstrating Claude Code improves productivity?
  - sarchertech 1y ago
    I don’t believe that CC is so much better than cursor using Claude models that it moves the needle enough to flip the results of that study. The key thing to look at is that even the participants that did objectively save time, overestimated time saved by a huge amount. But also you’re always likely to be at least one model ahead of any studies that come out.
- jama211 1y ago
  I mean, I used to average 2 hours of intense work a day and now it’s 1 hour.
  - sarchertech 1y ago
    How are you tracking that? Are you keeping a log, or are you just guessing? Do you have a mostly objective definition of intense work or are you just basing it on how you feel? Is your situation at work otherwise exactly the same, or have you gotten into a better groove with your manager? Are you working on exactly the same thing? Have you leveled up with some more experience? Have you learned the domain better? Is your work objectively the same quality? Is it possible that you are producing less but it’s still far above the minimum so no one has noticed? Is your work good enough for now, but a year from now when someone tries to change it, it will be a lot harder for them? Based on the only real studies we have, humans grossly overestimate AI time savings. It’s highly likely you are too.
globular-toast 1y ago
I don't write much boilerplate anyway. I long ago figured out ways to not do that (I use a computer to do repetitive tasks for me). So when people talk about boilerplate I feel like they're only just catching up to me, not surpassing me. As for catching bugs, maybe, but I feel like it's pot luck. Sometimes it can find something, sometimes it's just complete rubbish. Sometimes worth giving it a spin but still not convinced it's saving that much. Then again I don't spend much time hunting down bugs in unfamiliar code bases.
- jama211 1y ago
  Like any tool, it has use cases where it excels and use cases where it’s pretty useless. Unfamiliar code bases is a great example, if it’s able to find the bug it could do so almost instantly, as opposed to a human trying to read through the code base for ages. But for someone who is intimately familiar with a code base, they’ll probably solve the problem way faster, especially if it’s subtle. Also say if your job is taking image designs and building them in html/css, just feeding it an image getting it to dump you an html/css framework and then you just clean up the details of will save you a lot of time. But on the flip side if you need to make critically safe software where every line matters, you’ll be way faster on your own. People want to give a black and white “ai is bad” or “ai is great”, but the truth _as always_ is “it depends”. Humans aren’t very good at “it depends”.

utyop22 1y ago

My way of looking at this is simple. What are people doing this quote on quote, time that they have gained back? Working on new projects? Ok can you show me the impact on the financials (show me the money)? And then I usually get dead silence. And before someone mentions the layoffs - lmao get real. Its offshoring 2.0 so that the large firms can increase their internal equity to keep funding this effort. Most people are terrible at giving true informed opinions - they never dig deep enough to determine if what they are saying is proper.