5 ms·

I'm usually skeptical of doomer articles about new technology like this one, but reluctantly find myself agreeing with a lot of it. While AI is a great tool wit

• zero_shift 2y ago
  I'm sure I've read somewhere - and it annoys me immensely that I can't recall the source - that SWEs perceive they are more productive with AI, but the measurements say they aren't.
  • tartoran 2y ago
    More productive with AI or more productive in general?
  • bwestergard 2y ago
    This seems right, intuitively, but I'd love to see a source. I've noticed that when I detect chatbot-style code in a coworker's PR, I often find subtle problems. But it's harder for me to spot the problems in code I got out of a chatbot because I am primed by the very activity of writing the prompt to see what I desire in the output.
  • bluefirebrand 2y ago
    This is my observation. Not a scientific measurement by any means but from what I can see it isn't speeding anyone up If I had to guess, people feel more productive because they are doing less of the work they are used to and more review / testing, but to reach the same level of confidence the review takes much longer And the people who are not doing thorough review are producing absolute garbage, and are basically clueless
  • h4ny 2y ago
    Tangentially related, I feel that SWEs who claim that they are more productive with AI haven't actually demonstrated with real examples of how they are actually more productive. Nobody I follow (including some prominent bloggers and YouTubers) claiming productivity increase is recording or detailing any workflow or showing real world, non-hobby (scalable, maintainable, readable, secure, etc.) workflows of how to do it. It's like everyone who "knows what they are doing" is hiding what the secret sauce for a competitive edge or that they are all just mediocre SWEs hyping AI up and lying because it makes them more money. Even real SWEs in large companies I know can't really seem to tell me how their productivity is increasing, and when you dig deeper it always seem to be well-scoped and well-understood problems (which is great, but doesn't match the level of hype and productivity increase that everyone else is claiming) -- and they still have to be very careful with reviewing (for now). It's almost like AI makes SWE brains go mush and forget about logic and data.
    • simonw 2y ago
      I wrote 4,800 words about how I'm using LLMs to help me code here, because I was frustrated at how little detailed information there was on that topic: https://simonwillison.net/2025/Mar/11/using-llms-for-code/ https://simonwillison.net/2025/Mar/11/using-llms-for-code/
    • nerdponx 2y ago
      I'm a "data scientist", but I have absolutely improved my productivity in the last year or so by conversing with LLM chatbots to work through tough problems, get ideas, figure out project plans, etc. I can see the effect in my list of completed projects, the overall speed isn't that much higher, but the quality has definitely gone up, because I'm able to work through things more quickly and get to good solutions faster, so I can spend the more time iterating on good ideas and less time trying figure out which ideas are even good. For programming, meh, it helps when I'm really tired and don't want to read documentation. Can't imagine using it in a serious capacity for writing code except in a huge codebase, where I might want it to explain to me how things fit together in order to make some change or fix a bug.
    • MattSayar 2y ago
      In fairness, it's also still a NEW technology in the scale of tech. For comparison, it takes years after a gaming console is released for teams to optimize and squeeze every last ounce of performance out of the hardware. We're just getting started with AI, and we're still "stuck" in the chat interfaces because of the storming success of ChatGPT a few years ago. Cursor, GitHub Copilot etc. are cool but they're still "launch titles" to continue my analogy from above. New models are still coming out (but slowing down) with increased capabilities, context windows, etc. and I'm sure the killer app is still waiting to be unearthed. In the meantime, I'm having a lot of fun building my hobby code. Collectively, we're going to morph that into something more scalable and enterprisey, it's just a matter of time.
  • ohgr 2y ago
    It depends how you measure productivity and value. And who is measuring it. And who tells the story. If the developer writes 6,000 lines of utter dog shit with AI that causes your customers to leave, well.
  • abalashov 2y ago
    Interestingly, this is the conclusion reached by the major militaries, Axis and Allied alike, at the end of extensive experiments with amphetamines in WWII. They certainly made pilots and soldiers feel more confident, engaged and attentive, but the quality of the output was at best unchanged and at worst markedly inferior.
    • fc417fc802 2y ago
      Depending on how they're used. That's a pretty big caveat. You can't replace sleep with them and expect the same performance. They're still quite useful though.
  • bobbiechen 2y ago
    I can believe that, with personal experience from a non-AI tool! A few years back, I wrote a puzzle solving tool (semaphore decoder) that felt faster than using a lookup table manually, but was actually very similar in time. Those notes: https://bobbiechen.com/blog/2020/5/28/the-making-of-semaphore-decoder https://bobbiechen.com/blog/2020/5/28/the-making-of-semaphor... Regardless of the speed, it certainly felt easier because I didn't have to think as hard, and maybe that extra freshness would improve productivity for later tasks. I wonder if there's any effect like that for AI coding tools - it makes you happier to be less tired.
  • spacemadness 2y ago
    I perceive quite the opposite. Rarely do I see it producing workable solutions and it often just creates noise. What’s worse is the mistakes it makes sometimes are nuanced, and not the kind of mistakes a human coder would make, causing me to waste a lot of time finding the mistake. I think it’s more useful to get ideas from, or treat it like a trainer when learning a new language, but code generation seems really poor to me still. The only ones I see arguing that its not the case are junior coders making slop apps that do nothing all that interesting.
  • nyarlathotep_ 2y ago
    My experience is that "the rent ends up due" basically you "pay cognitively" up front (building an understanding from/while doing) or later (when you have to troubleshoot something in a largely LLM-generated tangle. Basically it moves from--"oh yeah I wrote something with this schema earlier" to "I saw some DB code fly out around an hour ago; maybe it's there. Where was it? `grep models ./src` wait, was it in `db` and other silly stuff like that. No free lunches or whatever. I'm not an extensive LLM user for programming and remain mostly agnostic on overall uses for development (sure a brand-new React thing and you're sailing, but a huge old crusty codebase, even in a language well-represented in the training set is a LOT less promising IME) However there's use cases where I'd straight up not do whatever until the latest minute possible that I use LLMs for now: Cloudformation, various utility bash scripts, simple AWS Lambda functions and other things I consider annoying chores. For me, these cases alone have been an unambiguous victory.
• simonw 2y ago
  > Surveys confirm that for many workers, AI tools like ChatGPT reduce their productivity I'm pretty suspicious of that survey (the one that always gets cited as proof that Copilot makes developers less productive, which then inevitably gets used to argue that all generative AI makes developers left productive): https://resources.uplevelteam.com/gen-ai-for-coding https://resources.uplevelteam.com/gen-ai-for-coding If I was running a company like https://uplevelteam.com/ https://uplevelteam.com/ that sells developer productivity metrics software, one of the smartest marketing moves I could make would be to put out a report that makes a bold, contrarian claim about a hot topic like AI coding productivity. Guaranteed to get a ton of press coverage from that. Is the survey itself any good? I filled in the "request a copy" form and it's a two page infographic! Precious few confirmable details on how they actually ran it: https://static.simonwillison.net/static/2025/uplevel-genai-productivity.pdf https://static.simonwillison.net/static/2025/uplevel-genai-p... Here's what they say about their methodology: > Metrics were evaluated prior to implementation of Copilot from January 9 through April 9, 2023 versus after implementation from January 8 through April 7, 2024. This time period was selected to remove the effects of seasonality. > Data on Copilot access was provided to Uplevel Data Labs across several enterprise engineering customers for a total of 351 developers in the TEST group (with Copilot access) and 434 in the CONTROL group (without Copilot access). The developers in the CONTROL group were similar to those in the TEST group in terms of role, working days, and PR volume in each period.
  • strict9 2y ago
    I said it in a confusing way, but I do believe it increases productivity, at least for me. But it's hazy and hard to measure. I am very rarely stuck on hard problems like I was 3+ years ago. But I lose time in other ways. I've never measured it so it's really just a feeling. I am also skeptical of a company selling something to sponsor or put out a report directly related to what they're selling.
    • pcthrowaway 2y ago
      If I add in all the time I spend reading about AI now, or wading through AI slop while researching something, any productivity gains I may see from actually using AI effectively are more than cancelled out. Interestingly, AI tooling existing in its current form may be making people collectively less productive, even if individually it might make one somewhat more productive at very specific tasks.
• deleted 2y ago
  [deleted]
• cmrdporcupine 2y ago
  Deskilling labour in order to endroute around conflict in the workplace is as old as capitalism. It's not just about automating to reduce expenses, but also about reducing both bargaining power and the "specialness" of the worker. Uppity Google employees raising a stink and not doing their job because of CEOs sexually harassing employees? Or passing around petitions about making weapons or unethical AI? Sharing their compensation package #s to improve bargaining position or deal with perceived injustices around salary differentials? They only get away with that because they have bargaining power that management would dearly like them not to have. The aggressive pivot to "AI" LLM tools is the desperate move of a managerial class sick of our uppity shit.
• nerdponx 2y ago
  This has been my complaint about AI from the beginning, and it hasn't gotten better. In the time spend I figuring out how to explain to the AI what I need it to do, I can just sit down and figure it out. AI, for me, has never been a useful assistant for writing code. Where AI has really improved my productivity is in acting like a colleague I can talk to. "I'm going to start working on X: how would you approach it? what should I know that isn't obvious from the beginning?" or "I am thinking about using Y approach for X problem but I don't see a lot of literature about it. Is there a reason it's not more common?". These "chats" only take 10-30 minutes and have already led me to learn a bunch of new things, and helps keep me moving on projects where in the past I'd have spent 2-3x as long working through ideas, searching for literature, and figuring things out. The combination of "every textbook and journal article in existence" with "more or less understands every topic in its training data" is incredibly powerful, especially for people who either didn't do a lot of formal school training or haven't had time to keep up with new developments. Beginners can benefit from this kind of interaction too, they'll just be talking about simpler topics (which the bot would do even better with) instead of esoterica.
  • zero_shift 2y ago
    From my own (limited) exploration with them, that's how I use them too. As a way to summarise basics and accelerate my progress with new technologies. I don't need anything esoteric here, I just need a source for the essentials that isn't SEO spam and doesn't assume I'm an absolute moron.
  • deleted 2y ago
    [deleted]
• intended 2y ago
  Additional example - Had to review submissions to a conference. You had to pry open a thick rind of words, to get seeds of meaning spread all over the place, and then reconstruct the points being made. Wordy, complex and tiring to analyze. Dumping it into ChatGPT to get answers was an act of frustration, and the output made you more frustrated. It gave you more words, but didn’t help with actual meaning, unless you just gave in and assumed it was accurate. It’s making the job of verification harder, and the job of content creation easier. This is not to society’s larger benefit, since the more challenging job is verification. I shudder to think what is happening with teachers and college at this point.
• formerphotoj 2y ago
  Gary is harsh or extreme, even, but the point largely stands: https://garymarcus.substack.com/p/what-if-generative-ai-turned-out https://garymarcus.substack.com/p/what-if-generative-ai-turn... Personally I think there will be things that gen-AI is useful for, such as rapid education for beginners and learning or performance feedback mechanisms. Those use cases are promising, and still in development. Hopefully they'll be cost-effective as well.