11 ms·

What happens when coding agents stop feeling like dialup?

137 points by martinald 1y ago

SirensOfTitan 1y ago
> Each of these 'phases' of LLM growth is unlocking a lot more developer productivity, for teams and developers that know how to harness it. I still find myself incredibly skeptical LLM use is increasing productivity. Because AI reduces cognitive engagement with tasks, it feels to me like AI increases perceptive productivity but actually decreases it in many cases (and this probably compounds as AI-generated code piles up in a codebase, as there isn't an author who can attach context as to why decisions were made). https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... I realize the author qualified his or her statement with "know how to harness it," which feels like a cop-out I'm seeing an awful lot in recent explorations of AI's relationship with productivity. In my mind, like TikTok or online dating, AI is just another product motion toward comfort maximizing over all things, as cognitive engagement is difficult and not always pleasant. In a nutshell, it is another instant gratification product from tech. That's not to say that I don't use AI, but I use it primarily as search to see what is out there. If I use it for coding at all, I tend to primarily use it for code review. Even when AI does a good job at implementation of a feature, unless I put in the cognitive engagement I typically put in during code review, its code feels alien to me and I feel uncomfortable merging it (and I employ similar levels of cognitive engagement during code reviews as I do while writing software).
- breakfastduck 1y ago
  It depends what environment you're operating within. I've used LLMs for code gen at work as well as for personal stuff. At work primarily for quick and dirty internal UIs / tools / CLIs it's been fantastic, but we've not unleashed it on our core codebases. It's worth noting all the stuff we've got out of out are things we'd not normally have the time to work on - so a net positive there. Outside of work I've built some bespoke software almost entirely generated with human tweaks here and there - again, super useful software for me and some friends to use for planning and managing music events we put on that I'd never normally have the time to build. So in those ways I see it as massively increasing productivity - to build lower stakes things that would normally just never get done due to lack of time.
  - phil21 1y ago
    I do wonder about the second order effects of the second bit. A lot of open source tooling gets created to solve those random "silly" things that are personal annoyances or needs. Then you find out others have the same or similar problem and entire standard tooling or libraries come into existence. I have pontificated on how easy access to immediate "one offs" will kill this idea exchange? Instead of one tool maintained by hundreds to fulfill a common need, we will end up with millions of one-off LLM generated that are not shared with anyone else. Might be a net win, or a net loss. I'm really not sure!
- add-sub-mul-div 1y ago
  > I realize the author qualified his or her statement with "know how to harness it," which feels like a cop-out I'm seeing an awful lot in recent explorations of AI's relationship with productivity. "You're doing AI wrong" is the new "you're doing agile wrong" which was the new "you're doing XP wrong".
  - bitwize 1y ago
    More like the new "you're holding it wrong"
  - pjmlp 1y ago
    Unfortunely many of us are old enough to know how those wrong eventually became the new normal, the wrong way.
  - CuriouslyC 1y ago
    At this point I don't even care to argue with people who disagree with me on this. History will decide the winner and if you think you're not on the losing side, best of luck.
    - jamesnorden 1y ago
      I'd say you're a bit biased.
      - CuriouslyC 1y ago
        I am biased, but by my personal experience, not the desire to sell anything to anyone. I don't even have products for sale right now, I'm just building free things to try and help people figure out how to ride this wave safely.
        svieira 1y ago
        > I don't even have products for sale right now The link in your bio is a long series of tools for various personas with a "schedule a consultation" link next to each one. I'm not sure what "consultation" is if not "a product". But maybe they're all free?
- polotics 1y ago
  My experience is exactly the opposite of "AI reduces cognitive engagement with tasks": I have to constantly be on my toes to follow what the LLMs are proposing and make sure they are not getting off track over-engineering things, or entering something that's likely to turn into a death loop several turns later. AI use definitely makes my brain run warmer, got to get a FLIR camera to prove it I guess...
  - walleeee 1y ago
    So, reduces cognitive engagement with the actual task at hand, and forces a huge attention share to hand-holding. I don't think you two are disagreeing. I have noticed this personally. It's a lot like the fatigue one gets from too long scrolling online. Engagement is shallower but not any less mentally exhausting than reading a book. You end up feeling more exhausted due to the involuntary attention-scattering.
    - notyourwork 1y ago
      It would be analogous to having to double check the IDE added the lines of code I actually typed. That’s not a great productivity boost, it’s a toy still in many ways.
      - deleted 1y ago
        [deleted]
    - deleted 1y ago
      [deleted]
    - JumpCrisscross 1y ago
      > reduces cognitive engagement with the actual task at hand, and forces a huge attention share to hand-holding You're in some senses managing an idiot savant, emphasis on the idiot part, except they're also a narcissist who will happily go out of scope if you let it. If you have management experience, the analogy is immediately obvious. If you don't, I can see how having speed run learning it with a kid running around with dynamite would be taxing.
- dist-epoch 1y ago
  > AI is just another product motion toward comfort maximizing over all things, as cognitive engagement is difficult and not always pleasant. In a nutshell, it is another instant gratification product from tech. For me is the exact opposite. When not using AI, while coding you notice various things that could be improved, you can think about the architecture and what features you want next. But AI codes so fast, that it's a real struggle keeping up to it. I feel like I need to focus 10 times harder to be able to think about features/architecture in a way that AI doesn't wait after me most of the time.
- vel0city 1y ago
  Isn't most technology generally about "product motion toward comfort maximizing over all things"? Isn't a can opener "comfort maximizing"? Bicycles comfort maximize over walking the same distance and leave more time for leisure. We use tractors to till the soil because tilling it by dragging plows by hand or by livestock was far less comfortable and led less time for other pursuits. "AI" might be a good thing or a bad thing especially depending on the context and implementation, but just generally saying something is about maximizing comfort as some inherently bad goal seems off to me.
- Fire-Dragon-DoL 1y ago
  I agree with you but have no metrics yet. It does help as a rubber duck though
- protocolture 1y ago
  >Because AI reduces cognitive engagement with tasks Just a sideways thing. Cognitive offloading is something humans do with each other quite a lot. People offload onto colleagues, underlings and spouses all the time. People engage with AI through the prism of reducing engagement with the thing they dont like, and increasing engagement with the thing they do like. It isnt a straight up productivity boost, but its more like going from being a screenwriter to a director.
- tptacek 1y ago
  It depends a lot on how you use it, and how much effort you put into getting a knack for using it (which is rough because you're always worried that knack might be out of date within a month or two). I use Claude, Codex, and the Gemini CLI (all "supervised command line agents"). I write Go. I am certain that agents improve my productivity in a couple common scenarios: 1. Unsticking myself from stalling at the start of some big "lift" (like starting a new project, adding a major feature that will pull in some new dependency). The LLM can get things very wrong (this happens to me maybe 20% of the time), but that doesn't matter, because for me the motion of taking something wrong and making it "righter" is much less effort than getting past a blank page, assembling all the resources I need to know how to hello-world a new dependency, that kind of thing. Along with "wrestling for weeks with timing-dependent bugs", this kind of inertia is one of like two principal components of "effort" in programming for me, so clearing it is a very big deal. I'm left at a point where I'm actually jazzed about taking the wheel and delivering the functionality myself. 2. Large mechanical changes (like moving an interface from one component to another, or major non-architectural refactors, or instrumentation). Things where there's one meta-solution and it's going to be repeated many times across a codebase. Easy to review, a lot of typing, no more kidding myself for 20 minutes that I can make just the right Emacs macro to pull it off. 3. Bug hunting. To be clear: I am talking here about code I wrote, not the LLM. I run it, it does something stupid. My first instinct now is to drop into Claude or Gemini, paste the logs and an @-reference to a .go file as a starting point, and just say "wtf". My hit rate on it spotting things is very good. If the hit rate was even "meh borderline" that would be a huge win for the amount of effort it takes, but it isn't, it's much better. I'm guessing a lot of people do not have these 3 experiences with LLM agents. I'm sorry! But I do think that if you stipulate that I'm not just making this up, it's hard to go from here to "I'm kidding myself about the value this is providing". Note that these are three cases that look nothing at all like vibe-coding.
  - defatigable 1y ago
    This matches my experience exactly. #3 is the one I've found most surprising, and it can work outside the context of just analyzing your own code. For example I found a case where an automated system we use started failing due to syntax changes, despite no code changes on our part. I gave Claude the error message and the context that we had made no code changes, and it immediately and correctly identified the root cause as a version bump from an unpinned dependency (whoops) that introduced breaking syntax changes. The version bump had happened four hours prior. Could I have found this bug as quickly as Claude? Sure, in retrospect the cause seems quite obvious. But I could just as easily rabbit holed myself looking somewhere else, or taken a while to figure out exactly which dependency caused the issue. It's definitely the case that you cannot blindly accept the LLM's output, you have to treat it as a partner and often guide it towards better solutions. But it absolutely can improve your productivity.
- srcreigh 1y ago
  > We do not provide evidence that: > AI systems do not currently speed up many or most software developers > AI systems in the near future will not speed up developers in our exact setting > There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting
- Art9681 1y ago
  It shifts your cognitive tasks to other things. Like every tool. The tool itself is an abstraction over tedium. We built it for a reason. You will spend less time thinking about some things, and more time thinking about others. In that regard, nothing will change.
- naasking 1y ago
  > and this probably compounds as AI-generated code piles up in a codebase, as there isn't an author who can attach context as to why decisions were made I don't see why. If anything there's more opportunity for documentation because the code was generated from a natural language prompt that documents exactly what was generated, and often why. Recording prompts in source control and linked to the generated code is probably the way to go here.
- zahlman 1y ago
  > increases perceptive productivity increases the perception of productivity?
- enkrs 1y ago
  I use LLMs (like claude-code and codex-cli) the same way accountants use calculators. Without one, you waste all your focus on adding numbers; with one, you just enter values and check if the result makes sense. Programming feels the same—without LLMs, I’m stuck on both big problems (architecture, performance) and small ones (variable names). With LLMs, I type what I want and get code back. I still think about whether it works long-term, but I don’t need to handle every little algorithm detail myself. Of course there are going to be discussions what is real programming (like I'm sure there were discussions what is "real" accounting with the onset of a calculator) The moment we stop treating LLMs like people and see them as big calculators, it all clicks.
  - janalsncm 1y ago
    The issue with your analogy is that calculators do not hallucinate. They do not make mistakes. An accountant is able to fully offload the mental overhead of arithmetic because the calculator is reliable.
    - theshrike79 1y ago
      Replace calculator with the modern equivalent: Excel. It does make mistakes and is not reliable[0]. The user still needs to have a "feel" for the data. (to be pedantic "Excel" doesn't make mistakes, people trusting its defaults do) [0] https://timharford.com/2021/05/cautionary-tales-wrong-tools-cost-lives/ https://timharford.com/2021/05/cautionary-tales-wrong-tools-...
      - Jensson 1y ago
        > (to be pedantic "Excel" doesn't make mistakes, people trusting its defaults do) So what is your point? An expert that mastered excel don't have to check that excel calculated things correctly, he just need to check that he gave excel the right inputs and formulas. That is not true for LLM, you do have to check that it actually did what you asked regardless how good you are at prompting. The only thing I trust an LLM to do correctly are translations, they are very reliable at that, other than that I always verify.
- athrowaway3z 1y ago
  Talking about a specific set-up they use isn't the goal of the post, so I don't think it's a cop out. "How to harness it" is very clearly the difference between users right now, and I'd say we're currently bottom heavy with poor users stuck in a 'productivity illusion'. But there is the question of "what is productivity?" I'm finding myself (having AI) writing better structured docs and tests to make sure the AI can do what I ask it to. I suspect that turns into compounding interests (or lack of technical debt). For an industry where devs have complained, for decades, about productivity metrics being extremely hard or outright bullshit, I see way too many of the same people now waving around studies regarding productivity.
joz1-k 1y ago
From the article: Anthropic has been suffering from pretty terrible reliability problems. In the past, factories used to shut down when there was a shortage of coal for steam engines or when the electricity supply failed. In the future, programmers will have factory holidays when their AI-coding language model is down.
- corentin88 1y ago
  Same as GitHub or Slack downtimes severely impact productivity.
  - thw_9a83c 1y ago
    I would argue that dependency on GitHub and Slack is not the same as dependency on AI coding agents. GitHub/Slack are just straightforward tools. You can run them locally or have similar emergency backup tools ready to run locally. But depending on AI agents is like relying on external brains that have knowledge you suddenly don't have if they disappear. Moreover, how many companies could afford to run these models locally? Some of those models aren't even open.
    - danielbln 1y ago
      There are plenty of open weight agentic coding models out there. Small ones you can run on a Macbook, big heavy ones you can run on some rented cloud instance. Also, if Anthropic is down, there is still Google, OpenAI, Mistral, Deepseek and so on. This seems like not much of an issue, honestly.
      - thw_9a83c 1y ago
        The small ones that you can run on a MacBook are quite useless for programming. Once you have access to a state-of-the-art model, it's difficult to accept any downgrade. That's why I think AI-driven programming will always rely on data centers and the best models. > if Anthropic is down, there is still Google, OpenAI, Mistral, Deepseek and so on No company is going to pay for subscriptions to all of them. Either way, we'll see a new layer of fragility caused by overdependence on AI. Surely, though, we will adapt by learning from every major event related to this.
howmayiannoyyou 1y ago
I expected to see OpenAI, Google, Anthropic, etc. provide desktop applications with integrated local utility models and sandboxed MCP functionality to reduce unnecessary token and task flow, and I still expect this to occur at some point. The biggest long-term risk to the AI giant's profitability will be increasingly capable desktop GPU and CPU capability combined with improving performance by local models.
- mordymoop 1y ago
  From experience it seems like preempting context scoping and routing decisions to smaller models just results in those models making bad judgements at a very high speed. Whenever I experiment with agent frameworks that spawn subagents with scoped subtasks and restricted context, things go off the rails very quickly. A subagent with reduced context makes poorer choices and hallucinates assumptions about the greater codebase, and very often lacks a basic sense the point of the work. This lack of situational awareness is where you are most likely to encounter js scripts suddenly appearing in your Python repo. I don’t know if there is a “fix” for this or if I even want one. Perhaps the solution, in the limit, actually will be to just make the big-smart models faster and faster, so they can chew on the biggest and most comprehensive context possible, and use those exclusively. eta: The big models have gotten better and better at longer-running tasks because they are less likely to make a stupid mistake that derails the work at any given moment. More nines of reliability, etc. By introducing dumber models into this workflow, and restricting the context that you feed to the big models, you are pushing things back in the wrong direction.
- casey2 1y ago
  Yup. I expected a google LLM to coordinate with many local expert LLMs with knowledge of local tools and other domain expert LLMs in the cloud. I they don't see a viable path forward without specialty hardware
mmmllm 1y ago
Speed is not a problem for me. I feel they are at the right speed now where I am able to see what it is doing in real time and check it's on the right track. Honestly if it were any faster I would want a feature to slow it down, as I often intervene if it's going in the wrong direction.
infecto 1y ago
Cursor imo is still one of the only real players in the space. I don’t like the claude code style of coding, I feel too disconnected. Cursor is the right balance for me and it is generally pretty darn quick and I only expect it to get quicker. I hope there are more players that pop up in this space.
- sealeck 1y ago
  Have you tried https://zed.dev https://zed.dev ?
  - dmix 1y ago
    How is the pricing? I see it says "500 prompts a month" and only Claude. Cursor is built around token usage and distributes them across multiple models when you hit limits on one which turns out to be pretty economical.
    - 85392_school 1y ago
      You don't have to use the built in subscription. They have support for curated APIs (OpenAI, Anthropic, GitHub Copilot, OpenRouter, etc), any OpenAI-compatible API, and agents like Gemini CLI and Claude Code in the AI panel.
    - ethmarks 1y ago
      Zed supports BYOK so you can connect it to GitHub Copilot or Anthropic or OpenRouter or whatever. The 500-per-month limit is only for Zed-hosted prompts. I personally switch between Zed-hosted, GH Copilot, Gemini API, and Ollama. Zed's AI integration isn't quite as "it just works" as Cursor's, but it's still very good and it gives you much more freedom.
  - infecto 1y ago
    Yea and for some reason it was not my cup of tea. I think partly due to their paid version feels like an afterthought.
- CuriouslyC 1y ago
everyone 1y ago
Ive used chatGPT to help me learn new stuff about which I know nothing (this is it's best use imo) and also write boilerplatey functions, eg. write a function that does precisely X. Having it integrated into my IDE sounds like a nightmare though. Even the "intellisense" stuff in visual studio is annoying af and I have to turn it off to stop it auto-wrecking my code (eg. adding tonnes of pointless using statement). I dont know how the integrated llm would actually work, but I defo dont want that.
- vrighter 1y ago
  Writing the mindless boilerplate is when I'm thinking about the next step. If I didn't write it, i'd still have to take the time to think things through for the next step.
- arealaccount 1y ago
  Eventually you start to anticipate what it will output and you can get ahead of it tabbing through tons of code that you were intending to write. But when it doesn’t output what you want you spend mental energy and extra time reorienting to get back on track. About 50% of the time it works every time.
- theshrike79 1y ago
  > "An LLM agent runs tools in a loop to achieve a goal." This is the magic sauce compared to copy-pasting snippets to a web browser. It can automatically read code, search through it and modify multiple files automatically. Then it can compile it, run tests, run the actual application to check it works etc.
curtisszmania 1y ago
[dead]
dmix 1y ago
I looked up the graph they are using https://openrouter.ai/rankings https://openrouter.ai/rankings It says "Grok Code Fast 1" is ranked first in token usage? That's surprising. Maybe it's just OpenRouter bias or how the LLM is used. I would have assumed Claude would be #1
- Dlanv 1y ago
  Everyone I know who uses Claude does not use it through openrouter.
- CuriouslyC 1y ago
  Grok code fast is a legit good model. https://www.youtube.com/watch?v=Y-SyfYXupTQ https://www.youtube.com/watch?v=Y-SyfYXupTQ
  - dmix 1y ago
    Grok 4 was really good in my experience, but it was really slow. Might try the fast version if my Claude runs out of tokens. I stick to Claude because I know the models output patterns and flaws (which are more predictable than GPT-5).
    - theshrike79 1y ago
      Just yesterday I had Cline+Grok Code Fast fix an issue caused by Claude ... who ran out of credits mid-fix. (LLMs seem to think Go embeds can use ../../style relative paths, they cannot. And when they notice it doesn't work like that, they use the weirdest shit to try to fix it.)
- Bolwin 1y ago
  Xai has been offering it for free. Loom at the #1 user. It's kilo code, they've been giving grok code for free for weeks.
  - theshrike79 1y ago
    Cline too. It was free for a week(end?) and they extended it for a longer time. It's pretty good tbh. Some quirks but it's efficient at making changes and seems to understand code pretty well.
bryanlarsen 1y ago
Multiple contexts is hard, and often counter-productive. It used to be popular on HN to talk about keeping your "flow", and railing against everything that broke a programmer's flow. These slow AI's constantly break flow.
- solumunus 1y ago
  The new flow is cycling between 5 concurrent agent sessions while watching YouTube.
  - handfuloflight 1y ago
    as they say, "few"
  - theshrike79 1y ago
    And my ADHD brain loves it :D Except no YouTube, I just watch my shows off Plex.
    - joquarky 1y ago
      This is why one-size-fits-all doesn't work if a company legitimately cares about productivity.
- hamdingers 1y ago
  The ideal for me as a flow-seeker is the quick-edit feature of Void and presumably other editors. It saves me the context switching to google or docs to find the right syntax and method names for what I want to express, without requiring I waste my time waiting for the LLM to figure out what I already know (the what and the where).
  - bryanlarsen 1y ago
    The auto complete provided by your lsp server to your editor does a much better job of this than LLM's do, in my opinion.
arisAlexis 1y ago
Cerebras with Qwen and Mistral with Cerebras already feel like magic
cadamsdotcom 1y ago
There’s a quality piece too - I don’t mind dialup speeds of tokens per second if the quality is high enough to avoid rework. If you want better speeds, your coding agent might perform better outside US office hours :)
ryanmcgarvey 1y ago
The only reason I want these things to be any smarter is because I need them to do more work over longer periods screwing up. The only reason I need them to do more work over long periods is because they are too slow to properly pair with. If I could have it read more of my project in a single gulp and produce the 10-1000 lines of code I want in a few seconds, I wouldn't need it to go off and write the thousands of lines on its own in the background. But because even trivial changes can take minutes by the time it slurps up the right context and futzes with the linter and types, that ideal pair programmer loop is less attractive.
MangoToupe 1y ago
I'd take quality improvements over speed any day. AI gets you 80% of the way there, but you often spend more time fixing the last 20% than you would have approaching the problem yourself.
stuaxo 1y ago
This completely ignores energy usage of "infinite" token usage. These things are somewhat useful, but it doesn't come for free, and I'm talking externalities here, which are not priced in.
brador 1y ago
Imagine when we measure in Mtoks/s. That with guardrails off will be insane.
oceanparkway 1y ago
This article mentions Cerebras, I tried it out and was 1) disappointed and 2) they started a subscription but gave me no way of cancelling, their billing page is broken :/
mark_l_watson 1y ago
I rate the value of LLM based AI on how it improves me personally. Positive experiences include being able to more easily. read scientific papers with an AI filling in details for embedded math, and after a vibe coding session when I feel like I understand the problem better because I was engaged with the process and I understand the resulting code. Negative experiences include times when I am lazy, turn off my brain and just accept LLM output. I am also a little skeptical about automating email handling, etc. that is cool technology but how useful is it, really? I can imagine insiders talking between themselves saying "feel the bubble!" but when talking with reporters they talk like "feel the AGI" or "oh no, our AI tech is so strong it will take over the world" - excellent strategies for pumping stock prices and valuations.
simne 1y ago
Well, I seen one suggestion for probable nearest future of AI, to stop on GPT-4 level, but distill and use optimizations like switch to FP8 for faster speed. So basically idea, model with very same capabilities, but distilled and optimized to have less size and faster inference. I cannot promise 100x speedup, but I think 10x is very real.

137 points by martinald 1y ago

> Each of these 'phases' of LLM growth is unlocking a lot more developer productivity, for teams and developers that know how to harness it. I still find myself incredibly skeptical LLM use is increasing productivity. Because AI reduces cognitive engagement with tasks, it feels to me like AI increases perceptive productivity but actually decreases it in many cases (and this probably compounds as AI-generated code piles up in a codebase, as there isn't an author who can attach context as to why decisions were made). https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... I realize the author qualified his or her statement with "know how to harness it," which feels like a cop-out I'm seeing an awful lot in recent explorations of AI's relationship with productivity. In my mind, like TikTok or online dating, AI is just another product motion toward comfort maximizing over all things, as cognitive engagement is difficult and not always pleasant. In a nutshell, it is another instant gratification product from tech. That's not to say that I don't use AI, but I use it primarily as search to see what is out there. If I use it for coding at all, I tend to primarily use it for code review. Even when AI does a good job at implementation of a feature, unless I put in the cognitive engagement I typically put in during code review, its code feels alien to me and I feel uncomfortable merging it (and I employ similar levels of cognitive engagement during code reviews as I do while writing software).

breakfastduck 1y ago
It depends what environment you're operating within. I've used LLMs for code gen at work as well as for personal stuff. At work primarily for quick and dirty internal UIs / tools / CLIs it's been fantastic, but we've not unleashed it on our core codebases. It's worth noting all the stuff we've got out of out are things we'd not normally have the time to work on - so a net positive there. Outside of work I've built some bespoke software almost entirely generated with human tweaks here and there - again, super useful software for me and some friends to use for planning and managing music events we put on that I'd never normally have the time to build. So in those ways I see it as massively increasing productivity - to build lower stakes things that would normally just never get done due to lack of time.
- phil21 1y ago
  I do wonder about the second order effects of the second bit. A lot of open source tooling gets created to solve those random "silly" things that are personal annoyances or needs. Then you find out others have the same or similar problem and entire standard tooling or libraries come into existence. I have pontificated on how easy access to immediate "one offs" will kill this idea exchange? Instead of one tool maintained by hundreds to fulfill a common need, we will end up with millions of one-off LLM generated that are not shared with anyone else. Might be a net win, or a net loss. I'm really not sure!
add-sub-mul-div 1y ago
> I realize the author qualified his or her statement with "know how to harness it," which feels like a cop-out I'm seeing an awful lot in recent explorations of AI's relationship with productivity. "You're doing AI wrong" is the new "you're doing agile wrong" which was the new "you're doing XP wrong".
- bitwize 1y ago
  More like the new "you're holding it wrong"
- pjmlp 1y ago
  Unfortunely many of us are old enough to know how those wrong eventually became the new normal, the wrong way.
- CuriouslyC 1y ago
  At this point I don't even care to argue with people who disagree with me on this. History will decide the winner and if you think you're not on the losing side, best of luck.
  - jamesnorden 1y ago
    I'd say you're a bit biased.
    - CuriouslyC 1y ago
      I am biased, but by my personal experience, not the desire to sell anything to anyone. I don't even have products for sale right now, I'm just building free things to try and help people figure out how to ride this wave safely.
      - svieira 1y ago
        > I don't even have products for sale right now The link in your bio is a long series of tools for various personas with a "schedule a consultation" link next to each one. I'm not sure what "consultation" is if not "a product". But maybe they're all free?
polotics 1y ago
My experience is exactly the opposite of "AI reduces cognitive engagement with tasks": I have to constantly be on my toes to follow what the LLMs are proposing and make sure they are not getting off track over-engineering things, or entering something that's likely to turn into a death loop several turns later. AI use definitely makes my brain run warmer, got to get a FLIR camera to prove it I guess...
- walleeee 1y ago
  So, reduces cognitive engagement with the actual task at hand, and forces a huge attention share to hand-holding. I don't think you two are disagreeing. I have noticed this personally. It's a lot like the fatigue one gets from too long scrolling online. Engagement is shallower but not any less mentally exhausting than reading a book. You end up feeling more exhausted due to the involuntary attention-scattering.
  - notyourwork 1y ago
    It would be analogous to having to double check the IDE added the lines of code I actually typed. That’s not a great productivity boost, it’s a toy still in many ways.
    - deleted 1y ago
      [deleted]
  - deleted 1y ago
    [deleted]
  - JumpCrisscross 1y ago
    > reduces cognitive engagement with the actual task at hand, and forces a huge attention share to hand-holding You're in some senses managing an idiot savant, emphasis on the idiot part, except they're also a narcissist who will happily go out of scope if you let it. If you have management experience, the analogy is immediately obvious. If you don't, I can see how having speed run learning it with a kid running around with dynamite would be taxing.
dist-epoch 1y ago
> AI is just another product motion toward comfort maximizing over all things, as cognitive engagement is difficult and not always pleasant. In a nutshell, it is another instant gratification product from tech. For me is the exact opposite. When not using AI, while coding you notice various things that could be improved, you can think about the architecture and what features you want next. But AI codes so fast, that it's a real struggle keeping up to it. I feel like I need to focus 10 times harder to be able to think about features/architecture in a way that AI doesn't wait after me most of the time.
vel0city 1y ago
Isn't most technology generally about "product motion toward comfort maximizing over all things"? Isn't a can opener "comfort maximizing"? Bicycles comfort maximize over walking the same distance and leave more time for leisure. We use tractors to till the soil because tilling it by dragging plows by hand or by livestock was far less comfortable and led less time for other pursuits. "AI" might be a good thing or a bad thing especially depending on the context and implementation, but just generally saying something is about maximizing comfort as some inherently bad goal seems off to me.
Fire-Dragon-DoL 1y ago
I agree with you but have no metrics yet. It does help as a rubber duck though
protocolture 1y ago
>Because AI reduces cognitive engagement with tasks Just a sideways thing. Cognitive offloading is something humans do with each other quite a lot. People offload onto colleagues, underlings and spouses all the time. People engage with AI through the prism of reducing engagement with the thing they dont like, and increasing engagement with the thing they do like. It isnt a straight up productivity boost, but its more like going from being a screenwriter to a director.
tptacek 1y ago
It depends a lot on how you use it, and how much effort you put into getting a knack for using it (which is rough because you're always worried that knack might be out of date within a month or two). I use Claude, Codex, and the Gemini CLI (all "supervised command line agents"). I write Go. I am certain that agents improve my productivity in a couple common scenarios: 1. Unsticking myself from stalling at the start of some big "lift" (like starting a new project, adding a major feature that will pull in some new dependency). The LLM can get things very wrong (this happens to me maybe 20% of the time), but that doesn't matter, because for me the motion of taking something wrong and making it "righter" is much less effort than getting past a blank page, assembling all the resources I need to know how to hello-world a new dependency, that kind of thing. Along with "wrestling for weeks with timing-dependent bugs", this kind of inertia is one of like two principal components of "effort" in programming for me, so clearing it is a very big deal. I'm left at a point where I'm actually jazzed about taking the wheel and delivering the functionality myself. 2. Large mechanical changes (like moving an interface from one component to another, or major non-architectural refactors, or instrumentation). Things where there's one meta-solution and it's going to be repeated many times across a codebase. Easy to review, a lot of typing, no more kidding myself for 20 minutes that I can make just the right Emacs macro to pull it off. 3. Bug hunting. To be clear: I am talking here about code I wrote, not the LLM. I run it, it does something stupid. My first instinct now is to drop into Claude or Gemini, paste the logs and an @-reference to a .go file as a starting point, and just say "wtf". My hit rate on it spotting things is very good. If the hit rate was even "meh borderline" that would be a huge win for the amount of effort it takes, but it isn't, it's much better. I'm guessing a lot of people do not have these 3 experiences with LLM agents. I'm sorry! But I do think that if you stipulate that I'm not just making this up, it's hard to go from here to "I'm kidding myself about the value this is providing". Note that these are three cases that look nothing at all like vibe-coding.
- defatigable 1y ago
  This matches my experience exactly. #3 is the one I've found most surprising, and it can work outside the context of just analyzing your own code. For example I found a case where an automated system we use started failing due to syntax changes, despite no code changes on our part. I gave Claude the error message and the context that we had made no code changes, and it immediately and correctly identified the root cause as a version bump from an unpinned dependency (whoops) that introduced breaking syntax changes. The version bump had happened four hours prior. Could I have found this bug as quickly as Claude? Sure, in retrospect the cause seems quite obvious. But I could just as easily rabbit holed myself looking somewhere else, or taken a while to figure out exactly which dependency caused the issue. It's definitely the case that you cannot blindly accept the LLM's output, you have to treat it as a partner and often guide it towards better solutions. But it absolutely can improve your productivity.
srcreigh 1y ago
> We do not provide evidence that: > AI systems do not currently speed up many or most software developers > AI systems in the near future will not speed up developers in our exact setting > There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting
Art9681 1y ago
It shifts your cognitive tasks to other things. Like every tool. The tool itself is an abstraction over tedium. We built it for a reason. You will spend less time thinking about some things, and more time thinking about others. In that regard, nothing will change.
naasking 1y ago
> and this probably compounds as AI-generated code piles up in a codebase, as there isn't an author who can attach context as to why decisions were made I don't see why. If anything there's more opportunity for documentation because the code was generated from a natural language prompt that documents exactly what was generated, and often why. Recording prompts in source control and linked to the generated code is probably the way to go here.
zahlman 1y ago
> increases perceptive productivity increases the perception of productivity?
enkrs 1y ago
I use LLMs (like claude-code and codex-cli) the same way accountants use calculators. Without one, you waste all your focus on adding numbers; with one, you just enter values and check if the result makes sense. Programming feels the same—without LLMs, I’m stuck on both big problems (architecture, performance) and small ones (variable names). With LLMs, I type what I want and get code back. I still think about whether it works long-term, but I don’t need to handle every little algorithm detail myself. Of course there are going to be discussions what is real programming (like I'm sure there were discussions what is "real" accounting with the onset of a calculator) The moment we stop treating LLMs like people and see them as big calculators, it all clicks.
- janalsncm 1y ago
  The issue with your analogy is that calculators do not hallucinate. They do not make mistakes. An accountant is able to fully offload the mental overhead of arithmetic because the calculator is reliable.
  - theshrike79 1y ago
    Replace calculator with the modern equivalent: Excel. It does make mistakes and is not reliable[0]. The user still needs to have a "feel" for the data. (to be pedantic "Excel" doesn't make mistakes, people trusting its defaults do) [0] https://timharford.com/2021/05/cautionary-tales-wrong-tools-cost-lives/ https://timharford.com/2021/05/cautionary-tales-wrong-tools-...
    - Jensson 1y ago
      > (to be pedantic "Excel" doesn't make mistakes, people trusting its defaults do) So what is your point? An expert that mastered excel don't have to check that excel calculated things correctly, he just need to check that he gave excel the right inputs and formulas. That is not true for LLM, you do have to check that it actually did what you asked regardless how good you are at prompting. The only thing I trust an LLM to do correctly are translations, they are very reliable at that, other than that I always verify.
athrowaway3z 1y ago
Talking about a specific set-up they use isn't the goal of the post, so I don't think it's a cop out. "How to harness it" is very clearly the difference between users right now, and I'd say we're currently bottom heavy with poor users stuck in a 'productivity illusion'. But there is the question of "what is productivity?" I'm finding myself (having AI) writing better structured docs and tests to make sure the AI can do what I ask it to. I suspect that turns into compounding interests (or lack of technical debt). For an industry where devs have complained, for decades, about productivity metrics being extremely hard or outright bullshit, I see way too many of the same people now waving around studies regarding productivity.

joz1-k 1y ago

From the article: Anthropic has been suffering from pretty terrible reliability problems. In the past, factories used to shut down when there was a shortage of coal for steam engines or when the electricity supply failed. In the future, programmers will have factory holidays when their AI-coding language model is down.

corentin88 1y ago
Same as GitHub or Slack downtimes severely impact productivity.
- thw_9a83c 1y ago
  I would argue that dependency on GitHub and Slack is not the same as dependency on AI coding agents. GitHub/Slack are just straightforward tools. You can run them locally or have similar emergency backup tools ready to run locally. But depending on AI agents is like relying on external brains that have knowledge you suddenly don't have if they disappear. Moreover, how many companies could afford to run these models locally? Some of those models aren't even open.
  - danielbln 1y ago
    There are plenty of open weight agentic coding models out there. Small ones you can run on a Macbook, big heavy ones you can run on some rented cloud instance. Also, if Anthropic is down, there is still Google, OpenAI, Mistral, Deepseek and so on. This seems like not much of an issue, honestly.
    - thw_9a83c 1y ago
      The small ones that you can run on a MacBook are quite useless for programming. Once you have access to a state-of-the-art model, it's difficult to accept any downgrade. That's why I think AI-driven programming will always rely on data centers and the best models. > if Anthropic is down, there is still Google, OpenAI, Mistral, Deepseek and so on No company is going to pay for subscriptions to all of them. Either way, we'll see a new layer of fragility caused by overdependence on AI. Surely, though, we will adapt by learning from every major event related to this.

howmayiannoyyou 1y ago

I expected to see OpenAI, Google, Anthropic, etc. provide desktop applications with integrated local utility models and sandboxed MCP functionality to reduce unnecessary token and task flow, and I still expect this to occur at some point. The biggest long-term risk to the AI giant's profitability will be increasingly capable desktop GPU and CPU capability combined with improving performance by local models.

mordymoop 1y ago
From experience it seems like preempting context scoping and routing decisions to smaller models just results in those models making bad judgements at a very high speed. Whenever I experiment with agent frameworks that spawn subagents with scoped subtasks and restricted context, things go off the rails very quickly. A subagent with reduced context makes poorer choices and hallucinates assumptions about the greater codebase, and very often lacks a basic sense the point of the work. This lack of situational awareness is where you are most likely to encounter js scripts suddenly appearing in your Python repo. I don’t know if there is a “fix” for this or if I even want one. Perhaps the solution, in the limit, actually will be to just make the big-smart models faster and faster, so they can chew on the biggest and most comprehensive context possible, and use those exclusively. eta: The big models have gotten better and better at longer-running tasks because they are less likely to make a stupid mistake that derails the work at any given moment. More nines of reliability, etc. By introducing dumber models into this workflow, and restricting the context that you feed to the big models, you are pushing things back in the wrong direction.
casey2 1y ago
Yup. I expected a google LLM to coordinate with many local expert LLMs with knowledge of local tools and other domain expert LLMs in the cloud. I they don't see a viable path forward without specialty hardware

mmmllm 1y ago

Speed is not a problem for me. I feel they are at the right speed now where I am able to see what it is doing in real time and check it's on the right track. Honestly if it were any faster I would want a feature to slow it down, as I often intervene if it's going in the wrong direction.

infecto 1y ago

Cursor imo is still one of the only real players in the space. I don’t like the claude code style of coding, I feel too disconnected. Cursor is the right balance for me and it is generally pretty darn quick and I only expect it to get quicker. I hope there are more players that pop up in this space.

sealeck 1y ago
Have you tried https://zed.dev https://zed.dev ?
- dmix 1y ago
  How is the pricing? I see it says "500 prompts a month" and only Claude. Cursor is built around token usage and distributes them across multiple models when you hit limits on one which turns out to be pretty economical.
  - 85392_school 1y ago
    You don't have to use the built in subscription. They have support for curated APIs (OpenAI, Anthropic, GitHub Copilot, OpenRouter, etc), any OpenAI-compatible API, and agents like Gemini CLI and Claude Code in the AI panel.
  - ethmarks 1y ago
    Zed supports BYOK so you can connect it to GitHub Copilot or Anthropic or OpenRouter or whatever. The 500-per-month limit is only for Zed-hosted prompts. I personally switch between Zed-hosted, GH Copilot, Gemini API, and Ollama. Zed's AI integration isn't quite as "it just works" as Cursor's, but it's still very good and it gives you much more freedom.
- infecto 1y ago
  Yea and for some reason it was not my cup of tea. I think partly due to their paid version feels like an afterthought.
CuriouslyC 1y ago

everyone 1y ago

Ive used chatGPT to help me learn new stuff about which I know nothing (this is it's best use imo) and also write boilerplatey functions, eg. write a function that does precisely X. Having it integrated into my IDE sounds like a nightmare though. Even the "intellisense" stuff in visual studio is annoying af and I have to turn it off to stop it auto-wrecking my code (eg. adding tonnes of pointless using statement). I dont know how the integrated llm would actually work, but I defo dont want that.

vrighter 1y ago
Writing the mindless boilerplate is when I'm thinking about the next step. If I didn't write it, i'd still have to take the time to think things through for the next step.
arealaccount 1y ago
Eventually you start to anticipate what it will output and you can get ahead of it tabbing through tons of code that you were intending to write. But when it doesn’t output what you want you spend mental energy and extra time reorienting to get back on track. About 50% of the time it works every time.
theshrike79 1y ago
> "An LLM agent runs tools in a loop to achieve a goal." This is the magic sauce compared to copy-pasting snippets to a web browser. It can automatically read code, search through it and modify multiple files automatically. Then it can compile it, run tests, run the actual application to check it works etc.

curtisszmania 1y ago

[dead]

dmix 1y ago

I looked up the graph they are using https://openrouter.ai/rankings https://openrouter.ai/rankings It says "Grok Code Fast 1" is ranked first in token usage? That's surprising. Maybe it's just OpenRouter bias or how the LLM is used. I would have assumed Claude would be #1

Dlanv 1y ago
Everyone I know who uses Claude does not use it through openrouter.
CuriouslyC 1y ago
Grok code fast is a legit good model. https://www.youtube.com/watch?v=Y-SyfYXupTQ https://www.youtube.com/watch?v=Y-SyfYXupTQ
- dmix 1y ago
  Grok 4 was really good in my experience, but it was really slow. Might try the fast version if my Claude runs out of tokens. I stick to Claude because I know the models output patterns and flaws (which are more predictable than GPT-5).
  - theshrike79 1y ago
    Just yesterday I had Cline+Grok Code Fast fix an issue caused by Claude ... who ran out of credits mid-fix. (LLMs seem to think Go embeds can use ../../style relative paths, they cannot. And when they notice it doesn't work like that, they use the weirdest shit to try to fix it.)
Bolwin 1y ago
Xai has been offering it for free. Loom at the #1 user. It's kilo code, they've been giving grok code for free for weeks.
- theshrike79 1y ago
  Cline too. It was free for a week(end?) and they extended it for a longer time. It's pretty good tbh. Some quirks but it's efficient at making changes and seems to understand code pretty well.

bryanlarsen 1y ago

Multiple contexts is hard, and often counter-productive. It used to be popular on HN to talk about keeping your "flow", and railing against everything that broke a programmer's flow. These slow AI's constantly break flow.

solumunus 1y ago
The new flow is cycling between 5 concurrent agent sessions while watching YouTube.
- handfuloflight 1y ago
  as they say, "few"
- theshrike79 1y ago
  And my ADHD brain loves it :D Except no YouTube, I just watch my shows off Plex.
  - joquarky 1y ago
    This is why one-size-fits-all doesn't work if a company legitimately cares about productivity.
hamdingers 1y ago
The ideal for me as a flow-seeker is the quick-edit feature of Void and presumably other editors. It saves me the context switching to google or docs to find the right syntax and method names for what I want to express, without requiring I waste my time waiting for the LLM to figure out what I already know (the what and the where).
- bryanlarsen 1y ago
  The auto complete provided by your lsp server to your editor does a much better job of this than LLM's do, in my opinion.

arisAlexis 1y ago

Cerebras with Qwen and Mistral with Cerebras already feel like magic

cadamsdotcom 1y ago

There’s a quality piece too - I don’t mind dialup speeds of tokens per second if the quality is high enough to avoid rework. If you want better speeds, your coding agent might perform better outside US office hours :)

ryanmcgarvey 1y ago

The only reason I want these things to be any smarter is because I need them to do more work over longer periods screwing up. The only reason I need them to do more work over long periods is because they are too slow to properly pair with. If I could have it read more of my project in a single gulp and produce the 10-1000 lines of code I want in a few seconds, I wouldn't need it to go off and write the thousands of lines on its own in the background. But because even trivial changes can take minutes by the time it slurps up the right context and futzes with the linter and types, that ideal pair programmer loop is less attractive.

MangoToupe 1y ago

I'd take quality improvements over speed any day. AI gets you 80% of the way there, but you often spend more time fixing the last 20% than you would have approaching the problem yourself.

stuaxo 1y ago

This completely ignores energy usage of "infinite" token usage. These things are somewhat useful, but it doesn't come for free, and I'm talking externalities here, which are not priced in.

brador 1y ago

Imagine when we measure in Mtoks/s. That with guardrails off will be insane.

oceanparkway 1y ago

This article mentions Cerebras, I tried it out and was 1) disappointed and 2) they started a subscription but gave me no way of cancelling, their billing page is broken :/

mark_l_watson 1y ago

I rate the value of LLM based AI on how it improves me personally. Positive experiences include being able to more easily. read scientific papers with an AI filling in details for embedded math, and after a vibe coding session when I feel like I understand the problem better because I was engaged with the process and I understand the resulting code. Negative experiences include times when I am lazy, turn off my brain and just accept LLM output. I am also a little skeptical about automating email handling, etc. that is cool technology but how useful is it, really? I can imagine insiders talking between themselves saying "feel the bubble!" but when talking with reporters they talk like "feel the AGI" or "oh no, our AI tech is so strong it will take over the world" - excellent strategies for pumping stock prices and valuations.

simne 1y ago

Well, I seen one suggestion for probable nearest future of AI, to stop on GPT-4 level, but distill and use optimizations like switch to FP8 for faster speed. So basically idea, model with very same capabilities, but distilled and optimized to have less size and faster inference. I cannot promise 100x speedup, but I think 10x is very real.