8 ms·

People who are saying they're not seeing productivity boost, can you please share where is it failing? Because, I am terrified by the output I am getting while

• boxedemp 7mo ago
  I'm with you. The project I'm working on is moving at phenomenal velocity. I'm basically spending my time writing specs and performing code reviews. As long as my code review comments and design docs are clear I get a secure, scalable, and resilient system. Tests were always important, but now they are the gatekeepers to velocity.
• truetraveller 7mo ago
  You were probably deficient in RESEARCH skills before. No offense to you, since I was also like this once. LLMs research and put the results on the plate. Yes, for people who were deficient in research skills, I can see 2-3x improvements. Note1: I have "expert" level research skills. But LLMs still help me in research, but the boost is probably 1.2x max. But Note2: By research, I mean googling, github search, forum search, etc. And quickly testing using jsfiddle/codepen, etc.
  • throwaw12 7mo ago
    no worries, I do not get offended quickly. But I also think you are overestimating your RESEARCH skills, even if you are very good at research, I am sure you can't read 25 files in parallel, summarize them (even if its missing some details) in 1 minute and then come up with somewhat working solution in the next 2 minutes. I am pretty sure, humans can't comprehend reading 25 code files with each having at least 400 lines of non-boilerplate code in 2 minutes. LLM can do it and its very very good at summarizing. I can even steer its summarizing skills by prompting where to focus on when its reading files (because now I can iterate 2-3 times for each RESEARCH task and improve my next attempt based on shortcomings in the previous attempt)
    • truetraveller 7mo ago
      OK, it's not just RESEARCH, but "RESEARCHability" of the source content [in this case code], and also critical analysis ability [not saying you are deficient in anything, speaking in general terms]. In this example, if the 25 files are organized nicely, and I had I nice IDE that listed class/namespace members of each file neatly, I might take 30 minutes to understand the overall structure. Morever, If I critically analyzed this, I would ask "how many times does this event of summarizing 25 files happen"? I mean, are we changing codebases every day? No, it's a one time cost. Moreover, manually going through will provide insight not returned by LLM. Obviously, every case is different, and perhaps you do need to RESEARCH new codebases often, I dunno!
  • throwaw12 7mo ago
    please don't change your comment constantly (or at maybe add UPDATE 1/2/3), because you had different words before, like you were saying something in this fashion: * you probably lack good RESEARCH skills * I can see at most 1.25x improvements - now it is 2-3x By updating your comment you are making my reply irrelevant to your past response
    • truetraveller 7mo ago
      Apologies, I changed this within a ~10 minute period. Never knew you would actually see it so fast.
  • siva7 7mo ago
    Ok Mr. Expert Level Researcher, go back and read the comment of parent again to find out that it has nothing to do with deficiency in research skills.
    • truetraveller 7mo ago
      Lol! Didn't mean any harm, just giving my 2cents!
• therealdrag0 7mo ago
  I can only explain it by people not having used Agentic tools and or only having tried it 9 months ago for a day before giving up or having such strict coding style preferences they burn time adjusting generated code to their preferences and blaming the AI even though they’re non-functional changes and they didn’t bother to encode them into rules. The productivity gains are blatantly obvious at this point. Even in large distributed code bases. From jr to senior engineer.
  • MattGaiser 7mo ago
    I can see someone who is very particular about their way being the right way having issues with it. I’m very much the kind of person who believes that if I can’t write a failing test, I don’t have a very serious case. A lot of devs aren’t like that.
    • layer8 7mo ago
      Sometimes you’re unable to write a failing test because the code is such that you can’t reliably reason about it, and hence have a hard time finding the cases where it will do the wrong thing. Being able to reason about code in that way is as important as for the code to be testable.
• aurareturn 7mo ago
  People who are saying they're not seeing productivity boost, can you please share where is it failing? Believe it or not, I still know many devs who do not use any agents. They're still using free ChatGPT copy and paste. I'm going to guess that many people on HN are also on the "free ChatGPT isn't that good at programming" train.
  • dataflow 7mo ago
    Which one would you recommend as the best right now? Claude Code?
    • redhed 7mo ago
      I have been having a lot of success with Cursor. I like being able to switch between Anthropic and OpenAI models. Claude Code does gives way more tokens/$ than Cursor right now though.
  • throwaw12 7mo ago
    > They're still using free ChatGPT copy and paste Probably that's the reason why some people are sure their job is still safe. Nature of job is changing rapidly
    • aurareturn 7mo ago
      I totally get tech CEOs who threaten to fire their devs who do not embrace AI tools. I'm not a tech CEO but people who are anti-LLM for programming have no place on my team.
      • salawat 7mo ago
        And you are paying for their tokens on top of their salary, right? Right?
        • aurareturn 7mo ago
          You can do a lot with just $20 Codex CLI subscription. Tokens are cheap compared to the $20k we're paying for a dev each month.
• kodablah 7mo ago
  > People who are saying they're not seeing productivity boost, can you please share where is it failing? At review time. There are simply too many software industries that can't delegate both authorship _and_ review to non-humans because the maintenance/use of such software, especially in libraries and backwards-compat-concerning environments, cannot justify an "ends justifies the means" approach (yet).
• fulafel 7mo ago
  A terminology tangent because it's an econ publication: Notice that the article doesn't talk about productivity. Productivity is a term of art in economics and means you generate more units of output (for example per person, per input, per wages paid) but doesn't take quality or otherwise desireability into account. It's best suited for commodities and industrial outputs (and maybe slop?).
• dataflow 7mo ago
  I feel like this might be heavily dependent on both your task and the AI you're using? What language do you code in and what AI do you use? And are your tasks pretty typical/boilerplate-y with prior art to go off of, or novel/at-the-edge-of-tech?
• yoyohello13 7mo ago
  The companies laying off people have no vision. My company is a successful not for profit and we are hiring like crazy. It’s not a software company, but we have always effectively unlimited work. Why would anyone downsize because work is getting done faster? Just do more work, get more done, get better than the competition, get better at delivering your vision. We put profits back in the community and actually make life better for people. What a crazy fucking concept right?
  • threatofrain 7mo ago
    These are words without weights. At some point the put money into software option will max out. Perhaps what we should all be doing is hiring more lawyers, there's always more legal work to be done. When you don't have weights then you can reason like this.
    • yoyohello13 7mo ago
      I don’t know what kind of software your used to but software is pretty much universally dog shit these days. I could probably count on one hand the number of programs that I actually like using. There is an astronomical room for improvement. I don’t think we are hitting diminishing returns any time soon.
      • laurentiurad 7mo ago
        I talk about this at length in one of my previous posts here: https://news.ycombinator.com/item?id=39963058 https://news.ycombinator.com/item?id=39963058. I definitely share your opinion and I think this will be exacerbated by vibe coding and having LoC as the main KPI for engineering teams.
  • throwaw12 7mo ago
    > Just do more work, get more done That's one of the reasons why I am terrified, because it can lead to burn out, and I personally don't like to babysit bunch of agents, because the output doesn't feel "mine", when its not "mine" I don't feel ownership. And I am deliberately hitting the brake from time to time not to increase expectations, because I feel like driving someone else's car while not understanding fully how they tuned their car (even though I did those tunings by prompting)
• pinkmuffinere 7mo ago
  I asked opus 4.6 how to administer an A/B test when data is sparse. My options are to look at conversion rate, look at revenue per customer, or something else. I will get about 10-20k samples, less than that will add to cart, less than that will begin checkout, and even less than that will convert. Opus says I should look at revenue per customers. I don't know the right answer, but I know it is not to look at revenue per customers -- that will have high variance due to outlier customers who put in a large order. To be fair, I do use opus frequently, and it often gives good enough answers. But you do have to be suspicious of its responses for important decisions. Edit: Ha, and the report claims it's relatively good at business and finance... Edit 2: After discussion in this thread, I went back to opus and asked it to link to articles about how to handle non-normally distributed data, and it actually did link to some useful articles, and an online calculator that I believe works for my data. So I'll eat some humble pie and say my initial take was at least partially wrong here. At the same time, it was important to know the correct question to ask, and honestly if it wasn't for this thread I'm not sure I would have gotten there.
  • onion2k 7mo ago
    A/B tests are a statistical tool, and outliers will mess with any statistical measure. If your data is especially prone to that you should be using something that accounts for them, and your prompt to Opus should tell it to account for that. A good way to use AI is to treat it like a brilliant junior. It knows a lot about how things work in general but very little about your specific domain. If your data has a particular shape (e.g lots of orders with a few large orders as outliers) you have to tell it that to improve the results you get back.
    • pinkmuffinere 7mo ago
      I did tell it that I expect to see something like a power-law distribution in order value, so I think I pretty much followed your instructions here. Btw, if you do know the right thing to do in my scenario, I'd love to figure it out. This is not my area of expertise, and just figuring it out through articles so far.
• apsurd 7mo ago
  AI dramatically increases velocity. But is velocity productivity? Productivity relative to which scope: you, the team, the department, the company? The question is really, velocity _of what_? I got this from a HN comment. It really hit for me because the default mentality for engineers is to build. The more you build the better. That's not "wrong" but in a business setting it is very much necessary but not sufficient. And so whenever we think about productivity, impact, velocity, whatever measure of output, the real question is _of what_? More code? More product surface area? That was never really the problem. In fact it makes life worse majority of the time.
  • mattmanser 7mo ago
    The real question is, is it increasing their velocity? They've already admitted they just 'throw the code away and start again'. I think we've got another victim of perceived productivity gains vs actual productivity drop. People sitting around watching Claude churn out poor code at a slower rate than if they just wrote it themselves. Don't get me wrong, great for getting you started or writing a little prototype. But the code is bad, riddled with subtle bugs and if you're not rewriting it and shoving large amounts of AI code into your codebase, good luck in 6-12 months time.
• wasmainiac 7mo ago
  Because its failure rate is too high. Beyond boilerplate code and CRUD apps, if I let AI run freely on the projects I maintain, I spend more time fixing its changes than if I just did it myself. It hallucinates functionally, it designs itself into corners, it does not follow my instructions, it writes too much code for simple features. It’s fine at replacing what stack overflow did nearly a decade ago, but that isn’t really an improvement from my baseline.
  • leptons 7mo ago
    That's my experience too. It's okay at a few things that save me some typing, but it isn't really going to do the hard work for me. I also still need to spend significant amounts of time figuring out what it did wrong and correcting it. And that's frustrating. I don't make those mistakes, and I really dislike being led down bad paths. If "code smells" are bad, then "AI" is a rotting corpse.
    • thewebguyd 7mo ago
      > If "code smells" are bad, then "AI" is a rotting corpse. This is what's so frustrating about the hype bros for me. In most cases, everything AI spits out are code smells. We're all just supposed to toss out every engineering principle we've learned all so the owner class can hire less developers and suppress wages? I'm sure it's working great for everyone working on SaaS CRUD or web apps, but it's still not anywhere close to solving problems outside that sphere. Native? It's very hit and miss. It has very little design sense (because, why would it? It's a language model) so it chokes on SwiftUI, it also can't stop using deprecated stuff. And that's not even that specialized. It still hallucinates cmdlets if you try to do anything with PowerShell, and has near zero knowledge about the industry I work in, a historically not tech-forward industry where things are still shared in handcrafted PDF reports emailed out to subscribers. I'm going to leave this field entirely if the answer just becomes "just make everything in React/React Native because it's what the AI does best."
      • fragmede 7mo ago
        That's interesting because I've been having pretty good luck with AI on SwiftUI. It didn't use to be but at some point they got good at that.
• RandomLensman 7mo ago
  Outside of coding/non-physical areas, the impact can be quite muted. I haven't seen much impact on surgical procedures, for example (but maybe others have?).
• KronisLV 7mo ago
  I’m currently working across like 5 projects (was 4 last week but you know how it is). I now do more in days than others might in a week. Yesterday a colleague didn’t quite manage to implement a loading container with a Vue directive instead of DOM hacks, it was easier for me to just throw AI at the problem and produced a working and tested solution and developer docs than to have a similarly long meeting and have them iterate for hours. Then I got back to training a CNN to recognize crops from space (ploughing and mowing will need to be estimated alongside inference, since no markers in training data but can look at BSI changes for example), deployed a new version of an Ollama/OpenAI/Anthropic proxy that can work with AWS Bedrock and updated the docs site instructions, deployed a new app that will have a standup bot and on-demand AI code review (LiteLLM and Django) and am working on codegen to migrate some Oracle forms that have been stagnating otherwise. It’s not funny how overworked I am and sure I still have to babysit parallel Claude Code sessions and sometimes test things manually and write out changes, but this is a completely different work compared to two or three years ago. Maybe the problem spaces I’m dealing with are nothing novel, but I assume most devs are like that - and I’d be surprised at people’s productivity not increasing. When people nag in meetings about needing to change something in a codebase, or not knowing how to implement something and its value add, I’ll often have something working shortly after the meeting is over (due to starting during it). Instead of sending adding Vitest to the backlog graveyard, I had it integrated and running in one or two evenings with about 1200 tests (and fixed some bugs). Instead of talking about hypothetical Oxlint and Oxfmt performance improvements, I had both benchmarked against ESLint and Prettier within the hour. Same for making server config changes with Ansible that I previously didn’t due to additional friction - it is mostly just gone (as long as I allow some free time planned in case things vet fucked up and I need to fix them). Edit: oh and in my free time I built a Whisper + VLM + LLM pipeline based on OpenVINO so that I can feed it hours long stream VODs and get an EDL cut to desired length that I can then import in DaVinci Resolve and work on video editing after the first basic editing prepass is done (also PyScene detect and some audio alignment to prevent bad cuts). And then I integrated it with subscription Claude Code, not just LiteLLM and cloud providers with per-token costs for the actual cuts making part (scene description and audio transcriptions stay local since those don't need a complex LLM, but can use cloud for cuts). Oh and I'm moving from my Contabo VPSes to running stuff inside of a Hetzner Server Auction server that now has Proxmox and VMs in that, except this time around I'm moving over to Ansible for managing it instead of manual scripts as well, and also I'm migrating over from Docker Swarm to regular Docker Compose + Tailscale networks (maybe Headscale later) and also using more upstream containers where needed instead of trying to build all of mine myself, since storage isn't a problem and consistency isn't that important. At the same time I also migrated from Drone CI to Woodpecker CI and from Nexus to Gitea Packages, since I'm already using Gitea and since Nexus is a maintenance burden. If this becomes the new “normal” in regards to everyone’s productivity though, there will be an insane amount of burnout and devaluation of work.
• belZaah 7mo ago
  I don’t think the objections are not necessarily in terms of lack of productivity although my personal experience is not that of massive productivity increases. The fact that you are producing code much faster is likely just to push the bottleneck somewhere else. Software value cycles are long and complicated. What if you run into an issue in 5 years the LLM fails to diagnose or fix due to complex system interactions? How often would that happen? Would it be feasible to just generate the whole thing anew matching functionality precisely? Are you making the right architecture choices from the perspective of what the preferred modus operandi of an llm is in 5 years? We don’t know. The more experienced folks tend to be conservative as they have experienced how badly things can age. Maybe this time it’ll be different?
• msvana 7mo ago
  I work as an ML engineer/researcher. When I implement a change in an experiment it usually takes at least an hour to get the results. I can use this time to implement a different experiment. Doesn't matter if I do it by hand or if I let an agent do it for me, I have enough time. Code isn't the bottleneck. I also heard an opinion that since writing code is cheap, people implement things that have no economic value without really thinking it through.
  • apsurd 7mo ago
    +1 on the economic value line. Not everything needs to be about money but if you get paid to ship code it's about money. And now we have coworkers shipping insane amounts of "features" because it's all free to ship and being an engineer, it ends there. Only it doesn't, there's product positioning, UX, information architecture, onboarding and training, support, QA, change management, analytics, reporting… sigh
    • embedding-shape 7mo ago
      > but if you get paid to ship code it's about money. Tip to budding software engineers: try to not work in these sort of places, as they're about "looking busy" rather than engineering software, where the latter is where real long-lasting things are built, and the former is where startup founders spend most their money. The last paragraph is where the tricky and valuable parts are, and also where AI isn't super helpful today, and where you as a human can actually help out a lot if you're just 10% better than the rest of the "engineers" who only want to ship as fast as possible.
• sivanmz 7mo ago
  It’s been my experience as of recently. I point it at an issue tracker and ask it to investigate, write a test to reproduce the problem and plan a fix together. There’s lots of hand holding from me but it saves me a lot of work and I’ve been surprised by its comfort with legacy code bases. For now I feel empowered, and I’m actually working more intensively, but I was wondering to myself if I’m going run out of work this year. Interestingly, our metrics show that output is slowed by increased workload on reviewers.
• jpollock 7mo ago
  The last time I tried AI, I tested it with a stopwatch. The group used feature flags... if (a) { // new code } else { // old code } void testOff() { disableFlag(a); // test it still works } void testOn() { enableFlag(a); // test it still works } However, as with any cleanup, it doesn't happen. We have thousands of these things lying around taking up space. I thought "I can give this to the AI, it won't get bored or complain." I can do one flag in ~3minutes. Code edit, pr prepped and sent. The AI can do one in 10mins, but I couldn't look away. It kept trying to use find/grep to search through a huge repo to find symbols (instead of the MCP service). Then it ignored instructions and didn't clean up one or the other test, left unused fields or parameters and generally made a mess. Finally, I needed to review and fix the results, taking another 3-5 minutes, with no guarantee that it compiled. At that point, a task that takes me 3 minutes has taken me 15. Sure, it made code changes, and felt "cool", but it cost the company 5x the cost of not using the AI (before considering the token cost). Even worse, the CI/CD system couldn't keep up the my individual velocity of cleaning these up, using an automated tool? Yeah, not going to be pleasant. However, I need to try again, everyone's saying there was a step change in December.
  • sdf2df 7mo ago
    [dead]
  • laserlight 7mo ago
    I did my own experiment with Claude Code vs Cursor tab completion. The task was to convert an Excel file to a structured format. Nothing fancy at all. Claude Code took 4 hours, with multiple prompts. At the end, it started to break the previous fixes in favor of new features. The code was spaghetti. There was no way I could fix it myself or steer Claude Code into fixing it the right way. Either it was a dead-end or a dice roll with every prompt. Then I implemented my own version with Cursor tab completion. It took the same amount of time, 4 hours. The code had a clear object-oriented architecture, with a structure for evolution. Adding a new feature didn't require any prompts at all. As a result, Claude Code was worse in terms of productivity: the same amount of time, worse quality output, no possibility of (or at best very high cost of) code evolution.
• staticassertion 7mo ago
  When it comes to novel work, LLMs become "fast typers" for me and little more. They accelerate testing phases but that's it. The bar for novelty isn't very high either - "make this specific system scale in a way that others won't" isn't a thing an LLM can ever do on its own, though it can be an aid. LLMs also are quite bad for security. They can find simple bugs, but they don't find the really interesting ones that leverage "gap between mental model and implementation" or "combination of features and bugs" etc, which is where most of the interesting security work is imo.
  • asadm 7mo ago
    I think your analysis is a bit outdated these days or you may be holding it wrong. I am doing novel work with codex but it does need some prompting ie. exploring possibilities from current codebase, adding papers to prompt etc. For security, I think I generally start a new thread before committing to review from security pov.
    • staticassertion 7mo ago
      You can do novel work with an LLM. You can. The LLM can't. It can be an aid - exploring papers, gathering information, helping to validate, etc. It can't do the actual novel part, fundamentally it is limited to what it is trained on. If you are relying on the LLM and context, then unless your context is a secret your competitor is only ever one prompt behind you. If you're willing to pursue true novelty, you need a human and you can leap beyond your competition.
      • bdangubic 7mo ago
        of course you need a human but do not need nearly as many humans as there are currently in the labor force
        • staticassertion 7mo ago
          Maybe, but I'm not really convinced. LLMs make some aspects of the job faster, mainly I don't have to type anymore. But... that was always a relatively small portion of the job. Design, understanding constraints, maintaining and operating code, deciding what to do, what not to do, when to do it, gaining consensus across product, eng, support, and customers, etc. I do all of those things as an engineer. Coding faster is really awesome, it's so nice, and I can whip up POCs for the frontend etc now, and that's accelerating development... but that's it. The reality is that a huge portion of my time is spent doing similar work and what LLMs largely do is pick up the smaller tasks or features that I may not have prioritized otherwise. Revolutionary in one sense, completely banal and a really minor part of my job in many others.
• kranke155 7mo ago
  I work in commercials. We can now make 1$ million dollar commercials with 100,000$ or less. So a 90% reduction in costs - if we use AI. The issue is they don’t look great. AI isn’t that great at some key details. But the agencies are really trying to push for it. They think this is the way back to the big flashy commercials of old. Budgets are lower than ever, and shrinking. Big issue here is really the misunderstanding of cause - budgets are lower, because advertising has changed in general (TV is less and less important ) and a lot of studies showed that advertising is actually not all that effective. So they are grabbing onto a lifeboat. But I’m worried there’s no land. I’ve planned my exit.
  • uxcolumbo 7mo ago
    Advertisement is not that effective in general or just for certain channels, i.e. TV? Also what are you existing to?
    • kranke155 7mo ago
      So my understanding - from a friend at WPP who told me the same and from a freakonomics episode - is that advertising was wildly oversold before digital. When the metrics arrived with digital, they saw that advertising, in some ways, was just not as effective as they’d hoped. In some ways the ROI wasn’t there. Seth Godin agrees. He says that advertising in the digital era could be as simple as just having a good product. I think this is Tesla’s position on it - make the best product and the internet takes care of it. Legacy companies have kept large ad budgets but those are diminishing. From what I spoke with my friend at WPP, he said their data science team showed that outside of a new product or a product that is not recognised by consumers, the actual outcomes from ads are marginal or incremental. Thats what he told me. If your product is already known to consumers, the ROI is questionable.
      • mikkupikku 7mo ago
        Advertising's foremost job is to sell the premise of advertising to business management. Selling the business's product is always secondary to that.
• oytis 7mo ago
  > I have stopped writing code, occasionally I jump into the changes proposed by LLM and make manual edits if it is feasible, otherwise I revert changes and ask it to generate again but based on my learnings from the past rejected output Isn't it a very inefficient way to learn things? Like, normally, you would learn how things work and then write the code, refining your knowledge while you are writing. Now you don't learn anything in advance, and only do so reluctantly when things break? In the end there is a codebase that no one knows how it works.
  • hobofan 7mo ago
    > you would learn how things work and then write the code In a legacy codebase this may require learning a lot of things about how things work just to make small changes, which may be much less efficient.
    • oytis 7mo ago
      I might still be naive about the industry, but if you don't know how the legacy codebase works, you might either delegate the change to someone else in the company who does, or, if there is no one left, use this opportunity to become the person who knows at least something about it.
  • throwaw12 7mo ago
    > Isn't it a very inefficient way to learn things? It is. But there are 2 things: 1. Do I want to learn that? (if I am coming back to this topic again in 5 months, knowledge accumulates, but there is a temptation to finish the thing quickly, because it is so boring to swim in huge legacy codebase) 2. How long it takes to grasp it and implement the solution? If I can complete it with AI in 2 days vs on my own in 2 weeks, I probably do not want to spend too much time on this thing as I mentioned in other comments, this is exactly makes me worried about future of the work I will be doing, because there is no attachment to the product in my brain, no mental models being built, no muscles trained, it feels someone else's "work", because it explores the code, it writes the code. I just judge it when I get a task
    • 7mo ago
• iugtmkbdfil834 7mo ago
  I don't want to generalize from my specific situation too much, but I want to offer an anecdote from my neck of the woods. On my personal sub, I agree it is kinda crazy the kind of projects I can get into now with little to no prior knowledge. On the other hand, our corporate AI is.. not great atm. It was briefly kinda decent and then suddenly it kinda degraded. Worst case is, no one is communicating with us so we don't know what was changed. It is possible companies are already trying to 'optimize'. I know it is not exactly what you are asking. You are saying capability is there, but I am personally starting to see a crack in corporate willingness to spend.
• tripledry 7mo ago
  Something I've been thinking about lately is if there is value in understanding the systems we produce and if we expected to? If I can just vibe and shrug when someone asks why production is down globally then I'm sure the amount of features I can push out increases, but if I am still expected to understand and fix the systems I generate, I'm not convinced it's actually faster to vibe and then try to understand what's going on rather than thinking and writing. In my experience the more I delegate to AI, the less I understand the results. The "slowness and thinking" might just be a feature not a bug, at times I feel that AI was simply the final straw that finally gave the nudge to lower standards.
  • joe_mamba 7mo ago
    >if I can just vibe and shrug when someone asks why production is down globally You're pretty high up in the development, decision and value-addition chain, if YOU are the responsible go-to person for these questions. AI has no impact on your position.
    • tripledry 7mo ago
      Naa, I'm just a programmer. Experience may vary depending on company and country, for me this has been true from tiny startups to global corporations. Tangential, I don't even know what "responsible" in the corporate world means anymore, it seems to me no one is really responsible for anything. But the one thing that's almost certain is that I will fix the damn thing if I made it go boom.
• drekipus 7mo ago
  > my job is easier now, I do less. > I am terrified about what's coming. God I hope I never ever have to work with you
• dumfries 7mo ago
  "it works" is a very low standard when it comes to software engineering. Why are we not holding AI generated code to the same standards as we hold our peers during code reviews? I have never heard anyone say "it works" as a positive thing when reviewing code.. Yes, there is a productivity boost but you can't tell me there is no decrease in quality
• lm28469 7mo ago
  Meanwhile gemini tells me my go code doesn't compile (it does) Gaslight me by telling me I must be a time traveler because I use go 1.26 but the latest version actually is 1.24 And tell me I can't use wg.Go() because this function does not exist (it does)
• mirsadm 7mo ago
  I have an app which is fairly popular. This release cycle I used Claude Code and codex to implement all the changes / features. It definitely let me move much quicker than before. However now that it's in the beta stage the amount of issues and bugs is insane. I reviewed a lot of the code that went in as well. I suspect the bug fixing stage is going to take longer than the initial implementation. There are so many issues and my mental model of the codebase has severely degraded. It was an interesting experiment but I don't think I would do it again this way.
  • truetraveller 7mo ago
    Thanks for the insight!
  • maplethorpe 7mo ago
    Rather than trying to fix the bugs yourself, have you tried asking Claude to fix them for you?
    • mirsadm 7mo ago
      I have already been doing this. I could keep doing it but I'm not going to. I want to be able to understand my own code because that is what let's me make sound higher level decisions.
  • aurareturn 7mo ago
    The last 10% takes up 90% of the time. Usually, the time is spent finding issues you didn't even know about. This was true before LLMs.
  • johsole 7mo ago
    I make mistakes when writing code, but I know what types of mistakes I make. With AI it's like a coworker who makes mistakes, sometimes they're obvious to me and sometimes they're not, because I make different mistakes.
  • sdf2df 7mo ago
    Congrats. Now post this more often so the bozo's who downvote posts that push-against pro-LLM stuff f-off. I came to the same conclusion when producing a video with Grok. Did the job but utterly painful and it was definitely very costly - I used 50 free-trial accounts and maxed them out each day for a month. Im pretty sure these conclusions hold across all models and therefore the technology by extension.
• zozbot234 7mo ago
  > I am terrified about what's coming Why? This is great. AI fixing up huge legacy codebases is just taking the jobs humans would never want to do.
• gurghet 7mo ago
  Basically it tricks you into making the code less maintainable, so while it seems to boost productivity, it's just delaying huge failures.
  • rootusrootus 7mo ago
    Exactly this, IMO. We are in a race to see whether the mountain of technical debt that AI is creating grows faster than AI's ability to whittle it down later.
• rich_sasha 7mo ago
  I find LLMs are good at essentially boilerplate code. It's clear what to do and it needs to be typed in. Or areas where I really have no idea where to start, because I'm not familiar with the codebase. I find anything else, I spend more time coaxing them into doing 85% of what I need that I'm better off doing it myself. So they're not useless but there's only so many times in a week that I need a function to pretty-print a table in some fashion. And the code they write on anything more complex than a snippet is usually written poorly enough that it's a write-once-never-touch-again situation. If the code needs to be solid, maintainable, testable, correct (and these are kind of minimal requirements in my book) then LLMs make little impact on my productivity. They're still an improvement on Google and Stack exchange, but again - only gets you so far. YMMV
  • joenot443 7mo ago
    > I spend more time coaxing them into doing 85% of what I need that I'm better off doing it myself What was the last thing you built in which you felt this was the case?
  • vividfrier 7mo ago
    [flagged]
    • rich_sasha 7mo ago
      I often wonder what I am missing. Recently I wanted to wrap a low level vendor API with a callback API (make a request struct and request id, submit, provide a callback fn, which gets called with request IDs and messages received from vendor) to async Python (await make_request(...)). Kinda straightforward - lots of careful code of registering and unregistering callbacks, some careful thread synchronisation (callbacks get called in another thread), thinking about sane exception handling in async code. Fiddly but not rocket science. What I got sort of works, as in tests pass - this with Opus 4.5. It is usable, though it doesn't exist cleanly on errors despite working to death with Claude about this. On exception it exits dirtily and crashes, which is good enough for now. I had some fancy ideas about logging messages from the vendor to be able to replay them, to be able to then reproduce errors. Opus made a real hash of it, lots of "fuck it comment out the assert so the test passes". This part is unusable and worse, pollutes the working part of the project. It made a valiant effort at mocking the vendor API for testing but really badly, instead of writing 30 lines of general code, it wrote 200 lines of inconsistent special cases that don't even work altogether. Asked to fix it it just shuffles around the special cases and gets stuck. It's written messily enough that I wouldn't touch this even to remove the dead code paths. I could block a few days for it to fix but frankly in that time I can redo it all and better. So while it works I'm not gonna touch it. I did everything LLM proponents say. I discussed requirements. Agent had access to the API docs and vendor samples. I said think hard many times. Based on this we wrote a detailed spec, then detailed inplementation plan. I hand checked a lot of the high level definitions. And yet here I am. By the time Opus went away and started coding, we had the user facing API hammered out, key implementation details (callback -> queue -> async task in source thread routing messages etc), constraints (clean handling of exceptions, threadsafe etc). Tests it has to write. Any minor detail we didn't discuss to death was coded up like a bored junior. And this also wasn't my first attempt, this was attempt #3. First attempt was like, here's the docs and samples, make me a Python async API. That was a disaster. Second was more like, let's discuss, make a spec, then off you go. No good. Even just taking the last attempt time, I would have spent less time doing this by hand myself from scratch.
• motbus3 7mo ago
  I think you can more stuff done earlier but the quality is not good or it doesn't work as expected if you tinker with it enough. Fixing the issues from the generated code usually doesn't work at all
• exfalso 7mo ago
  It's failing when there is no data in the training set, and there are no patterns to replicate in the existing code base. I can give you many, many examples of where it failed for me: 1. Efficient implementation of Union-Find: complete garbage result 2. Spark pipelines: mostly garbage 3. Fuzzer for testing something: half success, non-replicateable ("creative") part was garbage. 4. Confidential Computing (niche): complete garbage if starting from scratch, good at extracting existing abstractions and replicating existing code. Where it succeeds: 1. SQL queries 2. Following more precise descriptions of what to do 3. Replicating existing code patterns The pattern is very clear. Novel things, things that require deeper domain knowledge, coming up with the to-be-replicated patterns themselves, problems with little data don't work. Everything else works. I believe the reason why there is a big split in the reception is because senior engineers work on problems that don't have existing solutions - LLMs are terrible at those. What they are missing is that the software and the methodology must be modified in order to make the LLM work. There are methodical ways to do this, but this shift in the industry is still in baby shoes, and we don't yet have a shared understanding of what this methodology is. Personally I have very strong opinions on how this should be done. But I'm urging everyone to start thinking about it, perhaps even going as far as quitting if this isn't something people can pursue at their current job. The carnage is coming:/
• vividfrier 7mo ago
  [flagged]
• girvo 7mo ago
  Sometimes I’m scared. Sometimes I realise that this particular task has been slower than if I’d done it myself when I take in to account full wall clock time. I can’t tell what type of task is going to work ahead of time yet.
• randusername 7mo ago
  I see an individual productivity boost, but not necessarily a collective one. I don't think features per hour is really what is holding back most established businesses. My experiences suggest that we still have some time before the people that understand the plumbing of the business _and_ AI bubble up to positions of authority through wielding it practically and successfully at increasingly greater scale.
• matheusmoreira 7mo ago
  Terrifying doesn't quite say it. The situation we're in is either we achieve a post-scarcity society or we'll all die.