4 ms·

>If agentic development actually worked the way any of them say it does I think its fascinating just how much of a gap there is between what's being claimed, a

• xendo 2mo ago
  @antirez is a very prominent open source contributor that gets lot of shit done with LLMs. Mitchel Hashimoto is also open about using LLMs to speed up his work. There are some caveats attached: neither of them is doing crazy loops or graphs producing thousands of lines of code, they are both amazing software engineers and they know what they are doing.
  • ryoshu 2mo ago
    Yes. Feels like the better you are at your craft the better the tools work.
    • disgruntledphd2 2mo ago
      AI is a power tool/factory with no safety features. Expertise is required to get anything useful from them.
      • padjo 2mo ago
        Yep, much like having access to a circular saw won't make me a carpenter, having access to an LLM won't make the average person a software engineer.
        • raffael_de 2mo ago
          you're moving the goal post as language models and agents aren't marketed as something like circular saw but a team of carpenters that you just tell what you want and they do it for you.
          • padjo 2mo ago
            I never claimed to agree with the marketing.
• kvark 2mo ago
  Or maybe open-source development is just poorly compatible with AI workflow? Today, projects may need a community, an issue tracker, but pull requests are becoming less important. I suspect the ban of AI in established projects to be a very complex decision. Even if core developers would like to use AI, they don't want to review all of the AI-generated code from the larger community. So the only consistent way to preserve sanity is to declare that nobody would use it. Good luck enforcing it though!
  • 20k 2mo ago
    Why would the PR format be bad for LLMs? Its just code review + merging in branches, which is what every company should be doing anyway Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors
    • TeMPOraL 2mo ago
      > Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors And if they're doing that, why would they tell you, or anyone? And if they're doing that responsibly - collaborating with AI, and reviewing the code - they don't even have an ethical reason to tell about AI involvement, any more than telling about the StackOverflow answers or blog posts they read before coming up with some implementation.
      • 20k 2mo ago
        If they were sneaking LLM generated code into these projects, we'd expect to see their feature development 10x even if they covertly aren't telling anyone. Which open source codebases have seen a truly massive 10x increase in delivered feature velocity since AI turned up? Where are all the new contributors who contribute features at 10x the rate of the existing meatcoding developers? Then all it'd take is one of these engineers blowing the lid on their whole secret LLM contribution strategy to absolutely break the industry wide open to the benefits of these massive productivity increases, and finally prove all the doubters wrong. They sure are good at keeping secrets that would directly benefit them to expose
• satvikpendem 2mo ago
  We have a greenfield project at our company, yes proprietary, now taking us months where previously it'd have taken weeks for even a single feature. We definitely see the (whatever)x performance boost with our own eyes.
  • vatsachak 2mo ago
    > now taking us months where it'd have taken weeks... So 0<whatever<1
    • marcosdumay 2mo ago
      Nah, the GP's claim is weeks for each feature, versus a few months in total. On a greenfield project. Weeks for each feature in a greenfield project... Yeah, LLMs make unworking code way faster than that.
    • satvikpendem 2mo ago
      Feature vs full project timeline
  • 20k 2mo ago
    Its always been possible to trade long term productivity for short term gains with technical debt. This is why the bar I'm interested in is long term projects, which have proven to have long term success, instead of a small disposable project where the code quality doesn't matter
    • deleted 2mo ago
      [deleted]
    • satvikpendem 2mo ago
      Difference is whether it's a false dichotomy now, because models like Fable can write better code than most devs I know.
      • 20k 2mo ago
• mark242 2mo ago
  > Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly? It could be that the engineers who are extremely productive with LLMs are landing PRs that look indistinguishable from good, hand-written PRs.
  • vatsachak 2mo ago
    LLM code is obvious to spot. If an LLM designed a screwdriver set it would make a screwdriver for each head instead of making a replaceable head. LLMs are superhuman at short term coding such as debugging and writing tests though and you're missing out by not using them there.
    • esseph 2mo ago
      > LLM code is obvious to spot. If an LLM designed a screwdriver set it would make a screwdriver for each head instead of making a replaceable head. Commercial and industrial electricians are LLMs now?
    • sublinear 2mo ago
      So... basically an autocomplete to help you type faster and spot check if you did something unusual in the parts you wrote by hand?
  • 20k 2mo ago
    If that were true, we'd expect to see massively accelerated velocity of open source projects by these engineers. They should be creating new open source projects at a truly astounding rate, with new tooling springing up every day that dwarfs the existing open source space as their productivity completely eclipses traditional development Instead, software is plodding along exactly the same as it did prior to LLM code generation, and there's no evidence of superprogrammers making superprojects in 1/10th of the time. With a 10x productivity gain, what used to take a year should take a month
    • mark242 2mo ago
• vatsachak 2mo ago
  Nah this take is wrong. I used Claude code with my custom skill and I wrote a more performant scheduler than the default Linux one in Rust. It's not just productivity, it's life changing.
  • 20k 2mo ago
    Sure, it just seems a little odd that no LLMgineer ever contributes their incredible more performant scheduler back though right? After all if you can do it with claude, anyone can, all it'd take is to ask claude to rewrite it. Linux accepts LLM generated PRs, all the code has to do is meet the review bar and one of the most critical pieces of software engineering on the planet gets better for everyone
    • vatsachak 2mo ago
      I would submit it but the pr was too long to fit on GitHub
      • deleted 2mo ago
        [deleted]
      • weakfish 2mo ago
        … doesn’t Linux work through email, not GitHub?
        • cyclopeanutopia 2mo ago
          It's a reference to Fermat's note.
    • Lerc 2mo ago
      Why does it seem odd when people who do attempt to contribute back are attacked? AI lowers the bar to making submissions which permits flawed subissions to be thoughtlessly submitted by people who don't really understand what they are doing, technically or culturally. It does, however, enable people who put in the effort to work on something significant. Those people are fewer in number, but exist. They do get to see the animosity that unwitting novices receive. I don't think it would be surprising for those people to opt out of engaging with a toxic environment.
• onion2k 2mo ago
  Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful. There are two incentives for contributing to open source. The first is to make the app better (add a feature, improve the code, fix a bug, etc.) It's possible that LLMs don't meet the bar but if a human has put the effort in it's not always obvious that it's AI. More likely LLM code is accepted when it's good and rejected when it's bad. The system works. However, the second incentive is that open source contributions are seen as a 'ahortcut' to making a name for yourself. Being a contributor on a big project goes on resumes and GitHub profiles. Often people who use AI for that don't review the code or even check it does what they say it does. That slop needs to be banned. The downside is losing the good contributions, but it's still a net win.
• TeMPOraL 2mo ago
  > Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly? Selection bias? OSS has a thing proprietary projects don't - an endless cohort of opportunists trying to wedge in a "contribution" for personal gain, be it a Hacktober t-shirt or resume boost or an occasional vulnerability (with the resume boosting being by far most likely). The good LLM-based contributions, you probably won't recognize as AI-assisted unless the author explicitly decided to label them, and if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement, much less any benefit. OSS involvement in general stopped being an indicator of skill once Github activity became a factor for job applications. Edit: There's also a second factor: many people (myself included) use LLMs to code one-off personal tools with no intent of them having more than one user, ever. Get an itch, have an LLM scratch it for you, carry on with your life. You'll see little trace of it, because it's hard to distinguish a single-user program that's good enough for specific purpose from a pile of autogenerated slop that probably doesn't work, and neither of these will show up as PRs to big OSS repos anyway.
  • 20k 2mo ago
    The the idea that engineers are covertly sneaking huge quantities of LLM generated code into these projects doesn't pass the smell test, as there's lots of evidence that this isn't true. It'd be incredibly obvious if new contributors were turning up en masse and contributing absolutely incredible amounts of high quality code, because everyone would be crowing about it Instead, if you check out something like ImGui, we can see that its largely just Omar as usual with a bunch of PRs. Nothing much seems to have changed, feature development carries on at the same pace as it did previously. This is replicated across nearly every single major long term open source project I can find The only reason developers now feel the need to hide their LLM usage is because the LLM contributions have all been shockingly bad, so you're reversing cause and effect. If LLM contributions had been incredible, they wouldn't be frowned upon socially now. Surely someone would blow the lid on this huge covert conspiracy about how incredible their contributions to.. SFML have been? This effect also wouldn't prevent LLMgineers from creating their own open source projects that should be absolutely outstripping the existing open source space. After all how old is ImGui? With a 10x productivity gain, it should take a year of development to easily match its features if those gains are truly real now >if they're really good and use LLMs responsibly, they probably don't even have any ethical obligation to label LLM involvement The copyright issues means that non disclosure is inherently unethical
• biql 2mo ago
  > I regularly see people claim that they are now 10x more productive with LLM code generation, and I just wonder where all the code is. Perhaps the biggest issue hasn't been the speed of development all along but people working on wrong things, hence why progress isn't as evident. LLM helping to build something that isn't needed faster isn't going to productivity more visible.
  • 20k 2mo ago
    There's no reason that some people wouldn't be using their new found 10x superpowers on open source projects though
    • Supermancho 2mo ago
      I use my powers to finish my own projects (until the next creative barrier) while my professional workload has been reduced to 20% effort with 0 stress. I wrote a game I left in a folder from 11 years ago in 2 months, front and back plus all the tooling, plus features I never would have attempted before. I recreated open-sourced projects from github, written in a language I wasn't familiar with, so I could better work with the application and include it in my backend codebase. I rewrote my mail server in less than 2 days, which took me over 2 weeks to configure and setup the last time. My life is easier, because the demands are the same but I have more free time. I research more, I learn more, and I try more things. Just like any software project, some are abandoned (even with LLMs) when running into unforeseen issues or mismatches with theoretical plans. Most importantly, I can make things that I think are exciting, with very little effort. The first 35 years of development, I struggled to be good enough to tackle sourcing various projects without becoming frustrated and mostly knowing myself well enough to avoid trying at all. Now I can see myself writing passion projects until I'm dead.
• vonnieda 2mo ago
  > I think its fascinating just how much of a gap there is between what's being claimed, and the verifiable observable data of the open source world. According to this list of top Github repos by stars[1], of the top 10, nine of them are informational and one is OpenClaw, which certainly has plenty of AI generated code. The next 10 include react which has a CLAUDE.md and a .claude, and lots of landed PRs that look like AI. And linux, which, we all read Linus' stance. He sees the value. And superpowers, ECC, and hermes-agent. All of which are AI stuff. I think it would be reasonable to ask which major open source projects aren't using AI assistance? > Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful. That is not why. Or at least not the majority of it. The biggest part is that maintainers just can't keep up. They don't have time to do in depth reviews for the number of PRs coming in to find out if it: works, does what it says it does, meets a need, has docs, is well written, meets criteria, etc. That does not mean all those PRs are trash. Some might be trash, some might be gold. Just like human generated ones. What it comes down to is vibes. Anti-AI folk don't like AI for reasons (that are valid) and they don't want to find any value in it. So they don't. But lots of people are. Lots of really cool, interesting, clever, new software is being written and getting used and building communities but people see CLAUDE.md and go "slop, no thanks" and won't even give it a try. [1]: https://github.com/EvanLi/Github-Ranking/blob/master/Top100/Top-100-stars.md https://github.com/EvanLi/Github-Ranking/blob/master/Top100/...
  • 20k 2mo ago
    >I think it would be reasonable to ask which major open source projects aren't using AI assistance? Where are all the vast quantity of LLM commits for these open source projects, if these engineers are truly 10xing their productivity with LLM code generation? Because I regularly browse the source of many of these, and can find almost no evidence that LLM code generation is anywhere near the level of productivity being claimed for: 1. ImGui 2. SFML 3. SDL 4. GLFW 5. OpenSSL 6. Linux 7. GCC 8. Clang/LLVM 9. Libc++/libstdc++/msstl 10. Rust 11. Godot 12. Nlohmann 13. Boost 14. Python 15. GNU tools Etc etc. There are even more major projects, but these are just the critical ones I can think of off the top of my head. If you go for a browse though eg Godot, you might think that LLMs don't exist. In fact if you go for a browse through any of these, you might think that LLMs are borderline never used for code generation in the open source world. This is a diverse collection of random projects, and yet for some reason none of these have experienced a 10x increase in productivity in any form from any contributors from LLM code generation >That is not why. Or at least not the majority of it Its not the quantity, if maintainers were receiving incredibly high quality PRs that could just be merged, they'd be very happy. Godot is an example of a project that was initially pro LLM, and then had to about face because the PRs were just absolutely crap >Some might be trash, some might be gold Is there any evidence that any of these PRs have ever been gold?
• kmclean 2mo ago
  > I strongly suspect that what we're seeing is that much of the tech code-writing economy had already become completely performative prior to AI turning up. It no longer matters in the current age if your code is good, or works, because your job is to give the illusion of product development while the stock market price gets pumped, until you all cash out your share value, get bought, or hop jobs in 2 years. I think this is probably the answer. Working as a professional software engineer actually has very little to do with writing good code and maybe never has. The job is to “ship products”, and as long as you keep up a sufficiently convincing illusion of progress toward that end the bills get paid and nobody really cares about the impact on end users.
• potsandpans 2mo ago
  [flagged]
• vinyl7 2mo ago
  LLMs work well for boring stuff that other people have already done a hundred times. Where the LLMs fail is solving difficult problems that aren't explored as well. The people seeing the 10x speed up are likely just doing CRUD apps or other software that have already been well documented by Stack Overflow or cloned a million times on Github. In my own use, it completely breaks down when for solving actually difficult things.
• etrautmann 2mo ago
  I find it game changing for experimental work and prototyping. As a research scientist this is a major bottleneck and clean code is often not the right optimization target.
• mancerayder 2mo ago
  I'm coming to the conclusion that the right way to use LLMs for code generation is asking it to solve specific problems, giving it existing code and asking it how it would propose changing something or other, or if it has an alternative to an existing logic flow that's problematic and asking it what it thinks the problem is and how it would suggest it be written. I'm coming from the perspective of infra management so that's my bias, and a very complex code base may make my perspective off mark. But nowhere does it seem it to be a good idea for the whole thing to be LLM generated. It seems to view a big holistic problem in steps in isolation, causing it to add complexity on top of complexity as it's going through it's later stages. For example it'll create infrastructure designs that add layers of unnecessary and redundant tooling and business logic, when a human would have re evaluated step 1's path when it realized step 3 was doing the same thing again in a more complex way.
• reiniertl 2mo ago
  [flagged]