4 ms·

> The productivity gains of 30% are probably overstated it maybe not, LLMs deliver clear value in coding tasks, but the thing is that competitors also will hav

> The productivity gains of 30% are probably overstated it maybe not, LLMs deliver clear value in coding tasks, but the thing is that competitors also will have gains, deliver more features, fixes and products.

darth_avocado 2y ago
I am yet to have Google, Microsoft, Amazon or Facebook suggest that they’ve achieved 30% productivity gains from AI. Those are companies at the forefront of AI investment and if they haven’t achieved that kind of increased engineering productivity, I am ready to bet money on the fact that any other company claiming so hasn’t either. Even if 30% productivity gains are true, they are probably not because of AI. They could have fired a bunch of low performers and overworked the rest of the engineers to achieve productivity gains, but even then, I’d be very skeptical if the 30% gains would stay there long term.
- riku_iki 2y ago
  It depends on vertical. Large corps likely have some huge codebases of overengineered spagetti code, which are hardly comprehendable by LLMs. But if you want to build for example some new/smaller web/mobile apps talking to various API, LLM can boost your productivity significantly, because it will easily generate ready to use code snippets.
  - darth_avocado 2y ago
    But, we are talking about Salesforce.
  - taurath 2y ago
    But the second those smaller prototype apps become big enough to need to scale or be maintained, you’re back at the same problem. The whole thing it gets you then is “faster experimentation”, which was something that was /already fast/ if you use modern tools of the day. 1 week vs 2 weeks for a prototype app may build you more prototypes but it doesn’t help when you can’t scale them. It does smooth some of the processes though - AI is maybe helpful for cognitive augmentation for coders, but it’s not going to be building you apps worth a damn.
    - spac 2y ago
      while I agree with the sentiment that it's hard to evaluate the real gains, it's definitely not just about code generation. Debugging is a a big part of the value, IMO.
  - mullingitover 2y ago
    > Large corps likely have some huge codebases of overengineered spagetti code, which are hardly comprehendable by LLMs. I think it's the other way around, though. Those code monstrosities aren't comprehendible by humans, especially after the wanton RIFs that have happened in the past couple years that have cut loose a lot of people who know where the bodies are buried. However, with copilot you can just figuratively walk up to any repo and ask "@workspace what's going on in this codebase" and it'll tell you. From experience I can say this can deliver results. Downright rotten code that would've taken me a good week to figure out can be figured out in an hour. It's damn near witchcraft.
    - angoragoats 2y ago
      I have never seen this work, at all, with a large codebase. Can you give a specific example of a large repo where this produced useful/coherent results?
      - onemoresoop 2y ago
        Id be curious as well but really doubt AI could make much headway into a legacy codebase I recently spent some time on. There is tacit knowledge that AI cannot pick up on, many things are named wrong, there are patches and workarounds that barely make any sense, architecture inconsistencies and transitions in style from 20 years ago to now. Maybe if there was some documentation and explanatory notes. Without needing to fill the gaps by simply asking the prompter for more information, im inclined to suspect the same old eagerness to bullshitting that LLMs have been programed in.
        angoragoats 2y ago
        I have seen multiple people claim that {Copilot|Claude|a local Llama model} has been great for them at understanding large codebases, but at least to date, I have yet to actually have anyone provide a concrete example when I ask. Maybe others have a different idea of what constitutes "large." At my job, our main repo is over 300k lines of just Ruby code, plus a bunch of JS, ERB templates, and other stuff. Every AI tool I've thrown at it is great at making surgical edits to single files (or small groups of files) but completely chokes if you ask it a question that requires it to understand context across the repo. I'm always hoping that I'm just using the tools wrong, but so far that doesn't seem to be the case.
- camdenreslink 2y ago
  The Google CEO did say that > 25% of Google's internal new code is written using AI. But the claim seems a bit dubious... https://news.ycombinator.com/item?id=41991291 https://news.ycombinator.com/item?id=41991291

darth_avocado 2y ago

I am yet to have Google, Microsoft, Amazon or Facebook suggest that they’ve achieved 30% productivity gains from AI. Those are companies at the forefront of AI investment and if they haven’t achieved that kind of increased engineering productivity, I am ready to bet money on the fact that any other company claiming so hasn’t either. Even if 30% productivity gains are true, they are probably not because of AI. They could have fired a bunch of low performers and overworked the rest of the engineers to achieve productivity gains, but even then, I’d be very skeptical if the 30% gains would stay there long term.

riku_iki 2y ago
It depends on vertical. Large corps likely have some huge codebases of overengineered spagetti code, which are hardly comprehendable by LLMs. But if you want to build for example some new/smaller web/mobile apps talking to various API, LLM can boost your productivity significantly, because it will easily generate ready to use code snippets.
- darth_avocado 2y ago
  But, we are talking about Salesforce.
- taurath 2y ago
  But the second those smaller prototype apps become big enough to need to scale or be maintained, you’re back at the same problem. The whole thing it gets you then is “faster experimentation”, which was something that was /already fast/ if you use modern tools of the day. 1 week vs 2 weeks for a prototype app may build you more prototypes but it doesn’t help when you can’t scale them. It does smooth some of the processes though - AI is maybe helpful for cognitive augmentation for coders, but it’s not going to be building you apps worth a damn.
  - spac 2y ago
    while I agree with the sentiment that it's hard to evaluate the real gains, it's definitely not just about code generation. Debugging is a a big part of the value, IMO.
- mullingitover 2y ago
  > Large corps likely have some huge codebases of overengineered spagetti code, which are hardly comprehendable by LLMs. I think it's the other way around, though. Those code monstrosities aren't comprehendible by humans, especially after the wanton RIFs that have happened in the past couple years that have cut loose a lot of people who know where the bodies are buried. However, with copilot you can just figuratively walk up to any repo and ask "@workspace what's going on in this codebase" and it'll tell you. From experience I can say this can deliver results. Downright rotten code that would've taken me a good week to figure out can be figured out in an hour. It's damn near witchcraft.
  - angoragoats 2y ago
    I have never seen this work, at all, with a large codebase. Can you give a specific example of a large repo where this produced useful/coherent results?
    - onemoresoop 2y ago
      Id be curious as well but really doubt AI could make much headway into a legacy codebase I recently spent some time on. There is tacit knowledge that AI cannot pick up on, many things are named wrong, there are patches and workarounds that barely make any sense, architecture inconsistencies and transitions in style from 20 years ago to now. Maybe if there was some documentation and explanatory notes. Without needing to fill the gaps by simply asking the prompter for more information, im inclined to suspect the same old eagerness to bullshitting that LLMs have been programed in.
      - angoragoats 2y ago
        I have seen multiple people claim that {Copilot|Claude|a local Llama model} has been great for them at understanding large codebases, but at least to date, I have yet to actually have anyone provide a concrete example when I ask. Maybe others have a different idea of what constitutes "large." At my job, our main repo is over 300k lines of just Ruby code, plus a bunch of JS, ERB templates, and other stuff. Every AI tool I've thrown at it is great at making surgical edits to single files (or small groups of files) but completely chokes if you ask it a question that requires it to understand context across the repo. I'm always hoping that I'm just using the tools wrong, but so far that doesn't seem to be the case.
camdenreslink 2y ago
The Google CEO did say that > 25% of Google's internal new code is written using AI. But the claim seems a bit dubious... https://news.ycombinator.com/item?id=41991291 https://news.ycombinator.com/item?id=41991291