2 ms·

The original calculation is not even right, because 100% of the work software engineers do probably isn’t even writing code. A lot of the time they’re debugging

The original calculation is not even right, because 100% of the work software engineers do probably isn’t even writing code. A lot of the time they’re debugging code or attending some kind of meeting. So if only 20% of their time is coding and they do 1/5th of it, they’re like what, 16% more productive overall? Now factor in the expense of using LLMs, it’s not worth it, the math isn’t mathing. You can’t even squeeze a 2x improvement, let alone the 5-10x that has become expected in the tech industry.

JohnMakin 2y ago
Also absent in these discussions is who maintains the 80% that wasn’t written by them when it needs to either be fixed or adapted to a new use case. This is where I typically see “fast” AI prototypes fall over completely. Will that get better? I’m ambivalent but could concede ‘probably’ but at the end of the day I don’t think you can get away from the fact that for every non trivial programming task, there does need to be a human somewhere to verify/validate work, or at the very least understand it enough to debug it. This seems like an inescapable fact.
- deadbabe 2y ago
  And one thing people don’t realize is that if you’re working in a codebase heavily coded by AI, you’re going to want to see the original prompts used if you don’t know the original engineers intent. And no one seems to track this. It becomes a game of Chinese whispers. Honestly this quickly becomes a shit show and I don’t know how anyone can seriously consider having a large amount of a codebase coded by AI.
  - trescenzi 2y ago
    How is this different from any legacy code base? Everywhere I've worked there's been swaths of code where the initial intent is lost to time and debugging them is basically archeology. Random AI code isn't going to be categorically different from random code copied from StackOverflow, or code simply written a few years ago. Reading code will always be harder than writing it.
    - onemoresoop 2y ago
      No, in time it’s going to look just like that but with hallucinations and large areas of the codebase no human has even read. No doubt digging through code and trying to understand will involve a lot more prompting back and forth
      - dragonwriter 2y ago
        > No, in time it’s going to look just like that but with hallucinations and large areas of the codebase no human has even read. An area of code that no human has ever read is no different to me than an area of code lots of humans who aren't me have read, and that's not that different from an area of code that I have read, but not particularly recently. I mean, except for who it is that I am going to be swearing about when it breaks and I finally do read it.
        onemoresoop 2y ago
        An area of code that’s been read a lot is likely to be like a paved road. Sure, it could have cracks or mud but it’s generally understood and really bad warts fixed or warning signs left behind. An area of code that runs rarely and is inspected even more rarely is likely to have bring more surprises and unexpected behaviors.
    - deadbabe 2y ago
      Legacy code is written by humans, and when you read stuff written by humans you can anticipate what their intent was because you are a human as well. With AI, there is no intent, the AI is a probabilistic model and you have to guess if the model got it right. It’s like the difference between driving next to a human driver vs a self driving car. You have some idea of what the human will do but the self driving car could flip out at any moment and do something irrational.
      - dragonwriter 2y ago
        > Legacy code is written by humans, and when you read stuff written by humans you can anticipate what their intent was because you are a human as well. Well, no. This is both technically categorically false (because you can only "anticipate" something in advance, and the intent is in the past by the time you are reading the code, though you might in principal infer their intent), and often practically false even when read with "infer" in place of "anticipate", as it is quite common for the intent of human-written code to be non-obvious, except in the sense of "the purpose of a system is what it does", in which sense AI-written code is not particularly more opaque.
  - dragonwriter 2y ago
    > And one thing people don’t realize is that if you’re working in a codebase heavily coded by AI, you’re going to want to see the original prompts used if you don’t know the original engineers intent. If I'm working in a codebase that is heavily coded by anyone who isn't "me, within the last week", I'm going to want to see the implementation-technology-neutral requirements (and, ideally, human-written tests/test scenarios, unit and acceptance and everywhere in between, operationalizing the requirements) because I can't know the original engineer's intent. This isn't any less true when the engine was wet biological goop instead of silicon.

JohnMakin 2y ago

Also absent in these discussions is who maintains the 80% that wasn’t written by them when it needs to either be fixed or adapted to a new use case. This is where I typically see “fast” AI prototypes fall over completely. Will that get better? I’m ambivalent but could concede ‘probably’ but at the end of the day I don’t think you can get away from the fact that for every non trivial programming task, there does need to be a human somewhere to verify/validate work, or at the very least understand it enough to debug it. This seems like an inescapable fact.

deadbabe 2y ago
And one thing people don’t realize is that if you’re working in a codebase heavily coded by AI, you’re going to want to see the original prompts used if you don’t know the original engineers intent. And no one seems to track this. It becomes a game of Chinese whispers. Honestly this quickly becomes a shit show and I don’t know how anyone can seriously consider having a large amount of a codebase coded by AI.
- trescenzi 2y ago
  How is this different from any legacy code base? Everywhere I've worked there's been swaths of code where the initial intent is lost to time and debugging them is basically archeology. Random AI code isn't going to be categorically different from random code copied from StackOverflow, or code simply written a few years ago. Reading code will always be harder than writing it.
  - onemoresoop 2y ago
    No, in time it’s going to look just like that but with hallucinations and large areas of the codebase no human has even read. No doubt digging through code and trying to understand will involve a lot more prompting back and forth
    - dragonwriter 2y ago
      > No, in time it’s going to look just like that but with hallucinations and large areas of the codebase no human has even read. An area of code that no human has ever read is no different to me than an area of code lots of humans who aren't me have read, and that's not that different from an area of code that I have read, but not particularly recently. I mean, except for who it is that I am going to be swearing about when it breaks and I finally do read it.
      - onemoresoop 2y ago
        An area of code that’s been read a lot is likely to be like a paved road. Sure, it could have cracks or mud but it’s generally understood and really bad warts fixed or warning signs left behind. An area of code that runs rarely and is inspected even more rarely is likely to have bring more surprises and unexpected behaviors.
  - deadbabe 2y ago
    Legacy code is written by humans, and when you read stuff written by humans you can anticipate what their intent was because you are a human as well. With AI, there is no intent, the AI is a probabilistic model and you have to guess if the model got it right. It’s like the difference between driving next to a human driver vs a self driving car. You have some idea of what the human will do but the self driving car could flip out at any moment and do something irrational.
    - dragonwriter 2y ago
      > Legacy code is written by humans, and when you read stuff written by humans you can anticipate what their intent was because you are a human as well. Well, no. This is both technically categorically false (because you can only "anticipate" something in advance, and the intent is in the past by the time you are reading the code, though you might in principal infer their intent), and often practically false even when read with "infer" in place of "anticipate", as it is quite common for the intent of human-written code to be non-obvious, except in the sense of "the purpose of a system is what it does", in which sense AI-written code is not particularly more opaque.
- dragonwriter 2y ago
  > And one thing people don’t realize is that if you’re working in a codebase heavily coded by AI, you’re going to want to see the original prompts used if you don’t know the original engineers intent. If I'm working in a codebase that is heavily coded by anyone who isn't "me, within the last week", I'm going to want to see the implementation-technology-neutral requirements (and, ideally, human-written tests/test scenarios, unit and acceptance and everywhere in between, operationalizing the requirements) because I can't know the original engineer's intent. This isn't any less true when the engine was wet biological goop instead of silicon.