4 ms·

Is it cost effective? If companies think that this way of working is cutting development time and/or cost then it's the future. At least in the professional wor

Is it cost effective? If companies think that this way of working is cutting development time and/or cost then it's the future. At least in the professional world. Even bugs are acceptable if the advantages of auto generated code are large enough.

ModernMech 4y ago
Depends on the bug and how much damage it causes, I guess. The more nefarious and subtle the bug, the more intimate you have to be with the code to understand it. For developers, this intimacy comes with actually writing the code. I know that when I write code manually, I'm not just tapping keys on a keyboard but building a mental model in my brain. When there's a bug, I can usually intuit where it's located if that model is detailed enough. Maybe developers of the future will just be code reviewers for ML models. Yuck, who wants to do that? So then will we just have ML models reviewing ML generated code? My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source. What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.
- spc476 4y ago
  It's the old "C vs. Assembly" all over again.
  - lazide 4y ago
    Not quite - C is predicable(ish). It would be like C vs Asm if there was no stable language spec, platform spec, or compiler. I think it’s more like WYSIWYG website generators (anyone remember Cold Fusion? The biz guys LOVED that - for like a year).
    - galdosdi 4y ago
      Cold Fusion isn't WYSIWYG (although it was intended to eventually be, and that never panned out haha), are you thinking of DreamWeaver or MS FrontPage?
      - lazide 4y ago
        I must be remembering the marketing copy and not how well it worked out.
  - mort96 4y ago
    Except that it's not. I can be reasonably sure that my C compiler will generate correct machine code if my C code is correct. I can also treat the generated machine code as a build artifact which can be thrown away and re-generated on demand; I can treat my C code as my only source of truth. None of these things are true for Copilot, you need to keep around and maintain the generated code, and you can't just assume that the generated code is correct if your input is correct.
mort96 4y ago
I don't disagree with that. My comment is more intended to be read as, "Is this the future we want?" rather than, "This doesn't seem like something which will take off".
- spc476 4y ago
  I don't think "we" have any say in this---it's a question of whether this is what the people with money want.
- lazide 4y ago
  Except for companies that are ‘Software Eng is the core business’ anyway, which is super rare, I doubt anyone on the business side cares if we want it. They care if it is cost effective/works. Which I personally suspect this will end up about as well as WYSIWYG HTML editors/site generators. Which do have a niche. That would be the ‘makes a lot of cheap crap quickly, but no one uses it for anything serious or at scale’ niche. Because what comes out the other end/under the hood is reprocessed garbage that only works properly in a very narrow set of circumstances. I suspect with this market correction however we’ll see a decrease in perceived value for many of these tools, as dev pay is going to be going down (or stay the same despite inflation), and dev availability is going to go up.
  - janoc 4y ago
    You can't have cost effective when the code doesn't work because it has been generated by a monkey that doesn't understand neither the logic nor the requirements. That the monkey is actually a computer and not an actual banana eating primate doesn't change anything there. Any business fool that tries this will discover this very quickly - that's like claiming we don't need drivers because we have cruise control in our cars! It is not perfect but it is cost effective/works (some of the time ...)!
    - aaaaaaaaaaab 4y ago
      Why are you comparing Indians to monkeys? That's totally not ok.
joe_the_user 4y ago
If companies think that this way of working is cutting development time and/or cost then it's the future. I think most companies' management would still leave decisions like that in the hands of developers/development-teams. At the level of management, I think companies aren't asking for "code" but for results (a website, an application, a feature). Results with blemishes or insecurities are fine but a stream of entirely broken code wouldn't be seen as cost effective.
- shagie 4y ago
  If shortsighted management can offshore to the lowest bidder, they can believe that a few junior devs and ML code generation can replace a team of seniors. Sure, they can leave the decisions of frameworks in the hands of developers and development teams, but if those people are asked to justify the budgetary costs when compared to ML code generation it is no longer a technical argument. The problem always comes later (often after the manager has moved on and is no longer aware of the mistakes that were made and the repercussions can't follow) where the maintenance costs of poorly written initial code come back to haunt the origination for years to come.

ModernMech 4y ago

Depends on the bug and how much damage it causes, I guess. The more nefarious and subtle the bug, the more intimate you have to be with the code to understand it. For developers, this intimacy comes with actually writing the code. I know that when I write code manually, I'm not just tapping keys on a keyboard but building a mental model in my brain. When there's a bug, I can usually intuit where it's located if that model is detailed enough. Maybe developers of the future will just be code reviewers for ML models. Yuck, who wants to do that? So then will we just have ML models reviewing ML generated code? My concern is that ML code generation will create very shallow mental models of code in developers' minds. If everyone in the organization is building code this way, then who has the knowledge and skills to debug it? I foresee showstopping bugs from ML-generated that bring entire organizations to a grinding halt in the future. I remember back in the day when website generators were in vogue, it became a trend for many webdevs to call their sites "handmade" if they typed all the code themselves. I predict we'll be seeing the same in the future in other domains to contrast "handwritten" source with ML generated source. What's that famous quote? "Debugging is twice as hard as writing the code in the first place." Well where does that leave us when you don't even have to write the code in the first place? I guess then we will have to turn to ML to handle that for us as well.

spc476 4y ago
It's the old "C vs. Assembly" all over again.
- lazide 4y ago
  Not quite - C is predicable(ish). It would be like C vs Asm if there was no stable language spec, platform spec, or compiler. I think it’s more like WYSIWYG website generators (anyone remember Cold Fusion? The biz guys LOVED that - for like a year).
  - galdosdi 4y ago
    Cold Fusion isn't WYSIWYG (although it was intended to eventually be, and that never panned out haha), are you thinking of DreamWeaver or MS FrontPage?
    - lazide 4y ago
      I must be remembering the marketing copy and not how well it worked out.
- mort96 4y ago
  Except that it's not. I can be reasonably sure that my C compiler will generate correct machine code if my C code is correct. I can also treat the generated machine code as a build artifact which can be thrown away and re-generated on demand; I can treat my C code as my only source of truth. None of these things are true for Copilot, you need to keep around and maintain the generated code, and you can't just assume that the generated code is correct if your input is correct.

mort96 4y ago

I don't disagree with that. My comment is more intended to be read as, "Is this the future we want?" rather than, "This doesn't seem like something which will take off".

spc476 4y ago
I don't think "we" have any say in this---it's a question of whether this is what the people with money want.
lazide 4y ago
Except for companies that are ‘Software Eng is the core business’ anyway, which is super rare, I doubt anyone on the business side cares if we want it. They care if it is cost effective/works. Which I personally suspect this will end up about as well as WYSIWYG HTML editors/site generators. Which do have a niche. That would be the ‘makes a lot of cheap crap quickly, but no one uses it for anything serious or at scale’ niche. Because what comes out the other end/under the hood is reprocessed garbage that only works properly in a very narrow set of circumstances. I suspect with this market correction however we’ll see a decrease in perceived value for many of these tools, as dev pay is going to be going down (or stay the same despite inflation), and dev availability is going to go up.
- janoc 4y ago
  You can't have cost effective when the code doesn't work because it has been generated by a monkey that doesn't understand neither the logic nor the requirements. That the monkey is actually a computer and not an actual banana eating primate doesn't change anything there. Any business fool that tries this will discover this very quickly - that's like claiming we don't need drivers because we have cruise control in our cars! It is not perfect but it is cost effective/works (some of the time ...)!
  - aaaaaaaaaaab 4y ago
    Why are you comparing Indians to monkeys? That's totally not ok.

joe_the_user 4y ago

If companies think that this way of working is cutting development time and/or cost then it's the future. I think most companies' management would still leave decisions like that in the hands of developers/development-teams. At the level of management, I think companies aren't asking for "code" but for results (a website, an application, a feature). Results with blemishes or insecurities are fine but a stream of entirely broken code wouldn't be seen as cost effective.

shagie 4y ago
If shortsighted management can offshore to the lowest bidder, they can believe that a few junior devs and ML code generation can replace a team of seniors. Sure, they can leave the decisions of frameworks in the hands of developers and development teams, but if those people are asked to justify the budgetary costs when compared to ML code generation it is no longer a technical argument. The problem always comes later (often after the manager has moved on and is no longer aware of the mistakes that were made and the repercussions can't follow) where the maintenance costs of poorly written initial code come back to haunt the origination for years to come.