4 ms·

> Could we end up having AI quality trend downwards due to AI ingesting its own old outputs and reinforcing bad habits? I think it's a particular risk for text

> Could we end up having AI quality trend downwards due to AI ingesting its own old outputs and reinforcing bad habits? I think it's a particular risk for text generation. This is exactly what I don't like about Copilot, maybe even more than the IP ethics of it. If it really succeeds, it's going to have a feedback loop that amplifies its own code suggestions. The same boilerplate-ish kind of code that developers generate over and over will get calcified, even if it's suboptimal or buggy. If we really need robots to help write our code, to me that means we don't yet have expressive enough languages.

sillysaurusx 4y ago
Devs can use their noggins to check whether copilot’s output is decent or not. It’s not a given that people will use it automatically.
- boringg 4y ago
  That and I bet the generic boilerplate code that co-pilot produces is on average much better than the boiler plate code that the average dev might use. So it removes the lower level work.
  - spacephysics 4y ago
    I think that can surely be the case, but just like any AI there may need to be manual review to asses what is the most optimal way to go about task X, and retrain. I can see this go both ways: boilerplate code being a great generic solution to a set of problems, but a more seasoned programmer may say “that works, but for our use case the trade offs don’t make sense” Or alternatively, “this code wasn’t something I knew I could do in language X, and it’s far more efficient”
  - brookst 4y ago
    It is. There are a lot of problems with copilot, but one magical thing is the way it rewards best practices like thinking through what a function is supposed to do before starting to write it. If you write a good comment describing name, inputs, outputs, logic, and exceptions, the "generate code from comment" capability is kind of amazing. I'm a terrible, hacky programmer, and it has wholly converted me to documentation-first.
    - Cyberdog 4y ago
      The only Copilot-type system I'd consider using is one that did the opposite - generated comments from my code, so my lazy butt doesn't have to write them manually.
      - brookst 4y ago
        I hear ya, but I’m a PM so I’m looking at it as spec-to-code. Though I would love to see code-to-comments, especially really opinionated comments.
- spacephysics 4y ago
  I think many developers of today agree. Todays programmers can see copilot output and probably think “well that’s not optimal”. Fast forward five years, new CS grads are using Copilot 3.0, and are used to specific auto-completes that copilot gives for certain tasks, as they may have never needed to go beyond some of the more basic suggestions. It “feels” like an older programmer seeing a younger web dev and going “you’re wasting MB of memory!” While true the web has gotten slower in many regards, and indeed memory may have been wasted, business value creation typically doesn’t care if a few MB is sub optimally wasted, while the previous generation does.
  - mikewarot 4y ago
    >It “feels” like an older programmer seeing a younger web dev and going “you’re wasting MB of memory!” I crammed a gigabyte of ASCII into a string in FreePascal, it was glorious! Waste all the MB you want! (I'm 59)
  - partdavid 4y ago
    This is a whole class of problem, it seems to me. A semi-automated approach can seem fine when there's an executive function over the top of it, exercised by someone who has not just the knowledge, but current and honed knowledge. But over time, what's keeping that knowledge current and honed? The airline industry has talked about this, of course, and adoption of robotic surgery has opened up a whole new training problem, because its escape hatches when it goes wrong or can't complete a procedure is often "complete the surgery manually". Which is fine on Day 1 of robotic surgery, but what about day 2 when surgeons typically don't have hundreds of similar procedures under their belts? And where the only time they're called on to exercise the skill is in difficult edge cases? We have basically turned driving a standard transmission into a weird old person quirk or niche enthusiant skill in the United States. If an automatic transmission required a similar manual fallback or check, how well would that work? Well, it would work fine if basically everyone already had a lot of practice driving a manual--but now? It wouldn't work well at all. Of course, automatic transmission don't fail like that, and are a lot better at switching gears than AI assistants are at generating code. I worry about the semi-automated approach to self-driving, where the driver may not actually have currency with their driving skill, and where--in the instance that it's necessary--a driver has to react to more complicated situations (they don't have practice with the simple ones, and they have to react not to a hazard but to their car's failure to react to a hazard).
- harimau777 4y ago
  Until managers threaten to fire them for "wasting time" correcting the output.
  - dymk 4y ago
    Why do you worry that’s going to be a real problem?
    - bluefirebrand 4y ago
      Because managers already lean on engineers for "wasting time" fixing tech debt, building scalable solutions, etc.
- toss1 4y ago
  >>Devs can use their noggins Can =/= Will And as also pointed out by others, it not only requires effort, but knowledge, and that knowledge will be systematically degraded the more AI-ish code generation is used. OTOH, those with super-diligent hacker attitudes will start to learn how to find the flaws in generated code and optimize it, so leveraging the tool, but most will just move on to the next task/ticket as soon as the AIish code passes the unit tests. So, super-leveraging AI-generated code will be rare.
  - brookst 4y ago
    > And as also pointed out by others, it not only requires effort, but knowledge, and that knowledge will be systematically degraded the more AI-ish code generation is used. How is that different than the plague of junior devs we've always had? Devs will get more senior by identifying and correcting issues in AI code rather than code from their peers. Seems OK, like we just got a whole lot more coding capacity.
LesZedCB 4y ago
but enterprise FizzBuzz is a demonstration that exactly that phenomenon will happen without AI, merely with books, youtube videos, or blogs (the calcification) and cargo culting (the lazy application).
- oceanplexian 4y ago
  For real, most developers repeatedly copy/paste things they find on the internet without understanding how they work. So the AI isn’t doing anything special that humans don’t already do. Source: Bitter old SRE who has had to fix many broken software patterns ripped out of StackOverflow and the like.
smrtinsert 4y ago
Not really the case. I reject maybe 90٪ of copilot suggestions. If they leverage that I'm sure it will improve

sillysaurusx 4y ago

Devs can use their noggins to check whether copilot’s output is decent or not. It’s not a given that people will use it automatically.

boringg 4y ago
That and I bet the generic boilerplate code that co-pilot produces is on average much better than the boiler plate code that the average dev might use. So it removes the lower level work.
- spacephysics 4y ago
  I think that can surely be the case, but just like any AI there may need to be manual review to asses what is the most optimal way to go about task X, and retrain. I can see this go both ways: boilerplate code being a great generic solution to a set of problems, but a more seasoned programmer may say “that works, but for our use case the trade offs don’t make sense” Or alternatively, “this code wasn’t something I knew I could do in language X, and it’s far more efficient”
- brookst 4y ago
  It is. There are a lot of problems with copilot, but one magical thing is the way it rewards best practices like thinking through what a function is supposed to do before starting to write it. If you write a good comment describing name, inputs, outputs, logic, and exceptions, the "generate code from comment" capability is kind of amazing. I'm a terrible, hacky programmer, and it has wholly converted me to documentation-first.
  - Cyberdog 4y ago
    The only Copilot-type system I'd consider using is one that did the opposite - generated comments from my code, so my lazy butt doesn't have to write them manually.
    - brookst 4y ago
      I hear ya, but I’m a PM so I’m looking at it as spec-to-code. Though I would love to see code-to-comments, especially really opinionated comments.
spacephysics 4y ago
I think many developers of today agree. Todays programmers can see copilot output and probably think “well that’s not optimal”. Fast forward five years, new CS grads are using Copilot 3.0, and are used to specific auto-completes that copilot gives for certain tasks, as they may have never needed to go beyond some of the more basic suggestions. It “feels” like an older programmer seeing a younger web dev and going “you’re wasting MB of memory!” While true the web has gotten slower in many regards, and indeed memory may have been wasted, business value creation typically doesn’t care if a few MB is sub optimally wasted, while the previous generation does.
- mikewarot 4y ago
  >It “feels” like an older programmer seeing a younger web dev and going “you’re wasting MB of memory!” I crammed a gigabyte of ASCII into a string in FreePascal, it was glorious! Waste all the MB you want! (I'm 59)
- partdavid 4y ago
  This is a whole class of problem, it seems to me. A semi-automated approach can seem fine when there's an executive function over the top of it, exercised by someone who has not just the knowledge, but current and honed knowledge. But over time, what's keeping that knowledge current and honed? The airline industry has talked about this, of course, and adoption of robotic surgery has opened up a whole new training problem, because its escape hatches when it goes wrong or can't complete a procedure is often "complete the surgery manually". Which is fine on Day 1 of robotic surgery, but what about day 2 when surgeons typically don't have hundreds of similar procedures under their belts? And where the only time they're called on to exercise the skill is in difficult edge cases? We have basically turned driving a standard transmission into a weird old person quirk or niche enthusiant skill in the United States. If an automatic transmission required a similar manual fallback or check, how well would that work? Well, it would work fine if basically everyone already had a lot of practice driving a manual--but now? It wouldn't work well at all. Of course, automatic transmission don't fail like that, and are a lot better at switching gears than AI assistants are at generating code. I worry about the semi-automated approach to self-driving, where the driver may not actually have currency with their driving skill, and where--in the instance that it's necessary--a driver has to react to more complicated situations (they don't have practice with the simple ones, and they have to react not to a hazard but to their car's failure to react to a hazard).
harimau777 4y ago
Until managers threaten to fire them for "wasting time" correcting the output.
- dymk 4y ago
  Why do you worry that’s going to be a real problem?
  - bluefirebrand 4y ago
    Because managers already lean on engineers for "wasting time" fixing tech debt, building scalable solutions, etc.
toss1 4y ago
>>Devs can use their noggins Can =/= Will And as also pointed out by others, it not only requires effort, but knowledge, and that knowledge will be systematically degraded the more AI-ish code generation is used. OTOH, those with super-diligent hacker attitudes will start to learn how to find the flaws in generated code and optimize it, so leveraging the tool, but most will just move on to the next task/ticket as soon as the AIish code passes the unit tests. So, super-leveraging AI-generated code will be rare.
- brookst 4y ago
  > And as also pointed out by others, it not only requires effort, but knowledge, and that knowledge will be systematically degraded the more AI-ish code generation is used. How is that different than the plague of junior devs we've always had? Devs will get more senior by identifying and correcting issues in AI code rather than code from their peers. Seems OK, like we just got a whole lot more coding capacity.

LesZedCB 4y ago

but enterprise FizzBuzz is a demonstration that exactly that phenomenon will happen without AI, merely with books, youtube videos, or blogs (the calcification) and cargo culting (the lazy application).

oceanplexian 4y ago
For real, most developers repeatedly copy/paste things they find on the internet without understanding how they work. So the AI isn’t doing anything special that humans don’t already do. Source: Bitter old SRE who has had to fix many broken software patterns ripped out of StackOverflow and the like.

smrtinsert 4y ago

Not really the case. I reject maybe 90٪ of copilot suggestions. If they leverage that I'm sure it will improve