9 ms·

> code that's only 95% right is just wrong, I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs

by edanm 4y ago

> code that's only 95% right is just wrong, I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs, big bugs, critical bugs, security bugs, everything. No code is immune. The largest software used by millions every day has bugs. Library code that has existed and been in use for 30 years has bugs. I don't think you were actually thinking of this in your comparison, but I think it's actually a great analogy - code, like art, can be 95% complete, and that's usually enough. (For art, looks good and is what I wanted is enough, for code, does what I want right now, nevermind edge cases is enough.)

CapmCrackaWaka 4y ago
This depends entirely on _how_ the code is wrong. I asked chatGPT to write me code in python that would calculate SHAP values when given a sklearn model the other day. It returned code that ran, and even _looked_ like it did the right thing at a cursory glance. But I've written SHAP a package before, and there were several manipulations it got wrong. I mean completely wrong. You would never have known the code was wrong unless you knew how to write the code in the first place. To me, code that is 95% correct will either fail catastrophically or give very wrong results. Imagine if the code you wrote was off 5% for every number it was supposed to generate. Code that is 99.99% correct will introduce subtle bugs. * No shade to chatGPT, writing a function that calculates shap values is tough lol, I just wanted to see what it could do. I do think that, given time, it'll be able to write a days worth of high quality code in a few seconds.
- nmfisher 4y ago
  Over the weekend I tried to tease out a sed command that would fix an uber simple compiler error from ChatGPT [0]. I gave up after 4 or 5 tries - while it got the root cause correct ("." instead of "->" because the property was a pointer), it just couldn't figure out the right sed command. That's such a simple task, its failure doesn't inspire confidence in getting more complicated things correct. This is the main reason I haven't actually incorporated any AI tools into my daily programming yet - I'm mindful that I might end up spending more time tracking down issues in the auto-generated code than I saved using it in the first place. [0] You can see the results here https://twitter.com/NickFisherAU/status/1601838829882986496 https://twitter.com/NickFisherAU/status/1601838829882986496
- Workaccount2 4y ago
  The thing about ChatGPT is that it warning shot. And all these people I see talking about it, laughing about how the shooter missed them. Clearly ChatGPT is going to improve, and AI development is moving at a breakneck pace and accelerating. Dinging it for totally fumbling 5% or 10% of written code is completely missing the forest for the trees.
  - jhbadger 4y ago
    Sure, it will improve, but I think a lot of people think "Hey, it almost looks human quality now! Just a bit more tweaking and it will be human quality or better!". But a more likely case is that the relatively simple statistical modeling tools (which are very different from how our brains work, not that we fully understand how our brains work) that chatGPT uses have a limit to how well they work and they will hit a plateau (and are probably near it now). I'm not one of those people who believe strong AI is impossible, but I have a feeling that strong AI will take more than that just manipulating a text corpus.
    - ben_w 4y ago
      I'd be surprised if it did only take text (or even language in general), but if it does only need that, then given how few parameters even big GPT-3 models have compared to humans, it will strongly imply that PETA was right all along.
  - throwaway82388 4y ago
    Anyone who has doubts has to look at the price. It’s free for now, and will be cheap enough when openai starts monetizing. Price wins over quality. It’s demonstrated time and time again.
    - ben_w 4y ago
      Depends on the details. Skip all the boring health and safety steps, you can make very cheap skyscrapers. They might fall down in a strong wind, but they'll be cheap.
- KIFulgore 4y ago
  I experienced ChatGPT confidently giving incorrect answers about the Schwarzchild radius of the black hole at the center of our galaxy, Saggitarius A-star. Both when asked about "the Scharzchild radius of a black hole with 4 million solar masses" (a calculation) and "the Scharzchild radius of Saggitarius A-star" (a simple lookup). Both answers were orders of magnitude wrong, and vastly different from each other. JS code suggested for a simple database connection had glaring SQL injection vulnerabilities. I think it's an ok tool for discovering new libraries and getting oriented quickly to languages and coding domains you're unfamiliar with. But it's more like a forum post from a novice who read a tutorial and otherwise has little experience.
  - mcguire 4y ago
    My understanding is that ChatGPT (and similar things) are purely language models; they do not have any kind of "understanding" of anything like reality. Basically, they have a complex statistical model of how words are related. I'm a bit surprised that it got a lookup wrong, but for any other domain, describing it as a "novice" is understating the situation a lot.
- malandrew 4y ago
  Who is going to debug this code when it is wrong? Whether 95% or 99.9% correct, when there is a serious bug, you're still going to need people that can fix the gap between almost correct and actually correct.
  - cool_dude85 4y ago
    Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed? Now maybe instead of 5 FTE equivalents doing that work, you just need the 1 guy to debug the AI's shot at it. Now 4 people are out of work, or applying to be the 1 guy at some other company.
    - mcguire 4y ago
      Does anyone remember the old maxim, "Don't write code as cleverly as you can because it's harder to debug than it is to write and you won't be clever enough"?
    - woah 4y ago
      Or the company just delivers features when they are estimated to be done, instead of it taking 5 times longer than expected
    - lmm 4y ago
      > Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed? It may be a significant chunk of the butt-in-seat-time under our archaic 40hour/week paradigm, but it's not a significant chunk of the programmer's actual mental effort. You're not going to be able to get people to work 5x more intensely by automating the boring stuff, that was never the limiting factor.
mejutoco 4y ago
I agree with you. Even software that had no bugs today (if that is possible) could start having bugs tomorrow, as the environment changes (new law, new hardware, etc.)
GoblinSlayer 4y ago
And GPT can't fix a bug, it can only generate new text that will have a different collection of bugs. The catch is that programming isn't text generation. But AI should be able to make good actually intelligent fuzzers, that should be realistic and useful.
- mlboss 4y ago
  It is only a matter of time. It can understand error stacktrace and suggest a fix. Somebody has to plug it to IDE then it will start converting requirements to code.
- Ajedi32 4y ago
  > GPT can't fix a bug It can't? I could've sworn I've seen (cherry-picked) examples of it doing exactly that, when prompted. It even explains what the bug is and why the fix works.
  - ipaddr 4y ago
    Which examples the ones where they were right or wrong. It goes back to trusting the source not to introduce new ever evolving bugs.
  - soerxpso 4y ago
    Those are cherry picked, and most importantly, all of the examples where it can fix a bug are examples where it's working with a stack trace, or with an extremely small section of code (<200 lines). At what point will it be able to fix a bug in a 20,000 line codebase, with only "When the user does X, Y unintended consequence happens" to go off of? It's obvious how an expert at regurgitating StackOverflow would be able to correct an NPE or an off-by-one error when given the exact line of code that error is on. Going any deeper, and actually being able to find a bug, requires understanding of the codebase as a whole and the ability to map the code to what the code actually does in real life. GPT has shown none of this. "But it will get better over time" arguments fail for this because the thing that's needed is a fundamentally new ability, not just "the same but better." Understanding a codebase is a different thing from regurgitating StackOverflow. It's the same thing as saying in 1980, "We have bipedal robots that can hobble, so if we just improve on that enough we'll eventually have bipedal robots that beat humans at football."
CuriouslyC 4y ago
Two issues. First, when a human gets something 5% wrong, it's more likely to be a corner case or similar "right most of the time" scenario, whereas when AI gets something 5% wrong, it's likely to look almost right but never produce correct output. Second, when a human writes something wrong they have familiarity with the code and can more easily identify the problem and fix it, whereas fixing AI code (either via human or AI) is more likely to be fraught.
- edanm 4y ago
  You (and everyone else) seem to be making the classic "mistake" of looking at an early version and not appreciating that things improve. Ten years ago, AI-generated art was at 50%. 2 years ago, 80%. Now it's at 95% and winning competitions. I have no idea if the AI that's getting code 80% right today will get it 95% right in two years, but given current progress, I wouldn't bet against it. I don't think there's any fundamental reason it can't produce better code than I can, at least not at the "write a function that does X" level. Whole systems are a way harder problem that I wouldn't even think of making guesses about.
  - yamtaddle 4y ago
    To be fair to those assumptions, there've been a lot of cases of machine-learning (among other tech) looking very promising, and advancing so quickly that a huge revolution seems imminent—then stalling out at a local maximum for a really long time.
  - ben_w 4y ago
    It might improve like Go AI and shock everyone by beating the world expert at everything, or it might improve like Tesla FSD which is annoyingly harder than "make creative artwork". There's no fundamental reason it can't be the world expert at everything, but that's not a reason to assume we know how to get there from here.
    - namelessoracle 4y ago
      What scares me is a death of progress situation. Maybe it cant be an expert, but it can be good enough, and now the supply pipeline of people who could be experts basically gets shut off, because to become an expert you needed to do the work and gain the experiences that are now completely owned by AI.
snickerbockers 4y ago
Maybe for certain domains it's okay to fail 5% of the time but a lot of code really does need to be perfect. You wouldn't be able to work with a filesystem that loses 5% of your files.
- mecsred 4y ago
  Or a filesystem that loses all of your files 5% of the time.
  - scarmig 4y ago
    No need to rag on btrfs.
tablespoon 4y ago
>> code that's only 95% right is just wrong, > I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs, big bugs, critical bugs, security bugs, everything. No code is immune. The largest software used by millions every day has bugs. Library code that has existed and been in use for 30 years has bugs. All software has bugs, but it's usually far better that "95% right." Code that's only 95% right probably wouldn't pass half-ass testing or a couple of days of actual use.
mr_toad 4y ago
When AI can debug its own code I’ll start looking for another career.
- ben_w 4y ago
  When it can do that, it's already too late.
scotty79 4y ago
Fixing the last 5% requires that you understand 100% of all. And understanding is the main value added by programmer, not typing characters into text editor.

by edanm 4y ago

CapmCrackaWaka 4y ago

This depends entirely on _how_ the code is wrong. I asked chatGPT to write me code in python that would calculate SHAP values when given a sklearn model the other day. It returned code that ran, and even _looked_ like it did the right thing at a cursory glance. But I've written SHAP a package before, and there were several manipulations it got wrong. I mean completely wrong. You would never have known the code was wrong unless you knew how to write the code in the first place. To me, code that is 95% correct will either fail catastrophically or give very wrong results. Imagine if the code you wrote was off 5% for every number it was supposed to generate. Code that is 99.99% correct will introduce subtle bugs. * No shade to chatGPT, writing a function that calculates shap values is tough lol, I just wanted to see what it could do. I do think that, given time, it'll be able to write a days worth of high quality code in a few seconds.

nmfisher 4y ago
Over the weekend I tried to tease out a sed command that would fix an uber simple compiler error from ChatGPT [0]. I gave up after 4 or 5 tries - while it got the root cause correct ("." instead of "->" because the property was a pointer), it just couldn't figure out the right sed command. That's such a simple task, its failure doesn't inspire confidence in getting more complicated things correct. This is the main reason I haven't actually incorporated any AI tools into my daily programming yet - I'm mindful that I might end up spending more time tracking down issues in the auto-generated code than I saved using it in the first place. [0] You can see the results here https://twitter.com/NickFisherAU/status/1601838829882986496 https://twitter.com/NickFisherAU/status/1601838829882986496
Workaccount2 4y ago
The thing about ChatGPT is that it warning shot. And all these people I see talking about it, laughing about how the shooter missed them. Clearly ChatGPT is going to improve, and AI development is moving at a breakneck pace and accelerating. Dinging it for totally fumbling 5% or 10% of written code is completely missing the forest for the trees.
- jhbadger 4y ago
  Sure, it will improve, but I think a lot of people think "Hey, it almost looks human quality now! Just a bit more tweaking and it will be human quality or better!". But a more likely case is that the relatively simple statistical modeling tools (which are very different from how our brains work, not that we fully understand how our brains work) that chatGPT uses have a limit to how well they work and they will hit a plateau (and are probably near it now). I'm not one of those people who believe strong AI is impossible, but I have a feeling that strong AI will take more than that just manipulating a text corpus.
  - ben_w 4y ago
    I'd be surprised if it did only take text (or even language in general), but if it does only need that, then given how few parameters even big GPT-3 models have compared to humans, it will strongly imply that PETA was right all along.
- throwaway82388 4y ago
  Anyone who has doubts has to look at the price. It’s free for now, and will be cheap enough when openai starts monetizing. Price wins over quality. It’s demonstrated time and time again.
  - ben_w 4y ago
    Depends on the details. Skip all the boring health and safety steps, you can make very cheap skyscrapers. They might fall down in a strong wind, but they'll be cheap.
KIFulgore 4y ago
I experienced ChatGPT confidently giving incorrect answers about the Schwarzchild radius of the black hole at the center of our galaxy, Saggitarius A-star. Both when asked about "the Scharzchild radius of a black hole with 4 million solar masses" (a calculation) and "the Scharzchild radius of Saggitarius A-star" (a simple lookup). Both answers were orders of magnitude wrong, and vastly different from each other. JS code suggested for a simple database connection had glaring SQL injection vulnerabilities. I think it's an ok tool for discovering new libraries and getting oriented quickly to languages and coding domains you're unfamiliar with. But it's more like a forum post from a novice who read a tutorial and otherwise has little experience.
- mcguire 4y ago
  My understanding is that ChatGPT (and similar things) are purely language models; they do not have any kind of "understanding" of anything like reality. Basically, they have a complex statistical model of how words are related. I'm a bit surprised that it got a lookup wrong, but for any other domain, describing it as a "novice" is understating the situation a lot.
malandrew 4y ago
Who is going to debug this code when it is wrong? Whether 95% or 99.9% correct, when there is a serious bug, you're still going to need people that can fix the gap between almost correct and actually correct.
- cool_dude85 4y ago
  Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed? Now maybe instead of 5 FTE equivalents doing that work, you just need the 1 guy to debug the AI's shot at it. Now 4 people are out of work, or applying to be the 1 guy at some other company.
  - mcguire 4y ago
    Does anyone remember the old maxim, "Don't write code as cleverly as you can because it's harder to debug than it is to write and you won't be clever enough"?
  - woah 4y ago
    Or the company just delivers features when they are estimated to be done, instead of it taking 5 times longer than expected
  - lmm 4y ago
    > Sure, but how much of the total work time in software development is writing relatively straightforward, boilerplate type code that could reasonably be copied from the top answer from stackoverflow with variable names changed? It may be a significant chunk of the butt-in-seat-time under our archaic 40hour/week paradigm, but it's not a significant chunk of the programmer's actual mental effort. You're not going to be able to get people to work 5x more intensely by automating the boring stuff, that was never the limiting factor.

mejutoco 4y ago

I agree with you. Even software that had no bugs today (if that is possible) could start having bugs tomorrow, as the environment changes (new law, new hardware, etc.)

GoblinSlayer 4y ago

And GPT can't fix a bug, it can only generate new text that will have a different collection of bugs. The catch is that programming isn't text generation. But AI should be able to make good actually intelligent fuzzers, that should be realistic and useful.

mlboss 4y ago
It is only a matter of time. It can understand error stacktrace and suggest a fix. Somebody has to plug it to IDE then it will start converting requirements to code.
Ajedi32 4y ago
> GPT can't fix a bug It can't? I could've sworn I've seen (cherry-picked) examples of it doing exactly that, when prompted. It even explains what the bug is and why the fix works.
- ipaddr 4y ago
  Which examples the ones where they were right or wrong. It goes back to trusting the source not to introduce new ever evolving bugs.
- soerxpso 4y ago
  Those are cherry picked, and most importantly, all of the examples where it can fix a bug are examples where it's working with a stack trace, or with an extremely small section of code (<200 lines). At what point will it be able to fix a bug in a 20,000 line codebase, with only "When the user does X, Y unintended consequence happens" to go off of? It's obvious how an expert at regurgitating StackOverflow would be able to correct an NPE or an off-by-one error when given the exact line of code that error is on. Going any deeper, and actually being able to find a bug, requires understanding of the codebase as a whole and the ability to map the code to what the code actually does in real life. GPT has shown none of this. "But it will get better over time" arguments fail for this because the thing that's needed is a fundamentally new ability, not just "the same but better." Understanding a codebase is a different thing from regurgitating StackOverflow. It's the same thing as saying in 1980, "We have bipedal robots that can hobble, so if we just improve on that enough we'll eventually have bipedal robots that beat humans at football."

CuriouslyC 4y ago

Two issues. First, when a human gets something 5% wrong, it's more likely to be a corner case or similar "right most of the time" scenario, whereas when AI gets something 5% wrong, it's likely to look almost right but never produce correct output. Second, when a human writes something wrong they have familiarity with the code and can more easily identify the problem and fix it, whereas fixing AI code (either via human or AI) is more likely to be fraught.

edanm 4y ago
You (and everyone else) seem to be making the classic "mistake" of looking at an early version and not appreciating that things improve. Ten years ago, AI-generated art was at 50%. 2 years ago, 80%. Now it's at 95% and winning competitions. I have no idea if the AI that's getting code 80% right today will get it 95% right in two years, but given current progress, I wouldn't bet against it. I don't think there's any fundamental reason it can't produce better code than I can, at least not at the "write a function that does X" level. Whole systems are a way harder problem that I wouldn't even think of making guesses about.
- yamtaddle 4y ago
  To be fair to those assumptions, there've been a lot of cases of machine-learning (among other tech) looking very promising, and advancing so quickly that a huge revolution seems imminent—then stalling out at a local maximum for a really long time.
- ben_w 4y ago
  It might improve like Go AI and shock everyone by beating the world expert at everything, or it might improve like Tesla FSD which is annoyingly harder than "make creative artwork". There's no fundamental reason it can't be the world expert at everything, but that's not a reason to assume we know how to get there from here.
  - namelessoracle 4y ago
    What scares me is a death of progress situation. Maybe it cant be an expert, but it can be good enough, and now the supply pipeline of people who could be experts basically gets shut off, because to become an expert you needed to do the work and gain the experiences that are now completely owned by AI.

snickerbockers 4y ago

Maybe for certain domains it's okay to fail 5% of the time but a lot of code really does need to be perfect. You wouldn't be able to work with a filesystem that loses 5% of your files.

mecsred 4y ago
Or a filesystem that loses all of your files 5% of the time.
- scarmig 4y ago
  No need to rag on btrfs.

tablespoon 4y ago

>> code that's only 95% right is just wrong, > I know what you mean, but thinking about it critically, this is just wrong. All software has bugs in it. Small bugs, big bugs, critical bugs, security bugs, everything. No code is immune. The largest software used by millions every day has bugs. Library code that has existed and been in use for 30 years has bugs. All software has bugs, but it's usually far better that "95% right." Code that's only 95% right probably wouldn't pass half-ass testing or a couple of days of actual use.

mr_toad 4y ago

When AI can debug its own code I’ll start looking for another career.

ben_w 4y ago
When it can do that, it's already too late.

scotty79 4y ago

Fixing the last 5% requires that you understand 100% of all. And understanding is the main value added by programmer, not typing characters into text editor.