4 ms·

Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe

by Grimm1 6y ago

Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers. I think it'll make neat autopilot functionality for developers but not replace the need to have someone look over and understand the code.

perl4ever 6y ago
This is a perfect satire of the logic people use to advocate self driving cars being rushed into production. Only every time I read something similar, I think "surely no programmer could think this". Are you a programmer?
- Grimm1 6y ago
  I sure am, and if I can code gen 90% of the boiler plate away I'll do it happily. Besides attacking me, do you have any point you'd like to make?
  - bobthebuilders 6y ago
    Do you want to die when your self driving car crashes? Debug issues when your app des at 12am? Same concept.
    - Grimm1 6y ago
      I don't want to die when I crash my own car, and I already debug my own apps at 12am. If your argument is that things need to be perfect than my god you must never leave your home! I'd trust a machine to drive more accurately than most people I see on the highway. Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things. That considered, I had a pretty strict qualifier in my above post which means the machine must perform better than the average human in the respective task and therefore I'd be more likely to die driving my own car than a machine meeting my prerequisites.
      - Judgmentality 6y ago
        > I'd trust a machine to drive more accurately than most people I see on the highway. Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things. Humans are much, much, much more capable than the absolute state-of-the-art robots when it comes to doing things in an uncontrolled environment. https://www.youtube.com/watch?v=g0TaYhjpOfo https://www.youtube.com/watch?v=g0TaYhjpOfo
        Hasnep 6y ago
        One of the advantages of an autonomous driver is that its superhuman reflexes, never driving while tired, never getting road rage, etc., will make it less likely to get into an uncontrolled environment. Would you prefer your pilots to fly your plane with no AI assistance?
        Judgmentality 6y ago
        > One of the advantages of an autonomous driver is that its superhuman reflexes, never driving while tired, never getting road rage, etc. First of all, when you actually understand a self-driving car stack, you'll realize those super-human reflexes are more human than you think. The stack is complicated and not only are there delays to be expected, some hardware syncing requirements guarantee certain delays in the perception pipeline. It's still better than a person, but it's nothing close to approaching instantaneous. Likewise, sensors can get dirty, and blah blah blah there are other weaknesses robots have that humans don't. My point is simple: robots aren't perfect. In fact, they are almost always much worse than most people realize. > will make it less likely to get into an uncontrolled environment You're misunderstanding me. I'm not saying less likely to get into an accident. I'm saying the world, where cars drive, is an uncontrolled environment - and the current state of robotics is such that humans are better for doing things in the real world. There is no "less likely to get into an uncontrolled environment" because by definition you are always putting it into that situation. > Would you prefer your pilots to fly your plane with no AI assistance? AI assistance is fine. AI replacement is not.
        6y ago
kulig 6y ago
Its not that simple. People are understanding when car crashes happen in busy roads amongst other cars. They are not understanding if a self-driving car swerves into the sidewalk and kills a group of children.
ben_w 6y ago
I disagree that that is enough to be useful. To give a deliberately extreme example: if it produces code which has half the number of bugs as a human, but it only outputs Malbolge source code, nobody else will be able to fix those bugs which remain.
yuliyp 6y ago
This is a very simplistic of what code is and the role it plays in a system. There are many implementations that can fulfill a set of requirements. Not all of them are created equal. The ways in which they behave as the system changes can be wildly different. Well-written code will be able to handle those changes gracefully. Poorly-written code may end up proving brittle and bug-prone. Generated code will be completely unpredictable. Imagine you're trying to build a street network for a city. Some designs are much more predictable than others. If you've played Factorio, the distinction between a spaghetti base and one that has some design is abundant. Even if they currently fulfill the same requirements now, the ability to improve upon and reason about how it will behave after changes is vastly different.
- Grimm1 6y ago
  I don't know what you're arguing against but it sure isn't what I wrote. "Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers." Point to the part you're arguing against because you way extrapolated what "have usefulness" means I think.
  - yuliyp 6y ago
    It's of limited usefulness if it just leads to a system that will end up performing worse / less predictably.
danfang 6y ago
This is naive. The point is that code is a well defined system with clear rules that can be expressed through logic and mathematics. GPT is suited to approximate systems where the rules are not well defined. Until AI can actually learn the principles of logic, it may not be useful for code generation on a meaningful scale, other than things just like simple auto-completions. Not only that, AI would also have to learn the principles of system design, performance, security, readability, maintainability. That's what makes "good" software. It's a far stretch to say that AI could achieve anything of the sort based on current abilities.
- Grimm1 6y ago
  Good thing I didn't say it would, I'm really not sure what you think I wrote.
  - danfang 6y ago
    Your only metric of "good" code generation is # of bugs. My assertion was that this is not accurate. I also disagree with the premise that code generation by AI will be very useful for programmers, for the reasons stated above.
    - Grimm1 6y ago
      It's clearly not if you read the rest of that sentence, that's my metric for it to be useful. My metric for it being good is much deeper. Which is why I questioned what you responded with. We are already at the point of useful and context aware code generation anyway which is why I've found everyone on this thread questioning it to be kind of funny, Microsoft was demoing complex generations a year ago. So we're well on our way. https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb?t=29m19s https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb?t=29m19s https://www.infoworld.com/article/3518429/jetbrains-taps-machine-learning-for-full-line-code-completion.html https://www.infoworld.com/article/3518429/jetbrains-taps-mac...

by Grimm1 6y ago

perl4ever 6y ago

This is a perfect satire of the logic people use to advocate self driving cars being rushed into production. Only every time I read something similar, I think "surely no programmer could think this". Are you a programmer?

Grimm1 6y ago
I sure am, and if I can code gen 90% of the boiler plate away I'll do it happily. Besides attacking me, do you have any point you'd like to make?
- bobthebuilders 6y ago
  Do you want to die when your self driving car crashes? Debug issues when your app des at 12am? Same concept.
  - Grimm1 6y ago
    I don't want to die when I crash my own car, and I already debug my own apps at 12am. If your argument is that things need to be perfect than my god you must never leave your home! I'd trust a machine to drive more accurately than most people I see on the highway. Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things. That considered, I had a pretty strict qualifier in my above post which means the machine must perform better than the average human in the respective task and therefore I'd be more likely to die driving my own car than a machine meeting my prerequisites.
    - Judgmentality 6y ago
      > I'd trust a machine to drive more accurately than most people I see on the highway. Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things. Humans are much, much, much more capable than the absolute state-of-the-art robots when it comes to doing things in an uncontrolled environment. https://www.youtube.com/watch?v=g0TaYhjpOfo https://www.youtube.com/watch?v=g0TaYhjpOfo
      - Hasnep 6y ago
        One of the advantages of an autonomous driver is that its superhuman reflexes, never driving while tired, never getting road rage, etc., will make it less likely to get into an uncontrolled environment. Would you prefer your pilots to fly your plane with no AI assistance?
        Judgmentality 6y ago
        > One of the advantages of an autonomous driver is that its superhuman reflexes, never driving while tired, never getting road rage, etc. First of all, when you actually understand a self-driving car stack, you'll realize those super-human reflexes are more human than you think. The stack is complicated and not only are there delays to be expected, some hardware syncing requirements guarantee certain delays in the perception pipeline. It's still better than a person, but it's nothing close to approaching instantaneous. Likewise, sensors can get dirty, and blah blah blah there are other weaknesses robots have that humans don't. My point is simple: robots aren't perfect. In fact, they are almost always much worse than most people realize. > will make it less likely to get into an uncontrolled environment You're misunderstanding me. I'm not saying less likely to get into an accident. I'm saying the world, where cars drive, is an uncontrolled environment - and the current state of robotics is such that humans are better for doing things in the real world. There is no "less likely to get into an uncontrolled environment" because by definition you are always putting it into that situation. > Would you prefer your pilots to fly your plane with no AI assistance? AI assistance is fine. AI replacement is not.
        6y ago

kulig 6y ago

Its not that simple. People are understanding when car crashes happen in busy roads amongst other cars. They are not understanding if a self-driving car swerves into the sidewalk and kills a group of children.

ben_w 6y ago

I disagree that that is enough to be useful. To give a deliberately extreme example: if it produces code which has half the number of bugs as a human, but it only outputs Malbolge source code, nobody else will be able to fix those bugs which remain.

yuliyp 6y ago

This is a very simplistic of what code is and the role it plays in a system. There are many implementations that can fulfill a set of requirements. Not all of them are created equal. The ways in which they behave as the system changes can be wildly different. Well-written code will be able to handle those changes gracefully. Poorly-written code may end up proving brittle and bug-prone. Generated code will be completely unpredictable. Imagine you're trying to build a street network for a city. Some designs are much more predictable than others. If you've played Factorio, the distinction between a spaghetti base and one that has some design is abundant. Even if they currently fulfill the same requirements now, the ability to improve upon and reason about how it will behave after changes is vastly different.

Grimm1 6y ago
I don't know what you're arguing against but it sure isn't what I wrote. "Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers." Point to the part you're arguing against because you way extrapolated what "have usefulness" means I think.
- yuliyp 6y ago
  It's of limited usefulness if it just leads to a system that will end up performing worse / less predictably.

danfang 6y ago

This is naive. The point is that code is a well defined system with clear rules that can be expressed through logic and mathematics. GPT is suited to approximate systems where the rules are not well defined. Until AI can actually learn the principles of logic, it may not be useful for code generation on a meaningful scale, other than things just like simple auto-completions. Not only that, AI would also have to learn the principles of system design, performance, security, readability, maintainability. That's what makes "good" software. It's a far stretch to say that AI could achieve anything of the sort based on current abilities.

Grimm1 6y ago
Good thing I didn't say it would, I'm really not sure what you think I wrote.
- danfang 6y ago
  Your only metric of "good" code generation is # of bugs. My assertion was that this is not accurate. I also disagree with the premise that code generation by AI will be very useful for programmers, for the reasons stated above.
  - Grimm1 6y ago
    It's clearly not if you read the rest of that sentence, that's my metric for it to be useful. My metric for it being good is much deeper. Which is why I questioned what you responded with. We are already at the point of useful and context aware code generation anyway which is why I've found everyone on this thread questioning it to be kind of funny, Microsoft was demoing complex generations a year ago. So we're well on our way. https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb?t=29m19s https://www.pscp.tv/Microsoft/1OyKAYWPRrWKb?t=29m19s https://www.infoworld.com/article/3518429/jetbrains-taps-machine-learning-for-full-line-code-completion.html https://www.infoworld.com/article/3518429/jetbrains-taps-mac...