4 ms·

So would the AI generated code be like: 1) it’s new code that works well in 90-95% of the cases, but fails in unexpected ways and cannot be debugged, you have

by yumraj 5y ago

So would the AI generated code be like: 1) it’s new code that works well in 90-95% of the cases, but fails in unexpected ways and cannot be debugged, you have to generate a new piece of code that works in the above failure but has its own: works 90-95% times and fails in mysterious ways OR 2) it looks like code, it reads like code but humans can’t figure out what it does OR 3) it is a collection of blocks of code put in order automatically to do a certain job, that works in 90-95% of cases and… OR 4) ??

qsort 5y ago
Even assuming that there is a specification, which is already a tall order: - if this is something like automated theorem provers, it's something like (2). It's correct, provably correct even, but it's a bunch of nonsense that provides no insight to humans. - if this is something like natural language query generators, it's something like (1), but with a much lower success rate (closer to 50% IME). I don't even need to specify that AI-generated code is interesting, but we aren't even remotely close to being there.
sneeuwpopsneeuw 5y ago
A lot of IDE's and editors already support autocomplete, intellisense or something similar. I think AI generated code is never going to work but implemented as a autocomplete it can be helpful. I am using https://www.tabnine.com/ https://www.tabnine.com/ for vs code now and that works nice but its very limited The thing that i'm mostly missing are configuration for the aggression and length of the provided options.
- MattRix 5y ago
  Why would AI generated code never work? If anything, it seems like the perfect domain for AI.
  - qsort 5y ago
    I'm not saying it will never work, never is a long time, but it's not the perfect domain for AI. It's kind of the opposite, actually. First and most importantly, programming computers is a very precise endeavor. The logic needs to be exactly correct, not only statistically correct. 'Close enough' won't cut it, not even remotely, and not even for relatively unimportant software. Second, the general problem is undecidable. This isn't a roadblock per se, because we are reasonably good at other undecidable problems (viz. garbage collection), but it means simple algorithmical approaches won't work. Third, software is in a weird place because it requires working at different abstraction levels simultaneously. Often, top-level specifications are fuzzy and incomplete, but some parts require absolute precision and we need to "drop down" to a lower level of abstraction. Humans are able to make the process work (kinda) using lots of common sense, something machines are currently very bad at. If you require the operator of the 'AI' to fill in the details, you just invented a very complicated compiler. Finally, rarely if ever present-day software is made once and never changed: the output needs to be inspectable and maintainable, other software might need to call into it, etc. If the pipeline is more complicated than the software itself, I might as well be writing the code myself. I can see some minor, specific tasks being increasingly done with AI, and I can see tools making more and more use of AI technology, but AI generated code isn't even on the horizon.
    - MattRix 5y ago
      It seems like many of those issues would be solved by having a human write a comprehensive test suite, and then the AI would write an implementation that can pass all of the tests. Eventually you could also have another separate AI that learns to generate the test suite itself from instructions given to it by a human (or even another AI!). Most program implementations would not be perfect, but as we know, software written by humans certainly isn’t perfect either.
      - ruchirpuri 5y ago
        This is precisely how we do this in many other domains of automation. Specify a test set, and let the generator meet that test spec.
- mrjin 5y ago
  Are you sure of that? Microsoft has recently introduced so called AI into VisualStudio, and IMO, it's idiocy rather than intelligence. If you don't know what I mean and you do conveniently have a VS license, try turn the AI shit one and you will soon see it yourselves.
  - sneeuwpopsneeuw 5y ago
    I'm using an old version of codium. But thanks for the recommendation. I will check it out when I ever have to use VisualStudio again or when it gets added to vs code.

by yumraj 5y ago

qsort 5y ago

Even assuming that there is a specification, which is already a tall order: - if this is something like automated theorem provers, it's something like (2). It's correct, provably correct even, but it's a bunch of nonsense that provides no insight to humans. - if this is something like natural language query generators, it's something like (1), but with a much lower success rate (closer to 50% IME). I don't even need to specify that AI-generated code is interesting, but we aren't even remotely close to being there.

sneeuwpopsneeuw 5y ago

A lot of IDE's and editors already support autocomplete, intellisense or something similar. I think AI generated code is never going to work but implemented as a autocomplete it can be helpful. I am using https://www.tabnine.com/ https://www.tabnine.com/ for vs code now and that works nice but its very limited The thing that i'm mostly missing are configuration for the aggression and length of the provided options.

MattRix 5y ago
Why would AI generated code never work? If anything, it seems like the perfect domain for AI.
- qsort 5y ago
  I'm not saying it will never work, never is a long time, but it's not the perfect domain for AI. It's kind of the opposite, actually. First and most importantly, programming computers is a very precise endeavor. The logic needs to be exactly correct, not only statistically correct. 'Close enough' won't cut it, not even remotely, and not even for relatively unimportant software. Second, the general problem is undecidable. This isn't a roadblock per se, because we are reasonably good at other undecidable problems (viz. garbage collection), but it means simple algorithmical approaches won't work. Third, software is in a weird place because it requires working at different abstraction levels simultaneously. Often, top-level specifications are fuzzy and incomplete, but some parts require absolute precision and we need to "drop down" to a lower level of abstraction. Humans are able to make the process work (kinda) using lots of common sense, something machines are currently very bad at. If you require the operator of the 'AI' to fill in the details, you just invented a very complicated compiler. Finally, rarely if ever present-day software is made once and never changed: the output needs to be inspectable and maintainable, other software might need to call into it, etc. If the pipeline is more complicated than the software itself, I might as well be writing the code myself. I can see some minor, specific tasks being increasingly done with AI, and I can see tools making more and more use of AI technology, but AI generated code isn't even on the horizon.
  - MattRix 5y ago
    It seems like many of those issues would be solved by having a human write a comprehensive test suite, and then the AI would write an implementation that can pass all of the tests. Eventually you could also have another separate AI that learns to generate the test suite itself from instructions given to it by a human (or even another AI!). Most program implementations would not be perfect, but as we know, software written by humans certainly isn’t perfect either.
    - ruchirpuri 5y ago
      This is precisely how we do this in many other domains of automation. Specify a test set, and let the generator meet that test spec.
mrjin 5y ago
Are you sure of that? Microsoft has recently introduced so called AI into VisualStudio, and IMO, it's idiocy rather than intelligence. If you don't know what I mean and you do conveniently have a VS license, try turn the AI shit one and you will soon see it yourselves.
- sneeuwpopsneeuw 5y ago
  I'm using an old version of codium. But thanks for the recommendation. I will check it out when I ever have to use VisualStudio again or when it gets added to vs code.