3 ms·

Just like the first Go playing programs. Just you wait until it is trained by "self-play" against a compiler.

by Iv 5y ago

Skyy93 5y ago
The difference is that in Go you have a defined goal, its static. This is different from writing code or a novel. The goal is not only to write a book or a program, it has to be a entertaining book or a working code that solves MY problem but not any other problem. AI currently does not understand language well enough, it recognises patterns but does not understand what its doing and why.
- Alekhine 5y ago
  You could make the goal more static with unit tests, though they would need to be a lot of them and very specific, otherwise it would just game them.
  - Skyy93 5y ago
    This might be a beginning, but its far away from real code understanding. You still have a problem with copyright code that a AI currently produces and with unsecure code. AI is still too far away from real understanding.
Jensson 5y ago
And what kind of program would your AI write? Would it write a web-app? How would it figure out what the API should look like just based on self training against a compiler? Neither the compiler nor the AI code you mentioned have any understanding of web API's, there is no way it could code that. AI isn't magic, it can't solve problems you don't give it. When doing alpha go they gave the AI all the rules for GO and told it to optimize for those rules. That works fine. But if you want it to make a web-app, what rules would you give it to optimise for? Do you have a web-app evaluator lying around somewhere we can use? If not I don't see it happening. The current code helper solved that by telling the AI to solve the problem "write code that looks like this bunch of human written code". The AI can do that just fine, but code that looks like human written code isn't terribly useful since the AI doesn't understand what makes the code good, all it knows is that the code looks similar to what a human once wrote. This is cool, but as you can see this is very different from the real deal where the AI solves the real coding problem rather than "write something that looks like code" problem.
- Iv 5y ago
  There are several datasets of programming puzzles, of increasing difficulty. Basically, you write a test, let the algorithm find the program that passes it. Hopefully, at one point you reach GPT-3 level performances where it is able to imagine programs for tests it never saw.
  - Jensson 5y ago
    We can discuss that scenario when it happens. Currently we aren't there and it isn't obvious to me that the current algorithm can get there. > Hopefully, at one point you reach GPT-3 level performances where it is able to imagine programs for tests it never saw. You mean nonsense programs just like GPT-3 generates nonsense articles? GPT-3 doesn't remember the logic in its sentences, and in order to solve programming competition problems you need to translate logic from human text into code. I agree that it might be possible to get something useful this way, but until it actually works I'll doubt it will work. There is just way too much coherence required that doesn't seem to be there yet, and from what I've seen the coherence problem gets exponentially worse as you get larger problems.
    - Iv 5y ago
      GPT-3 stays incredibly in-topic. You can give it logic problems it won't be able to solve, but GPT-3 is capable of things that, as a GOFAI supporter, I never thought brute-force connexionist approach could do. I am now very cautious before saying a task can't be done by DL because it requires "understanding". These approaches discover concepts and the relationship between them, and use that in their tasks. It is not far-fetched to say that there is some kind of understanding there. For now we have trained it to generate fake text and basically made a master bullshitter, but I have no doubt that it can easily extract meaning and intent from text.
      - perl4ever 5y ago
        >I have no doubt that it can easily extract meaning and intent from text. Have you met humans? People do not supply complete or self consistent information on what their goals are. They do not form objections to the output of a program based on an accurate and complete model of it either. Also, they hate to communicate via text - how many times have you heard "ugh, let's discuss it on a call"? But that does not mean you can BS them endlessly. The fact that people have no idea about the technical details doesn't mean they are going to accept failure. I'd like to see an AI that can dominate https://en.wikipedia.org/wiki/Nomic https://en.wikipedia.org/wiki/Nomic

Just like the first Go playing programs. Just you wait until it is trained by "self-play" against a compiler.

by Iv 5y ago

Just like the first Go playing programs. Just you wait until it is trained by "self-play" against a compiler.

The difference is that in Go you have a defined goal, its static. This is different from writing code or a novel. The goal is not only to write a book or a program, it has to be a entertaining book or a working code that solves MY problem but not any other problem. AI currently does not understand language well enough, it recognises patterns but does not understand what its doing and why.

Alekhine 5y ago
You could make the goal more static with unit tests, though they would need to be a lot of them and very specific, otherwise it would just game them.
- Skyy93 5y ago
  This might be a beginning, but its far away from real code understanding. You still have a problem with copyright code that a AI currently produces and with unsecure code. AI is still too far away from real understanding.

Jensson 5y ago

And what kind of program would your AI write? Would it write a web-app? How would it figure out what the API should look like just based on self training against a compiler? Neither the compiler nor the AI code you mentioned have any understanding of web API's, there is no way it could code that. AI isn't magic, it can't solve problems you don't give it. When doing alpha go they gave the AI all the rules for GO and told it to optimize for those rules. That works fine. But if you want it to make a web-app, what rules would you give it to optimise for? Do you have a web-app evaluator lying around somewhere we can use? If not I don't see it happening. The current code helper solved that by telling the AI to solve the problem "write code that looks like this bunch of human written code". The AI can do that just fine, but code that looks like human written code isn't terribly useful since the AI doesn't understand what makes the code good, all it knows is that the code looks similar to what a human once wrote. This is cool, but as you can see this is very different from the real deal where the AI solves the real coding problem rather than "write something that looks like code" problem.

Iv 5y ago
There are several datasets of programming puzzles, of increasing difficulty. Basically, you write a test, let the algorithm find the program that passes it. Hopefully, at one point you reach GPT-3 level performances where it is able to imagine programs for tests it never saw.
- Jensson 5y ago
  We can discuss that scenario when it happens. Currently we aren't there and it isn't obvious to me that the current algorithm can get there. > Hopefully, at one point you reach GPT-3 level performances where it is able to imagine programs for tests it never saw. You mean nonsense programs just like GPT-3 generates nonsense articles? GPT-3 doesn't remember the logic in its sentences, and in order to solve programming competition problems you need to translate logic from human text into code. I agree that it might be possible to get something useful this way, but until it actually works I'll doubt it will work. There is just way too much coherence required that doesn't seem to be there yet, and from what I've seen the coherence problem gets exponentially worse as you get larger problems.
  - Iv 5y ago
    GPT-3 stays incredibly in-topic. You can give it logic problems it won't be able to solve, but GPT-3 is capable of things that, as a GOFAI supporter, I never thought brute-force connexionist approach could do. I am now very cautious before saying a task can't be done by DL because it requires "understanding". These approaches discover concepts and the relationship between them, and use that in their tasks. It is not far-fetched to say that there is some kind of understanding there. For now we have trained it to generate fake text and basically made a master bullshitter, but I have no doubt that it can easily extract meaning and intent from text.
    - perl4ever 5y ago
      >I have no doubt that it can easily extract meaning and intent from text. Have you met humans? People do not supply complete or self consistent information on what their goals are. They do not form objections to the output of a program based on an accurate and complete model of it either. Also, they hate to communicate via text - how many times have you heard "ugh, let's discuss it on a call"? But that does not mean you can BS them endlessly. The fact that people have no idea about the technical details doesn't mean they are going to accept failure. I'd like to see an AI that can dominate https://en.wikipedia.org/wiki/Nomic https://en.wikipedia.org/wiki/Nomic