3 ms·

AlphaCode doesn't need to by perfect, or even particularly good. The question is when AlphaCode, or an equivalent, is good enough for a sufficient number of pro

AlphaCode doesn't need to by perfect, or even particularly good. The question is when AlphaCode, or an equivalent, is good enough for a sufficient number of problems. Like C code can always be made faster than Python, Python performance is good enough (often 30x slower than C) for a very wide set of problems while being much easier to use. In Norvig's example, the code is much slower than ideal (50x slower), it adds unnecessary code, and yet, it generated correct code many times faster than anyone could ever hope to. An easy to use black box that produces correct results can be good enough.

ALittleLight 4y ago
It seems like it would be easy enough to train the model to improve performance too once you have it writing correct code. Write N problems and N performance test suites - then train by self-play on those problems until you're writing high performance code.
alar44 4y ago
Absolutely. I've been using it to create Slack bots over the last week. It's cuts out a massive amount of time researching APIs and gives me good enough, workable, understandable starting points that saves me hours worth of fiddling and refactoring.
TheRealPomax 4y ago
But as an AI implementation, when considered "as an example of an AI implementation", an analysis of its behaviour by one of the foremost experts in the field is still entirely worth the effort, because we learn from that analysis. It teaches us what an expert in the field sees happening, where and how it's falling short, and so what kind of improvements would be worth continued focus on. Criticism by a lay person is a blog post. Criticism by Peter Norvig is a teaching moment that future work is based on. It's like having Knuth comment on your fundamental algorithm design: there is gold in them there text, and we got it for free.
weatherlite 4y ago
It's good enough for a human to go over it and change it (maybe change it in a big way even). This wouldn't have passed code review probably. Its still probably quite a boost for productivity but it doesn't take the human out of the equation. I can also imagine scenarios where relying too much on generated code without understanding it will cause massive bugs and headaches, just like copy pasting stuff from Stackoverflow without understanding it.
Too 4y ago
The idea of python is to be human readable, allowing easy review of correctness and hopefully translating into choice of better algorithms. This AI generated example is neither readable nor algorithmically efficient, on top of running in slow interpreted python. Worst of all 3 dimensions. I do happily and regularly use copilot myself for boilerplate, where it does produce good enough and easily verifiable results, but hell no would I trust it for a complex algorithm like this, in its current state. Just understanding the problem domain enough to review the code is more than half of the challenge. How long until we reach that level of trust is our future mystery.

ALittleLight 4y ago

It seems like it would be easy enough to train the model to improve performance too once you have it writing correct code. Write N problems and N performance test suites - then train by self-play on those problems until you're writing high performance code.

alar44 4y ago

Absolutely. I've been using it to create Slack bots over the last week. It's cuts out a massive amount of time researching APIs and gives me good enough, workable, understandable starting points that saves me hours worth of fiddling and refactoring.

TheRealPomax 4y ago

But as an AI implementation, when considered "as an example of an AI implementation", an analysis of its behaviour by one of the foremost experts in the field is still entirely worth the effort, because we learn from that analysis. It teaches us what an expert in the field sees happening, where and how it's falling short, and so what kind of improvements would be worth continued focus on. Criticism by a lay person is a blog post. Criticism by Peter Norvig is a teaching moment that future work is based on. It's like having Knuth comment on your fundamental algorithm design: there is gold in them there text, and we got it for free.

weatherlite 4y ago

It's good enough for a human to go over it and change it (maybe change it in a big way even). This wouldn't have passed code review probably. Its still probably quite a boost for productivity but it doesn't take the human out of the equation. I can also imagine scenarios where relying too much on generated code without understanding it will cause massive bugs and headaches, just like copy pasting stuff from Stackoverflow without understanding it.

Too 4y ago

The idea of python is to be human readable, allowing easy review of correctness and hopefully translating into choice of better algorithms. This AI generated example is neither readable nor algorithmically efficient, on top of running in slow interpreted python. Worst of all 3 dimensions. I do happily and regularly use copilot myself for boilerplate, where it does produce good enough and easily verifiable results, but hell no would I trust it for a complex algorithm like this, in its current state. Just understanding the problem domain enough to review the code is more than half of the challenge. How long until we reach that level of trust is our future mystery.