6 ms·
So that's basically program synthesis from natural language (ish) specifications (i.e. the comments). I can see this being a useful tool [1]. However, I don't
by YeGoblynQueenne 6y ago
So that's basically program synthesis from natural language (ish)
specifications (i.e. the comments).
I can see this being a useful tool [1]. However, I don't expect any ability
for innovation. At best this is like having an exceptionally smart
autocomplete function that can look up code snippets on SO for you (provided
those code snippets are no longer than one line).
That's not to say that it can't write new code, that nobody has quite
written before in the same way. But in order for a tool like this to be useful
it must stick as close as possible to what is expected- or it will slow
development down rather than helping it. Which means it can only do what has
already been done before.
For instance- don't expect this to come up with a new sorting algorithm, out
of the blue, or to be able to write good code to solve a certain problem when
the majority of code solving that problem on github happens to be pretty bad.
In other words: everyone can relax. This will not take your job. Or mine.
____________
[1] I apologise to the people who know me and who will now be falling off
their chairs. OK down there?
- joshuak 6y agoI agree completely with your expectation of the abilities of such a system. However, I think very little programming labor is employed in the construction of new algorithms or even most business logic, even a casual stroll through github reveals a staggering amount of reimplementation. I think the promise here is the ability to code in a more conceptual way with less fiddling with the finicky details.
- Swizec 6y ago> I think the promise here is the ability to code in a more conceptual way with less fiddling with the finicky details. This is basically how product managers code. Or former engineers turned engineering managers. Or even team leads. Hell, maybe like an architect? You come up with a rough sketch, design the system, think through a couple edge cases, tell the computer what you need, and the computer figures out the details for you. Similar to being a high level engineer that designs/defines/codes the broad strokes of something and then lets the lower level minions handle details. We made a similar leap when compilers were invented.
- chii 6y ago> Similar to being a high level engineer that designs/defines/codes the broad strokes of something and then lets the lower level minions handle details and then the impl. turns out to have a bunch of details wrong that you didn't catch initially. And you wonder why there's so many bugs in software these days! I think the AI model is helpful, but the specification being ambiguous or under-specified is the problem, and the effort to sort that out is hard. I'm not sure an AI can help in that aspect, and that's where most of the value of programming comes from.
- bauerm97 6y agoI'm sure in the early days of compilers (I wasn't around back then, so I'm just assuming) they could also be fairly unreliable. Maybe they translated something in a completely idiotic way, lots of bugs, etc... But over time they improved and improved to the point that 99% of programmers never worry about anything except high level abstractions. This could be signalling the beginning of another such paradigm shift to a higher level abstraction
- sguthery 6y agoWould that that were the case (compilers getting better). See http://embed.cs.utah.edu/csmith/ http://embed.cs.utah.edu/csmith/ C compilers for the PDP-11 were pretty good if only because they were both simple, single threaded, and C was essentially a verbose version of the PDP-11 instruction set. Languages got more abstract and instruction sets (including their models of execution) got more complex. Optimizing the compilation of an abstraction with a faulty understanding of either the abstraction or the instruction set (or both) begets bugs you can't see as reported in the reference above. OB: I personally would like to see these folks point their AI code machine at netlib including the Collected Algorithms of the ACM. Generating numerical methods code is, in my experience, not the same as generating much of what is found on Github.
- YeGoblynQueenne 6y agoI agree- that's why I think such a system is not capable of innovation. In the same way, that's why I think it would be a useful tool: it promises to automate away the kind of coding that most programmers can do with eyes closed and that's the most boring and repetitive part of the job. Like, without trying to demean it, it sounds like a great boilerplate generator.
- pharke 6y agoYeah I'm thinking it would be more useful to have a really well indexed library of functions accessible by search.
- gwern 6y agoI think you are underselling the potential of a model which deeply understand programming. Imagine combining such a model with something like AutoML-Zero: https://arxiv.org/abs/2003.03384 https://arxiv.org/abs/2003.03384 It may not be 'creative', but used as tab-completion, it's not being rewarded or incentivized or used in any way which would expose its abilities towards creating a new sort algorithm.
- raghavgoyal14 6y agoI agree on the tab-completion part. Something like Gmail's smart-compose could have potentially huge benefits here. But I'm not sure about the "deeply understand programming" part. Language modelling and "AI", in its current form, uncovers only statistical correlations and barely scratches the surface of what "understanding" is. This has restricted deployment of majority of academic research into the real-world and this, I believe, is no different and will work only in constrained settings. Edit: typo
- um_ya 6y agoIt would be nice to have an AI that could write unit tests, or look over your code and understand and explain where you might have bugs.
- raghavgoyal14 6y ago> look over your code and understand and explain where you might have bugs. This would certainly be interesting. I'm not aware of active research going on in this area (any pointers would be helpful!). This would require an agent to have thorough understanding of the logic you're trying to implement, and locate the piece of code where it silently fails. For this you'd again need a training dataset where the input is a piece of code and the supervision signal (the output) is location of the bug. I could imagine some sort of self-supervision to tackle this initially where you'd intentionally introduce bugs in your code to generate training data. But not sure how far this can go!
- 6y ago
- gameswithgo 6y agoalphago and alphastar were certainly creative. this project in its current state may not have that capacity but it also may not be a huge leap to get there.
- westurner 6y ago> At best this is like having an exceptionally smart autocomplete function that can look up code snippets on SO for you (provided those code snippets are no longer than one line). Yeah, all it could do for you is autocomplete around what it thinks the specification might be at that point in time. > But what if Andy gets another dinosaur, a mean one? -- Toy Story (1995)
- random32840 6y agoAn AI like this can hold a hell of a lot more information in its head at one point than a human. Each decision it makes is based on way more context, it can manipulate the problem using much more information, much faster. The problem is that it can't think in abstractions. If AI gets to the point where it has a reasonable understanding of the shape of the data & the basic spatial manipulations being applied (not far off IMO), I'd expect it to be waaaaaay better at discovering certain types of new algorithms than humans. It can handle thinking about algorithms that have millions of independently moving parts in a way a human can't. Humans have the edge deriving algorithms that require a sequence of high-level steps on an abstraction. "Do this, then we get a thing, then we do some stuff to the thing, stretch it, squash it, massage it." AI sucks at that, it doesn't think in the same kind of flexible abstractions. But imagine if you build an understanding of how the code will be compiled & how that will interact with the cache into the AI. That's very difficult for humans because you can't think about all those mechanics at once, we have to focus on one at a time. An AI that really gets it? I could see it writing a better sorting algorithm for a specific, complex datatype than a human could, or at the very least having the competetive edge because it can do it basically instantly.
- izabera 6y agoHow often does the average programmer come up with a new sorting algorithm?
- DJHenk 6y ago> In other words: everyone can relax. This will not take your job. Or mine Of course not. This technology converts writing code into bug hunting in pre-written code. Finding bugs in code that you did not write is way harder than writing the code yourself. So if anything, this makes programming harder, not easier, and we will need more programmers, not less.
- OOPMan 6y agoOh dear. And then the model trains itself on the buggy code written and poorly debugged by these extra coders and then so on and so forth. Codepocalypse. Kill it with fire!
- BaronSamedi 6y agoI agree. While this is well done, it seems to be copying human programming techniques rather than allowing the AI to create code that it thinks is optimal. I think there is the potential to evolve efficient and secure code that is free from the constraints we impose on it due to the way our minds work. Such code may not be intelligible to us but could very well be much better than what we could write.
- gradys 6y agoI'd put it differently. This is going to take your job, just like an assembly programmer from the 70s might consider Python to have basically taken their job. In software, the job is constantly eating itself and transforming. It's part of the job to continually incorporate new capabilities and lever yourself up.