3 ms·

I'm not really sure that type of tool could really be anything else. How would a model become aware of all of the various edge cases that depend on which SQL d

I'm not really sure that type of tool could really be anything else. How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time?

gmadsen 5y ago
a large data set covering exactly what you just mentioned?
sbr464 5y ago
Can it submit pull requests to itself with if/else boolean logic/hacks?
TeMPOraL 5y ago
> I'm not really sure that type of tool could really be anything else. It can't be, because they've chosen to use a deep learning approach. That makes it a dead end right from the start. > How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time? A lot of things that we call "edge cases" are only a problem for humans. They're not "edge cases" from the point of view of the grammar / semantics of programming languages and libraries. The way a hypothetical, better Copilot could work, is by having directly encoded grammars and semantics metadata corresponding to popular languages and tools. It could generate code in principled and introspectable way, by having a model of the computation it wants to express and encoding it in a target language. Of course, such hypothetical Copilot is a harder task - someone would have to come up with a structure for explicitly representing understanding of the abstract computation the user wants to happen, and then translate user input into that structure. That's a lot of drudgery, and from my vague understanding of the "classical" AI space, there might be a bunch of unsolved problems on the way. Real Copilot uses DNNs, because they let you ignore all that - you just keep shoving code at it, until the black-box model starts to give you mostly correct answers. The hard work is done automagically. It makes sense for some tasks, less for others - and I think code generation is one of those things where black-box DNNs are a bad idea.
- jhgb 5y ago
  > The way a hypothetical, better Copilot could work, is by having directly encoded grammars and semantics metadata corresponding to popular languages and tools. It could generate code in principled and introspectable way, by having a model of the computation it wants to express and encoding it in a target language. But that sounds like too much work, let's just throw a lot of data into an NN and see what comes out! /s > and introspectable Which most importantly means "debuggable", I assume. From what I get there doesn't seem to be any way to ad-hoc fix an NN's output.

gmadsen 5y ago

a large data set covering exactly what you just mentioned?

sbr464 5y ago

Can it submit pull requests to itself with if/else boolean logic/hacks?

TeMPOraL 5y ago

> I'm not really sure that type of tool could really be anything else. It can't be, because they've chosen to use a deep learning approach. That makes it a dead end right from the start. > How would a model become aware of all of the various edge cases that depend on which SQL database you use or differences in language versions over time? A lot of things that we call "edge cases" are only a problem for humans. They're not "edge cases" from the point of view of the grammar / semantics of programming languages and libraries. The way a hypothetical, better Copilot could work, is by having directly encoded grammars and semantics metadata corresponding to popular languages and tools. It could generate code in principled and introspectable way, by having a model of the computation it wants to express and encoding it in a target language. Of course, such hypothetical Copilot is a harder task - someone would have to come up with a structure for explicitly representing understanding of the abstract computation the user wants to happen, and then translate user input into that structure. That's a lot of drudgery, and from my vague understanding of the "classical" AI space, there might be a bunch of unsolved problems on the way. Real Copilot uses DNNs, because they let you ignore all that - you just keep shoving code at it, until the black-box model starts to give you mostly correct answers. The hard work is done automagically. It makes sense for some tasks, less for others - and I think code generation is one of those things where black-box DNNs are a bad idea.

jhgb 5y ago
> The way a hypothetical, better Copilot could work, is by having directly encoded grammars and semantics metadata corresponding to popular languages and tools. It could generate code in principled and introspectable way, by having a model of the computation it wants to express and encoding it in a target language. But that sounds like too much work, let's just throw a lot of data into an NN and see what comes out! /s > and introspectable Which most importantly means "debuggable", I assume. From what I get there doesn't seem to be any way to ad-hoc fix an NN's output.