4 ms·
>> I don't think it is clear that such "fundamental weaknesses" exist. A text-based approach can get you incredibly far. Mnyeah, not really that "incredibly".
by YeGoblynQueenne 5y ago
>> I don't think it is clear that such "fundamental weaknesses" exist. A text-based
approach can get you incredibly far.
Mnyeah, not really that "incredibly". Remember that neural network models are
great at interpolation but crap at extrapolation. So as long as you accept that
the code generated by Copilot stays inside a well-defined area of the program
space, and that no true novelty can be generated that way, then yes, you can get
a lot out of it and I think, once the wrinkles are ironed out, Copilot might
be a useful, everyday tool in every IDE (or not; we'll have to wait and see).
But if you need to write a new program that doesn't look like anything anyone
else has written before, then Copilot will be your passenger.
How often do you need to do that? I don't know. But there's still many open
problems in programming that lots of people would really love to be able to
solve. Many of them have to do with efficiency and you can't expect Copilot to
know anything about efficiency.
For example, suppose we didn't know of a better sorting algorithm than
bubblesort. Copilot would not generate mergesort. Even given examples of
divide-and-conquer algorithms, it wouldn't be able to extrapolate to a
divide-and-conquer algorithm that gives the same inputs for the same outputs as
bubblesort. It wouldn't be able to, because it's trained to reproduce code from
examples of code, not to generate programs from examples of their inputs and
outputs. Copilot doesn't know anything about programs and their inputs and
outputs. It is a language model, not a magickal pink fairy of wish fulfillment
and so it doesn't know anything about things it wasn't trained on.
Again, how often do you need to write truly novel code? In the context of
professional software development, I think not that often. So if it turns out to
be a good boilerplate generator, Copilot can go a long way. As long as you don't
ask it to generate something else than boilerplate.
There are approaches that work very well in the task of generating programs
that they've never seen before from examples of their inputs and outputs, and
that don't need to be trained on billions of examples. True one-shot learning of
programs (without a model pre-trained on billions of examples) is possible.
With current approaches. But those approaches only work for languages like Prolog
and Haskell, so don't expect to see those approaches helping you write code in
your IDE anytime soon.