4 ms·
interesting topic. Could you point me to some resources so I can learn more about it? The following is a comprehensive introduction and survey of recent work i
by YeGoblynQueenne 5y ago
interesting topic. Could you point me to some resources so I can learn more
about it?
The following is a comprehensive introduction and survey of recent work in the
field:
https://www.microsoft.com/en-us/research/wp-content/uploads/2017/10/program_synthesis_now.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/...
Regarding novelty, neural networks are excellent at interpolation but awful at
extrapolation. What I mean by interploation is that the models learned by a
neural network represent a dense region of cartesian space circumscribed by the
neural net's training instances. The trained model can recognise and generate
instances included in this dense region, including instances not included in its
training set, by interpolation, but it is almost completely incapable of
representing instances outside this dense region by extrapolation [1].
So basically what I'm saying above is that deep learning language model code
generators can't extrapolate to programs that are not similar to the programs
they've been trained on. Hence, no real novelty. But you can go a long way
without such novelty hence my expectation that e.g. Copilot can be a nice
boilerplate generator.
______________
[1] My standard reference for this is the following article by François Chollet,
maintainer of Keras:
https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html
See also the second part of the series of articles where possible ways are
proposed to address this limitation:
https://blog.keras.io/the-future-of-deep-learning.html https://blog.keras.io/the-future-of-deep-learning.html
The latter is relevant to the conversation because the author basically proposes
augmanting current deep learning approaches with a form of program synthesis.