5 ms·

Show HN: Autocomplete Python with Deep Learning

109 points by vpj 7y ago

nikeee 7y ago
Is this similar to Microsoft's IntelliCode? How does it differ?
- veselin 7y ago
  This one is a character level language model. The Microsoft one is per token. This one should be the closest to the Microsoft model: https://miltos.allamanis.com/publicationfiles/allamanis2018learning/allamanis2018learning.pdf https://miltos.allamanis.com/publicationfiles/allamanis2018l... Btw, simple neural networks are far from doing the best for code. The kind of accuracy LSTM gives is already achieved with good n-gram models. E.g. see this one on a Linux code prediction: https://arxiv.org/pdf/1506.02078.pdf https://arxiv.org/pdf/1506.02078.pdf Also, there are other techniques that improve on top of the n-gram models: https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2017.pdf https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2...
  - ma2rten 7y ago
    I only skimmed the paper you linked, but I don't think it says what you say it says. "We found that scaling up the model almost entirely eliminates errors in the n-gram category."
    - p1esk 7y ago
      See table 1 in his last link.
  - p1esk 7y ago
    What about gpt-2?
Der_Einzige 7y ago
Hit me up when this competes with tabNine
- Matumio 7y ago
  This looks more like a research project. In contrast, tabNine is a closed-source commercial package. No need to compete with that.
bobajeff 7y ago
I've been thinking about using something like this for Swype-like keyboard for coding on phones.
- sho 7y ago
  I've been waiting for Apple to at least apply some ML in text prediction (and moreso, spellchecking) in its onscreen keyboards. What they have now is worse than useless and I would have thought it would be low-hanging fruit?
logicprog 7y ago
This is pretty cool, but I'm not sure how much better this approach would be compared to language server based completion (something like RLS).
make3 7y ago
I wished they had an online demo, or jupyter notebook with an easy to download pretrained model so I can test it in colab
etaioinshrdlu 7y ago
Another fun idea is to make a "code denoiser". Removing noise (bugs) from your code. It's quite simple to synthetically inject lots of noise into code, simply use the wrong variables, operators, numbers, structures, reverse statement ordering, transforming code at the AST level. Then you can learn mappings of bad code => good code! The same idea has worked for natural languages as well, correcting typos.

109 points by vpj 7y ago

nikeee 7y ago

Is this similar to Microsoft's IntelliCode? How does it differ?

veselin 7y ago
This one is a character level language model. The Microsoft one is per token. This one should be the closest to the Microsoft model: https://miltos.allamanis.com/publicationfiles/allamanis2018learning/allamanis2018learning.pdf https://miltos.allamanis.com/publicationfiles/allamanis2018l... Btw, simple neural networks are far from doing the best for code. The kind of accuracy LSTM gives is already achieved with good n-gram models. E.g. see this one on a Linux code prediction: https://arxiv.org/pdf/1506.02078.pdf https://arxiv.org/pdf/1506.02078.pdf Also, there are other techniques that improve on top of the n-gram models: https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2017.pdf https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2...
- ma2rten 7y ago
  I only skimmed the paper you linked, but I don't think it says what you say it says. "We found that scaling up the model almost entirely eliminates errors in the n-gram category."
  - p1esk 7y ago
    See table 1 in his last link.
- p1esk 7y ago
  What about gpt-2?

Der_Einzige 7y ago

Hit me up when this competes with tabNine

Matumio 7y ago
This looks more like a research project. In contrast, tabNine is a closed-source commercial package. No need to compete with that.

bobajeff 7y ago

I've been thinking about using something like this for Swype-like keyboard for coding on phones.

sho 7y ago
I've been waiting for Apple to at least apply some ML in text prediction (and moreso, spellchecking) in its onscreen keyboards. What they have now is worse than useless and I would have thought it would be low-hanging fruit?

logicprog 7y ago

This is pretty cool, but I'm not sure how much better this approach would be compared to language server based completion (something like RLS).

make3 7y ago

I wished they had an online demo, or jupyter notebook with an easy to download pretrained model so I can test it in colab

etaioinshrdlu 7y ago

Another fun idea is to make a "code denoiser". Removing noise (bugs) from your code. It's quite simple to synthetically inject lots of noise into code, simply use the wrong variables, operators, numbers, structures, reverse statement ordering, transforming code at the AST level. Then you can learn mappings of bad code => good code! The same idea has worked for natural languages as well, correcting typos.