4 ms·

Is this similar to Microsoft's IntelliCode? How does it differ?

• veselin 7y ago
  This one is a character level language model. The Microsoft one is per token. This one should be the closest to the Microsoft model: https://miltos.allamanis.com/publicationfiles/allamanis2018learning/allamanis2018learning.pdf https://miltos.allamanis.com/publicationfiles/allamanis2018l... Btw, simple neural networks are far from doing the best for code. The kind of accuracy LSTM gives is already achieved with good n-gram models. E.g. see this one on a Linux code prediction: https://arxiv.org/pdf/1506.02078.pdf https://arxiv.org/pdf/1506.02078.pdf Also, there are other techniques that improve on top of the n-gram models: https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2017.pdf https://files.sri.inf.ethz.ch/website/papers/charmodel-iclr2...
  • ma2rten 7y ago
    I only skimmed the paper you linked, but I don't think it says what you say it says. "We found that scaling up the model almost entirely eliminates errors in the n-gram category."
    • p1esk 7y ago
      See table 1 in his last link.
  • p1esk 7y ago
    What about gpt-2?