4 ms·
> I'm wowed by the fact that the model has learnt to negate words in if-else statements I know it learned natural languages from using GPT-2, but I am surprise
by zawerf 7y ago
> I'm wowed by the fact that the model has learnt to negate words in if-else statements
I know it learned natural languages from using GPT-2, but I am surprised it didn't get "confused" since words are used in such a different way in programming.
For example strong appears as the html tag <strong> with no corresponding <weak> tag. And weak appears in weak_ptr in C++ and there's no such thing as a strong_ptr.
- 0b01 7y agoI think the tokens are simply one hot encoded. Though some sort of word2vec embedding is also situationally plausible
- Felz 7y agoVanilla GPT2 actually does a half decent job of emulating the style of programming. Try entering "public static int main() {" into https://talktotransformer.com/ https://talktotransformer.com/