4 ms·
This looks incredibly cool; I'm wowed by the fact that the model has learnt to negate words in if-else statements, though I struggle to think of a case where th
by _9jgl 7y ago
This looks incredibly cool; I'm wowed by the fact that the model has learnt to negate words in if-else statements, though I struggle to think of a case where that particular completion would have been useful.
At the same time, I'm less excited about the fact that the model is cloud-only, both for security/privacy reasons and because I spend a not-insignificant amount of my time on limited-bandwidth/high-latency internet connections.
I'm also curious as to why the survey didn't ask about GPU specifications; most of the time I use my laptop to code whilst plugged in, and I'd happily use only LSP completions when on battery, so power consumption wouldn't be an issue (though fan noise might), and allegedly my GPU (a GTX 1050) can pull off almost 2 TFLOPs, which is well over the "10 billion floating point operations" mentioned in the post.
- jacob-jackson 7y agoGood point, I added it to the survey.
- frou_dh 7y agoSince it's asking for FLOPS, are you ready for people typing in 14-digit numbers? ;)
- jacob-jackson 7y agoI'm looking forward to some data cleaning :)
- Mil0dV 7y agoSo for reference (and checking) I entered 1862000000000 for my 1050 ti.
- zawerf 7y ago> I'm wowed by the fact that the model has learnt to negate words in if-else statements I know it learned natural languages from using GPT-2, but I am surprised it didn't get "confused" since words are used in such a different way in programming. For example strong appears as the html tag <strong> with no corresponding <weak> tag. And weak appears in weak_ptr in C++ and there's no such thing as a strong_ptr.
- 0b01 7y agoI think the tokens are simply one hot encoded. Though some sort of word2vec embedding is also situationally plausible
- Felz 7y agoVanilla GPT2 actually does a half decent job of emulating the style of programming. Try entering "public static int main() {" into https://talktotransformer.com/ https://talktotransformer.com/