2 ms·
I was always able to correctly identify GPT2 but on a few occasions, I misidentified human-written code as being written by GPT2. Usually when the code was poor
by cryptica 6y ago
I was always able to correctly identify GPT2 but on a few occasions, I misidentified human-written code as being written by GPT2. Usually when the code was poorly written or the comments were unclear.
GPT2's code looks like correct code at a glance but when you try to understand what it's doing, that's when you understand that it could not have been written by a human.
It's similar to the articles produced by GPT3; they have the right form but no substance.
- thombat 6y agoI ran into some value setters which simply logged their parameters but did not store them. Hah! obviously GPT2, not understanding that a setter should, well, set. Wrong, of course. Now maybe the human concerned was far down some inheritance tree and simply wanted to document misuse of a deliberately limited class but assert()ing would have been too punitive. Or may it was indeed "right form but no substance", but authored by an SWE.
- cryptica 6y agoHaha I also got tricked a few times by this sort of thing. One of the ones I got wrong was there was a comment which said '// end of for loop' after every closing brace of every 'for' loop and I thought that there is no way that a human would ever write such pointless comments... I did manage to get 100% correct after a while but it takes a thorough reading of the code.