3 ms·
This isn't exactly bot sumissions, and the process is not really scalable: > To quickly weed out inappropriate comments, I handpick from generated comments tho
by buboard 7y ago
This isn't exactly bot sumissions, and the process is not really scalable:
> To quickly weed out inappropriate comments, I handpick from generated comments those that ensure a high coherence and high relevance sample for submission.
So basically it's a validation of GPT-2 making sense with small amounts of text. Judging from the demo test page, they are pretty good texts, but he said it himself that larger texts betray the bot. So, i m not sure what he's trying to prove by using MTurkers, since this does not attack the problem mentioned in his introduction: the fake FCC comments were weeded out through text analysis, not via human work.
In all, i'm not sure if this is something that people didn't know about gpt-2. The title is certainly not justified, perhaps "Curated bot comments can't be distinguished by humans to be obviously fake" would be better, but also more banal.
- notahacker 7y agoI'm also wondering whether the 'handpick[ed]...to ensure a high coherence and high relevance' GPT-2 comments actually outperform the comparatively trivial sentence-spinning script in getting approved by MTurkers. Think https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/r/SubSimulatorGPT2/ is more impressive than a study where half of GPT-2 comments handpicked for being human-like by one human were accepted by another human. Particularly given that some of the comments in question were three or four words long...