4 ms·
I wonder if these detections would become just impossible once NLP researchers figure out how to integrate adversarial training (specially GANs) into their mode
by snrji 8y ago
I wonder if these detections would become just impossible once NLP researchers figure out how to integrate adversarial training (specially GANs) into their models.
- gwern 8y agoWhat OP is detecting is not a flaw in GPT-2 but in how GPT-2 is used to generate samples: it chooses only from the top k most likely words, so you can easily detect GPT-2 simply by running it and noting that no sampled word appears from below, say, the top-40 candidates, while human text will occasionally have a top-41 or a top-99 or a top-1000 word. If you fix that with a better sampling strategy like beam search or RL finetuning, unclear how well OP-like methods would work.