It seems like it would be challenging to get the corruptor to generate examples that are of the same Kind that humans make, while still being "productive" (in t

It seems like it would be challenging to get the corruptor to generate examples that are of the same Kind that humans make, while still being "productive" (in the linguistic sense, ie. not just overfitting on examples from a corpus of low quality text). It's easy enough to just drop random words or run Levenshtein-edits on single words to create wrong-in-this-context makes (then/than), but grammar errors include much more than can be covered by that method, and the method will generate many errors that are of a kind never made by humans. And if you restrict your method to things already seen in a corpus, it's easy to overfit and miss out on a whole lot of good stuff.