You could train a language model from a corpus of writing by e.g. ESL students or young children, and then disallow any words with p<p(thousandth_word) given th

You could train a language model from a corpus of writing by e.g. ESL students or young children, and then disallow any words with p<p(thousandth_word) given the preceding context. :)