P.S. They acknowledge this: “... our preliminary experiments show that such method might not be helpful, and even harmful, to open-ended text generation tasks.

P.S. They acknowledge this: “... our preliminary experiments show that such method might not be helpful, and even harmful, to open-ended text generation tasks. During generation, ∞-gram can make odd mistakes (e.g., predicting totally irrelevant tokens) which makes the model to digress. Thus this combined model is not ready to replace neural LMs. Additional investigation is required to make inf-gram best contribute to text generation.”