> The NYT, knowing the correct order, re-arranged them to fit the ordering in the article. > Even doing this, they were only able to get a hundred or so words

> The NYT, knowing the correct order, re-arranged them to fit the ordering in the article. > Even doing this, they were only able to get a hundred or so words out of the 15k+ word articles. OK, that’s less material than I believed, which shows the details matter. But we agree that the overfit material, while limited, is stored in the model. Of course, this can be (and surely is) mitigated by filtering the output, as long as the product is the output and not the model itself.