The public domain literature is big, but not that big and very far from infinite data - all the major current models already include all of the public domain li

The public domain literature is big, but not that big and very far from infinite data - all the major current models already include all of the public domain literature that has ever been digitized and much more (so all the public domain literature not only isn't abundant, but it isn't even sufficient to make anything competitive with even last generation models), and we would really like to get 10x or 100x more data than all the literature ever published (both PD and not), if we could. And we probably can, since the vast, vast majority of what people write happens in other contexts and only a tiny fraction of written words end up as published literature.