All the LLM advances these days are from synthetic or explicitly created data too. You need public data mostly because it contains facts about the world, or bec

All the LLM advances these days are from synthetic or explicitly created data too. You need public data mostly because it contains facts about the world, or because it's easier to talk about a book when it's "read" the book. But for a known topic area (as opposed to open Q&A) it's not critical since you can go and create or license it.