For a period like the Roman Empire, there's might be too little source material to even train the model to speak Latin, let alone to say anything about its worl

For a period like the Roman Empire, there's might be too little source material to even train the model to speak Latin, let alone to say anything about its world. IIRC, the entire surviving corpus of ancient Latin would fit into a couple of bookshelves - it's miniscule.