Very interesting! So what I’m wondering is this: Can you train a small model on this “child dataset“. And then, once it speaks fluently in this simple language,

Very interesting! So what I’m wondering is this: Can you train a small model on this “child dataset“. And then, once it speaks fluently in this simple language, add new layers on top of it, and, freezing the rest of the model, train those on a corpus of more complex material.