There are some efforts to do fully open LLMs including their training data. Allen AI released their model (OLMo) and the data used for training the model under

There are some efforts to do fully open LLMs including their training data. Allen AI released their model (OLMo) and the data used for training the model under permissive licenses. https://allenai.org/ https://allenai.org/