One issue with AI2 (who makes Olmo) is that some of their input data sets are filtered and curated, to remove things that they deem politically or ethically una

One issue with AI2 (who makes Olmo) is that some of their input data sets are filtered and curated, to remove things that they deem politically or ethically unacceptable like “toxic” content. This step obviously influences the LLM, but as far as I know it isn’t done transparently in an auditable way. I think being truly open would require a raw dataset to be available without this curation, along with a log of every single instance of curation. The dataset should also not be attached to a restrictive license as they have done. https://blog.allenai.org/dolma-3-trillion-tokens-open-llm-corpus-9a0ff4b8da64 https://blog.allenai.org/dolma-3-trillion-tokens-open-llm-co...