Sorry, since the Pile and C4 https://huggingface.co/datasets/c4 https://huggingface.co/datasets/c4 (and, more generally, common crawl) and BigCode https://www.b

Sorry, since the Pile and C4 https://huggingface.co/datasets/c4 https://huggingface.co/datasets/c4 (and, more generally, common crawl) and BigCode https://www.bigcode-project.org/ https://www.bigcode-project.org/ became available, this argument ceased being the real moat. The real moat is more about the lack of concentrated compute, ML engineering, and, more generally, prosaic lack of political with outside a few orgs.