The Pile and Red Pajama are primarily English language datasets. If you want something multilingual, I'd suggest having a look at the Bloom dataset https://arx

The Pile and Red Pajama are primarily English language datasets. If you want something multilingual, I'd suggest having a look at the Bloom dataset https://arxiv.org/abs/2210.14712 https://arxiv.org/abs/2210.14712