Something that I find very sad and is but discussed enough is using The Pile as training set (ok, for this one was will have to wait for the report). The Pile i

Something that I find very sad and is but discussed enough is using The Pile as training set (ok, for this one was will have to wait for the report). The Pile is a dataset of mostly English documents. So it won't have the multilingual capabilities of other models, like the ones by Open AI or even FlanT5. We are leaving a lot of people excluded from being benefited from open source models because of that...