With new direction of research, like pre-taining and subword vocab, the models now are mostly language agnostic. English or not, it is just sequence of tokens.

With new direction of research, like pre-taining and subword vocab, the models now are mostly language agnostic. English or not, it is just sequence of tokens. Difference mainly lies in data abundance, where the distribution is severly skewed.