Spanish tokens are just more tokens to predict. No tagging necessary. If the model can write in Spanish fluently then it saw enough Spanish language tokens to b

Spanish tokens are just more tokens to predict. No tagging necessary. If the model can write in Spanish fluently then it saw enough Spanish language tokens to be competent. "Enough" is a sliding target. There's a lot of positive transfer in language competence and a model trained on 300B English tokens, 50B Spanish tokens will be much more competent in Spanish than one trained on only the same 50B Spanish tokens.