Anthropic has a recent paper that talks about this [1]. See figure 16 in particular. The larger the model, the more it generalizes between languages. On the ot

Anthropic has a recent paper that talks about this [1]. See figure 16 in particular. The larger the model, the more it generalizes between languages. On the other hand, no matter how large the model they struggle to generalize between sentences of the form "George Washington was the first US president" and "The first US president was George Washington". Clearly generalization behavior is unintuitive, you can come up with a post hoc explanation (weights are independent between early and late layers) but I doubt anyone would have thought ahead of time that models would have an easier time generalizing between languages than between slightly different ways of saying the same thing in English. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf