3 ms·

Is that the right way to think of LLMs (that they "think in English")? I think a better way to think of them is as an n-dimensional "meaning space" where words

by slibhb 3y ago

Is that the right way to think of LLMs (that they "think in English")? I think a better way to think of them is as an n-dimensional "meaning space" where words/phrases are n-vectors, n is a very large number, and each dimension has semantic meaning. It may be the case that this meaning space is pretty much the same between all natural languages, which would be evidence that Sapir-Whorf is false, that differences between natural languages are largely cosmetic

numpad0 3y ago
That'll be the official explanation, but I've yet to see a working LLM that don't speak in translated American. As one possible counter example, I've seen one of 7B models insist in using a Chinese verb in a Japanese sentence, and while it's fascinating in itself, it's not necessarily in line with that "difference in languages are cosmetic and we just don't realize" narrative.