5 ms·

Do LLMs really encode knowledge regardless of language, and tap that knowledge irrespective of which language you query in? My mental model of LLMs being next-

Do LLMs really encode knowledge regardless of language, and tap that knowledge irrespective of which language you query in? My mental model of LLMs being next-word predictors with a long context window suggests they don't. Are there any papers on this?

LudwigNagasena 3y ago
What is a better way of predicting the next word if not knowledge encoding?
gabipurcaru 3y ago
this is fairly easy to test for yourself. What did you try?
- bberrry 3y ago
  I don't think it's easy at all without access to the training data. I could ask about some information I find obscure in my native language but I can't be confident someone didn't write about it in English on e.g. reddit.
- sebzim4500 3y ago
  How exactly are you proposing he tests this without access to hundreds of thousands of dollars worth of compute? Toy models don't work for this kind of thing, small language models behave qualitatively differently from large ones.
stevenhuang 3y ago
you may disabuse yourself this mental model by learning about the concept of universal function approximators and the possibility that intelligence itself can be approximated.
golol 3y ago
SGD in high dimensions is good at finding efficient, generalizing models. Naturally, it is more efficient to learn a representation of the abstract concept of a table rather than learn representations for the hundreds of different words for tables. I don't have reference but for me it feels very inthuitive that the former is the case.
DougBTX 3y ago
A big area of research in NLP is translation between natural languages, I’d go so far as to say that “translation tool” is a better mental model than “token predictor”. For example, think of summarisation as translation between verbose language and concise language, or code generation as translation between English and Python.
- HPsquared 3y ago
  They are called "transformers", after all.
the8472 3y ago
Predicting the next word is hard. To do this more efficiently than a dumb lookup table (the chinese room thought experiment) or a markov chain you basically need to build a world model. This model sits in the deeper layers of the LLMs (the deep in deep learning). Turning it into english or chinese only happens closer to the output layers. https://gwern.net/scaling-hypothesis#why-does-pretraining-work https://gwern.net/scaling-hypothesis#why-does-pretraining-wo...
sebzim4500 3y ago
Anthropic has a recent paper that talks about this [1]. See figure 16 in particular. The larger the model, the more it generalizes between languages. On the other hand, no matter how large the model they struggle to generalize between sentences of the form "George Washington was the first US president" and "The first US president was George Washington". Clearly generalization behavior is unintuitive, you can come up with a post hoc explanation (weights are independent between early and late layers) but I doubt anyone would have thought ahead of time that models would have an easier time generalizing between languages than between slightly different ways of saying the same thing in English. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf
kolinko 3y ago
There was a nice paper recently showing that even people encode the knowledge differently depending on which language they use. According to that paper, people made decisions more emotionally in their native languages compared to their second language - where they tend to be more logical. In case of LLMs there were some nice cases where GPT gave different replies depending on the language of the query. Factual information was roughly the same, but in some cases the model gave totally different replies depending on the language of the query. As a bilingual speaker - my experience is that the replies are very similar regardless of whether I speak in Polish or English, or mix them both within the same sentence.
justanotherjoe 3y ago
you say it like predicting next word is an easy task. Calculating the permutations with statistics alone would probably require more compute than the entire universe to the power of the entire universe. And there's just not enough data, even all the internet wont be enough. We abandoned these methods long ago. Yes, LLMs really encode knowledge. This is not up for debate, and it's obvious to anyone working in AI. It outputs words and input words, yes, but the magic is what is in between. The words are broken down immediately at the first layer. Next it involves 100s of billions of parameters doing god knows what. But it's safe to assume they encode the base reality as best it can. It is what i call 'language-induced reality model'. The task of next word predictor in a humongous dataset turns out, is best solved by creating an inner representation of the real world. It's still not a perfect model since it's just induced by language though, so obviously it has limitations. But it's really modeling it, and with higher fidelity than people imagined you can learn the world through words alone.
PeterisP 3y ago
Standard large transformers trained on corpora of multiple languages will generally perform next-word prediction in language A by using information that was only seen in training data in language B, demonstrating that they have managed to implicitly learn a capability for translation and/or multilingual perception.

LudwigNagasena 3y ago

What is a better way of predicting the next word if not knowledge encoding?

gabipurcaru 3y ago

this is fairly easy to test for yourself. What did you try?

bberrry 3y ago
I don't think it's easy at all without access to the training data. I could ask about some information I find obscure in my native language but I can't be confident someone didn't write about it in English on e.g. reddit.
sebzim4500 3y ago
How exactly are you proposing he tests this without access to hundreds of thousands of dollars worth of compute? Toy models don't work for this kind of thing, small language models behave qualitatively differently from large ones.

stevenhuang 3y ago

you may disabuse yourself this mental model by learning about the concept of universal function approximators and the possibility that intelligence itself can be approximated.

golol 3y ago

SGD in high dimensions is good at finding efficient, generalizing models. Naturally, it is more efficient to learn a representation of the abstract concept of a table rather than learn representations for the hundreds of different words for tables. I don't have reference but for me it feels very inthuitive that the former is the case.

DougBTX 3y ago

A big area of research in NLP is translation between natural languages, I’d go so far as to say that “translation tool” is a better mental model than “token predictor”. For example, think of summarisation as translation between verbose language and concise language, or code generation as translation between English and Python.

HPsquared 3y ago
They are called "transformers", after all.

the8472 3y ago

Predicting the next word is hard. To do this more efficiently than a dumb lookup table (the chinese room thought experiment) or a markov chain you basically need to build a world model. This model sits in the deeper layers of the LLMs (the deep in deep learning). Turning it into english or chinese only happens closer to the output layers. https://gwern.net/scaling-hypothesis#why-does-pretraining-work https://gwern.net/scaling-hypothesis#why-does-pretraining-wo...

sebzim4500 3y ago

Anthropic has a recent paper that talks about this [1]. See figure 16 in particular. The larger the model, the more it generalizes between languages. On the other hand, no matter how large the model they struggle to generalize between sentences of the form "George Washington was the first US president" and "The first US president was George Washington". Clearly generalization behavior is unintuitive, you can come up with a post hoc explanation (weights are independent between early and late layers) but I doubt anyone would have thought ahead of time that models would have an easier time generalizing between languages than between slightly different ways of saying the same thing in English. [1] https://arxiv.org/pdf/2308.03296.pdf https://arxiv.org/pdf/2308.03296.pdf

kolinko 3y ago

There was a nice paper recently showing that even people encode the knowledge differently depending on which language they use. According to that paper, people made decisions more emotionally in their native languages compared to their second language - where they tend to be more logical. In case of LLMs there were some nice cases where GPT gave different replies depending on the language of the query. Factual information was roughly the same, but in some cases the model gave totally different replies depending on the language of the query. As a bilingual speaker - my experience is that the replies are very similar regardless of whether I speak in Polish or English, or mix them both within the same sentence.

justanotherjoe 3y ago

you say it like predicting next word is an easy task. Calculating the permutations with statistics alone would probably require more compute than the entire universe to the power of the entire universe. And there's just not enough data, even all the internet wont be enough. We abandoned these methods long ago. Yes, LLMs really encode knowledge. This is not up for debate, and it's obvious to anyone working in AI. It outputs words and input words, yes, but the magic is what is in between. The words are broken down immediately at the first layer. Next it involves 100s of billions of parameters doing god knows what. But it's safe to assume they encode the base reality as best it can. It is what i call 'language-induced reality model'. The task of next word predictor in a humongous dataset turns out, is best solved by creating an inner representation of the real world. It's still not a perfect model since it's just induced by language though, so obviously it has limitations. But it's really modeling it, and with higher fidelity than people imagined you can learn the world through words alone.

PeterisP 3y ago

Standard large transformers trained on corpora of multiple languages will generally perform next-word prediction in language A by using information that was only seen in training data in language B, demonstrating that they have managed to implicitly learn a capability for translation and/or multilingual perception.