4 ms·

> extremely skeptical of Sapir-Worf, particularly the sort of stoner linguistics "what if we're all, like, made of language maaaaaan" I don't get how this kind

> extremely skeptical of Sapir-Worf, particularly the sort of stoner linguistics "what if we're all, like, made of language maaaaaan" I don't get how this kind of skepticism can exist under this current "arguably alive" LLM hype. They're machine-executable form of strong Sapir-Worf hypothesis, literally things that think* and speak solely by use of English language and English language alone(they sound quite like machine translation in other languages).

mcmoor 3y ago
Does the LLM performs differently in different languages? It'd be interesting to see a research about it.
- canjobear 3y ago
  If so it would likely be a function of amount of training data in that language.
- numpad0 3y ago
  Basically always worse in languages other than English. Not sure if it's just from volume of dataset or if it has to do with dataset quality, or the GPT architecture is inherently English-centric, but LLMs don't have like, a universal subconscious with superficial English frontend wrapping UG, like such that would support !(sapir-whorf). LLMs so far are kind of English-based thinking machine(if we were to recognize their apparent behavior as "thinking"). Below is just cherry-picked search results, selected largely by whether last few lines in the abstracts support my narrative, but I mean, it's a problem obvious enough that the rest of the world just knows. 0: "Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting": https://arxiv.org/abs/2305.07004 https://arxiv.org/abs/2305.07004 1: "Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries", https://arxiv.org/abs/2310.13132 https://arxiv.org/abs/2310.13132 2: "Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test": https://arxiv.org/abs/2402.02135 https://arxiv.org/abs/2402.02135 3: "Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance": https://arxiv.org/abs/2402.14531 https://arxiv.org/abs/2402.14531 4: "Exploring Multilingual Human Value Concepts in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?": https://arxiv.org/abs/2402.18120 https://arxiv.org/abs/2402.18120 5: "How do Large Language Models Handle Multilingualism?": https://arxiv.org/abs/2402.18815 https://arxiv.org/abs/2402.18815
slibhb 3y ago
Is that the right way to think of LLMs (that they "think in English")? I think a better way to think of them is as an n-dimensional "meaning space" where words/phrases are n-vectors, n is a very large number, and each dimension has semantic meaning. It may be the case that this meaning space is pretty much the same between all natural languages, which would be evidence that Sapir-Whorf is false, that differences between natural languages are largely cosmetic
- numpad0 3y ago
  That'll be the official explanation, but I've yet to see a working LLM that don't speak in translated American. As one possible counter example, I've seen one of 7B models insist in using a Chinese verb in a Japanese sentence, and while it's fascinating in itself, it's not necessarily in line with that "difference in languages are cosmetic and we just don't realize" narrative.
canjobear 3y ago
No, they’re multilingual.
ketralnis 3y ago
Shrug, you can read the book and find out
ecronant 3y ago
[flagged]
mistermann 3y ago
> I don't get how this kind of skepticism can exist... How are LLM's trained? How are humans trained?

mcmoor 3y ago

Does the LLM performs differently in different languages? It'd be interesting to see a research about it.

canjobear 3y ago
If so it would likely be a function of amount of training data in that language.
numpad0 3y ago
Basically always worse in languages other than English. Not sure if it's just from volume of dataset or if it has to do with dataset quality, or the GPT architecture is inherently English-centric, but LLMs don't have like, a universal subconscious with superficial English frontend wrapping UG, like such that would support !(sapir-whorf). LLMs so far are kind of English-based thinking machine(if we were to recognize their apparent behavior as "thinking"). Below is just cherry-picked search results, selected largely by whether last few lines in the abstracts support my narrative, but I mean, it's a problem obvious enough that the rest of the world just knows. 0: "Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting": https://arxiv.org/abs/2305.07004 https://arxiv.org/abs/2305.07004 1: "Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries", https://arxiv.org/abs/2310.13132 https://arxiv.org/abs/2310.13132 2: "Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A Study using the Multilingual Defining Issues Test": https://arxiv.org/abs/2402.02135 https://arxiv.org/abs/2402.02135 3: "Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance": https://arxiv.org/abs/2402.14531 https://arxiv.org/abs/2402.14531 4: "Exploring Multilingual Human Value Concepts in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages?": https://arxiv.org/abs/2402.18120 https://arxiv.org/abs/2402.18120 5: "How do Large Language Models Handle Multilingualism?": https://arxiv.org/abs/2402.18815 https://arxiv.org/abs/2402.18815

slibhb 3y ago

Is that the right way to think of LLMs (that they "think in English")? I think a better way to think of them is as an n-dimensional "meaning space" where words/phrases are n-vectors, n is a very large number, and each dimension has semantic meaning. It may be the case that this meaning space is pretty much the same between all natural languages, which would be evidence that Sapir-Whorf is false, that differences between natural languages are largely cosmetic

numpad0 3y ago
That'll be the official explanation, but I've yet to see a working LLM that don't speak in translated American. As one possible counter example, I've seen one of 7B models insist in using a Chinese verb in a Japanese sentence, and while it's fascinating in itself, it's not necessarily in line with that "difference in languages are cosmetic and we just don't realize" narrative.

canjobear 3y ago

No, they’re multilingual.

ketralnis 3y ago

Shrug, you can read the book and find out

ecronant 3y ago

[flagged]

mistermann 3y ago

> I don't get how this kind of skepticism can exist... How are LLM's trained? How are humans trained?