11 ms·

Linear algebra explains why some words are effectively untranslatable

171 points by mrcgnc 11mo ago

zvmaz 11mo ago
> I hope these rather unorthodox leaps between linguistics and mathematics helped make it almost obvious that some words and ideas are untranslatable in practice. I also hope you don't take the analogy too seriously, because it won't go much further than this. Phew! Thanks for clarifying.
d-lisp 11mo ago
Are they multiplying a 3x3 matrix by a 2 component vector ?
- tptacek 11mo ago
  In that one case, yeah; I don't think they're going for anything more than general illustration here.
  - magicalhippo 11mo ago
    The text that follows does take on a new meaning though, for those that know linear algebra: If the mere sight of the above is like a punch in the face for you, don't worry. Almost makes me wonder if it was intentional.
    - tptacek 11mo ago
      Bears. Beets. Battlestar Galactica!
  - nyeah 11mo ago
    But what does that illustrate?
    - tptacek 11mo ago
      Like, what a matrix looks like? Seems fine.
      - nyeah 11mo ago
        But, as a few people have pointed out, that particular matrix product is undefined. It's broken. That's really not quite what a matrix-vector product looks like. The author says he has PhD. That's fine. I suffer from the same condition. But it feels off to hear about expertise and see an invalid first example.
- seanhunter 11mo ago
  Yeah that made me twitch also.
- wjholden 11mo ago
  It also has mixed square brackets and curved parentheses. I stopped reading the article when I saw this.
  - nyeah 11mo ago
    Brackets can be any shape, it's fine. Like (3)*[4] is still 12. But that matrix-vector product is undefined.
- moron4hire 11mo ago
  Everyone knows you need a 4x4 matrix to do translation, anyway. Now, scale, rotation, and skew...
mannyv 11mo ago
Communication/language depends on shared context. The more context you share the shorter the trigger for evoking that thing and that context. And if you share no context communication becomes very difficult. I wasn't aware that that idea was in dispute.
- pixl97 11mo ago
  And honestly, without a lot more communication even with a person that speaks your language you have no idea if you actually have a shared context. While an American from NYC and one from some backwater town in Kansas share a lot of context but there is a lot of context they don't, so as communication becomes more detailed between them it's very likely that 'translation' between each other will be somewhat incorrect. This is also why lawyer speak is so particular. Language is fuzzy in most cases. Only language that relates to discrete physical objects gets closer to the binary state of exactness described in the article.
pjsg 11mo ago
The article seems to think that a word is untranslateable if there is no single word in the target language. If I'm not misreading the article, then this is completely obvious -- just consider the number of words in English and the number of words in almost any other language, and you will find that there are more English words than the other language. It is now clear that there exist English words that don't correspond to a single word in the other language.
- drivebyhooting 11mo ago
  That isn’t a proof. Synonyms can bolster the enumeration sans augmenting novelty.
  - manwe150 11mo ago
    That is the crux of the article premise: each synonym conveys similar denotations (principle component is I think what the article called it), but usually with some difference in connotations (the off axis contributions). You can nudge the languages vectors towards each other by adding enough synonyms and modifiers together, but they are always a little bit off even still
    - nomel 11mo ago
      So, really, this can be simplified to the question "can written text fully convey all human concepts", some of which having labels in only some languages, which is an obvious "no".
  - James_K 11mo ago
    I think the reasonable reader will conclude it's unlikely for any two languages to share exactly the same vocabulary, accounting for synonyms.
  - mannykannot 11mo ago
    True, but many languages now have words that were absent from their earlier vocabularies. Shakespeare did not have the option to use 'telephone', 'semiconductor' or 'entropy'.
behnamoh 11mo ago
Tangent: I really like vornoi diagrams and part of me thinks there's a hidden, precious concept they represent. I didn't get their relation to the article but was wondering if they have applications in engineering/sciences.
- HanClinto 11mo ago
  A Voronoi diagram is created when you color every point on an image according to which discrete point it is closest to. So in this case, I see the diagrams as representing the boundaries drawn when projecting / quantizing complex ideas into a set of central points that are insufficient for catching all of the nuance of the original. How well can you adapt a nuanced idea to a different space? If Language A has an idea that exists at one point in space, which is the closest word in Language B that might be used to represent it? A Voronoi diagram is one possible way of illustrating it. Tangent on your tangent: this GDC presentation from 2016 is probably my favorite real-world application of Voronoi Diagrams, and uses them for N-player split-screen camera control: https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s I have a lingering dream in the back of my mind to make a single-couch Liero-style casual game for N-players with good dynamic camera support using this technique.
rich_sasha 11mo ago
It's a tenuous analogy, but if you along with it, you can take it further. You could consider the "cost" of expressing a word as some kind of metric or norm on the vector. What in one language/basis is a simple Kronecker delta, in another is a very complex vector (of course if it were the same vector in two bases, it would have the same length, but we could rather think of translation as an affine transformation, say). And finally, with two bases, they need not span the same vector space. You can have a three-coordinate vector space all you like, if you have only two basis vectors you ain't spanning it. At best you can hope for an orthogonal projection from one to the other, and lose some nuance. Eventually, with bilinguality, you learn not to translate words. Concepts live in different languages and describe a reality. Usually you can describe that reality in two different languages, but sometimes not.
proteal 11mo ago
I think a succinct way to describe my thoughts on linear algebra/language is that language has high dimensionality (ie many different basis vectors that may not necessarily be orthogonal) and that individual languages use a unique coordinate system to express thought. Each language is a lossy approximation of all conceivable thought and some languages can more efficiently represent the “all thoughts” vector space because they have basis vectors that point in more uncommon directions (like the go to japan example). So while you can more or less point to any thought in any language, some thoughts are easier to express in certain languages, which the post (and me) agree to be untranslatable words. I tried to find the really interesting article about language and color that describes how some cultures use different naming schemes for colors but couldn’t find it. It talked about how back in the day we don’t know orange as a color, we just thought it was red-yellow and only after the fruit was distributed did the word for the color catch on. Here’s the best article I can find that talks about this phenomena https://burnaway.org/magazine/blue-language-visual-perception/ https://burnaway.org/magazine/blue-language-visual-perceptio...
- MangoToupe 11mo ago
  > Each language is a lossy approximation of all conceivable thought I'm not quite sure I understand this—I do have mental sensations/processes sans language, but I would not characterize them as "thoughts". To me, a thought is inherently linguistic, even if they relate to non-linguistic mental processes. So to me, learning a new language is very literally learning how to think differently.
  - proteal 11mo ago
    I think we’re in agreement, but I’m afraid I don’t have the philosophical language to precisely pin my mental model into words (what a meta conundrum lol). I’ll try my best here, but I may come back in a few days with an edit if I can more coherently write my ideas. I take a slightly more narrow definition of “thoughts” that may be more akin to “expressions” - ideas that can be communicated, so excluding non-linguistic mental processes. I think that may be where we disconnect. A lot of my idea about thoughts comes from the Borges story, Funes the memorius (short story about a dude who could not forget - interesting read and really clarifies my feelings on my definition of “all possible thought”). In the story he talks about tree leaves, but instead imagine needing a unique linguistic scheme for every single unique snowflake you ever see. It would be a linguistic nightmare! Therefore language must generalize otherwise it becomes noncommunicable and that generalization to me induces the “lossy approximation” I attribute to language in my prior comment. So, in my head Funes’s mind represent the abstract space of all possible thoughts. When we use language, we are stacking words/sentences/paragraphs/etc together almost like vector addition trying to reach a particular point in the thought vector space. Some languages have really clean ways of getting to certain thoughts while others take a mouthful and still don’t get you exactly there (物の哀れ example from link). I agree with your statement on new languages being different thinking. As you follow that vector addition process to get to the “thought,” different languages will take you on different paths to get to your destination thought because languages encode those vectors differently, even if the destination thought is the same. In my mental model, the act of thinking is putting those language vectors together and tracing their path to get to your thought. And if my comment still makes no sense - I might have to incubate this thought a bit more :) but I do recommend the story- it’s a quick, thought provoking read.
epistasis 11mo ago
> If the mere sight of the above is like a punch in the face for you, don't worry. I'm not going to math you to death in what follows. I will only remind you of a tiny basic part of it that I think relates to languages. Yes, that mathematical expression is like a punch in my face, but not for the reason you think. I am offended that the rank of the matrix does not match the dimension of the matrix, not that I'm seeing a matrix.
- tptacek 11mo ago
  It's a 3x3 matrix with 3 independent rows. The rank matches the dimension.
  - deleted 11mo ago
    [deleted]
  - deleted 11mo ago
    [deleted]
  - KalMann 11mo ago
    He probably meant to say "vector" the second time he said "matrix".
- trostaft 11mo ago
  You probably mean that the size of the matrix is incompatible with the size of the vector?
  - epistasis 11mo ago
    Yes, and embarrassingly there are two mistakes in the comment! I used "rank" of the matrix rather than dimension too.
triclops200 11mo ago
This article assumes that concepts are somehow precise coordinates within a single language; that's not the case, at best, speakers of a language mutually approximate a relatively consistent representation, but like, look at a word like yeet or whatever: we decided as a society on its meaning while it was being developed, as it were. Furthermore, it never rigorously defines what it means by translation. It claims 上京 is a single basis meaning moving to Tokyo, for example, but that isn't even an accurate translation: the individual components represent superior/greater/above and Tokyo and as an idiomatic phrase it represents the concept of moving to the capital for a better life. Something like "moving on up" or the like in some vernaculars of English, and idioms translating to idioms is a form of translation. It's disingenuous to represent the first concept as a single basis but not the second. Similarly, it claims mono no aware (物の哀れ) is unable to be translated, but, again, more literally "translated" is saying "the sorrow within things" character by character, and, only as an idiom has the full contextual understanding. It's not really a single point even if it's rather accurately located in a hypothetical embedding space by Japanese speakers. Imo, an English translation of the concept is "everything is dust in the wind", only 2 more individual conceptual units than the original Japanese phrase, and 3 of them are mainly just connecting words, but it's understood as a similar idiom/concept, here. Concepts are only usefully distinguished by context and use. By the author's own argumentation: nothing is translatable (or, generally, even communicatable) unless it has a fixed relative configuration to all other concepts that is precisely equivalent. In practice, we handle the fuzziness as part of communication and its useless to try and define a concept as untranslatable unless you're also of the camp that nothing is ever communicated (in which case, this response to the author's post is completely useless as nobody could possibly understand it enough internally for it to be useful. If you've read this far, congrats on squaring the circle somehow)
- DFHippie 11mo ago
  This. Two speakers of the same language only have approximately the same understanding of the meanings of the words they both use. Communication succeeds because we are constantly seeking and correcting misunderstandings that arise due to no two people speaking exactly the same language. The same process that allows two speakers of the same language to communicate adequately allows one to translate from one language to another. If it were truly impossible to translate from one language to another, we would be unable to perceive this and argue about it. The recognition and correction of errors is part of the process of translation just as it is part of the process of communication in a single language.
raincom 11mo ago
There is a better thesis coming from the late philosopher W.V.O Quine: indeterminacy of translation [1] [1] https://plato.stanford.edu/entries/quine/#IndeTran https://plato.stanford.edu/entries/quine/#IndeTran
- scoofy 11mo ago
  Ctrl+F: Quine That indeterminacy of translation isn’t mentioned is a huge shortcoming of this article.
nph278 11mo ago
Just read Wittgenstein (The Blue+Brown Books / Philosophical investigations), and this confusion will go away. The difference between translation, definition, and explanation needs to be understood.
- mcswell 11mo ago
  I'm thinking this is a joke, right? Wittgenstein always seemed to me to be a good way to get confused.
ralferoo 11mo ago
There's one aspect that I think the article starts to hint at, but doesn't quite make the jump to is that words in a language just map to a subset of concepts that don't necessarily have the same subset boundaries in other languages. If you think back to the meme from a decade or two ago about how men and women perceive colour [1], where e.g. "pink" to a man covers a whole range of colours to a woman, then that kind of hints at the idea. One example back in the realm of vocabulary is the English word "happy". This embodies a range of meanings from joy, willingness, pleased, contentment, satiation, etc. There might be some overlap in some of these meanings with other words like "joy" or "excited", that don't have the same overlaps in other languages. E.g. "happy" might be translated to French as "heureuse" for the senses of pleased or content, but not for willingness sense. Similarly, the French word "dommage" can be translated into a whole bunch of English words that aren't normally synonyms of each other - pity, damage, shame, harm. This kind of nuance can lead to two opposite problems when translating - when the meaning is limited to a subset of possible meanings by context, and the wrong one is chosen in the foreign language, and when the author's meaning embodied multiple meanings and the chosen translation doesn't cover all of them. Some of these features can lead to the humour in subtle jokes being lost in translation, e.g. "he'd be late to his own funeral". [1] e.g. https://www.psychologytoday.com/us/blog/brain-babble/201504/when-it-comes-to-color-men-women-arent-seeing-eye-to-eye https://www.psychologytoday.com/us/blog/brain-babble/201504/... or https://digitalsynopsis.com/design/male-vs-female-color-perceptions-preferences/ https://digitalsynopsis.com/design/male-vs-female-color-perc...
voxleone 11mo ago
My personal analogy, useful in my early days: Translating is like finding a vector in another space that points in the same direction or carries a similar magnitude of meaning. In other words: The source sentence is a vector in “language A space.” The target sentence is a vector in “language B space.” A good translation finds a vector that has the same direction (same meaning, intent, tone) even though it lies in a different coordinate system (the new language).
- aitchnyu 11mo ago
  when did you develop this analogy? Is it well before 2015, when Google demoed a vector model that solved Man:Woman,King:_____ ?
  - nyeah 11mo ago
    Yeah, I was hoping the article would say something about word vectors and linear algebra.
- energy123 11mo ago
  I know what you mean, but semantics is about relative positions of points in a given space. Comparing two points from two different spaces is apples and oranges. I feel like this analogy should be salvageable with a small tweak, however.
midtake 11mo ago
What a trendy article, in tune with our recently linear-algebraic turn in how we see language thanks to LLM's. But I think this exposes an even greater problem, where words thought to be direct translations will always drift in vector value as they are weighted for attention within their respective corpora. Are we on the brink of translation-nihilism? This isn't even limited to complex phenomena or shades of snow. Even "I like" is a different construction in many languages, in an unexpected way to new language learners.
sigbottle 11mo ago
It could be the case that it's not even "effectively", "in practice", etc. N^{any constant} is not bijective with a single R.
yubblegum 11mo ago
Interestingly enough for this morning's walk I was musing over the tension between the hypotheses that: 'LLMs can map between languages in the vector space' (thus languages are ~equivalent); and 'Language affects thoughts' (as in German is good for Philosophy and English for getting things done). If both these thoughts are true, then it would appear that languages have topological characteristics. We can (topologically) map from one to another, 'thoughts' (that is a complex of words) form 'paths on the language manifold' and certain paths may be more 'natural' in one topological form than the other.
- andoando 11mo ago
  My take is the human brain learns concepts primarily through differentiation. To a newly born child who has no concept of door or a wall, has no reason to see the two as being different parts. Different languages form different differentiations, but one can always compound concepts, and differentiate them differently.
  - grokgrok 11mo ago
    To extend: there will also be general alignment tendencies towards those readily mapped and expressed concepts within available language. Hard but useful concepts can get mapped to idioms. Modes of categorization will be influenced by these factors, which in turn influences many processes.
- thesz 11mo ago
  What is a word in one language is a collocated words in another, possibly context-dependent. We can look no further than English: "man can do something," "man can do not do something" (i.e., can do but does not have to), then pretty straightforward "man can not do something" and, all of sudden, to express that man cannot decline some obligation, we say "man can not help but do." It is not translation per se, but shows that some parts of language were evolved to tiptoe around non-customary things, in this case, double negation. And double negation is very easy in some other languages.
- mcswell 11mo ago
  "'Language affects thoughts' (as in German is good for Philosophy and English for getting things done). If both these thoughts are true..." Well, the second one isn't true (I omitted the first one in this reply). It is simply not the case that German is good for philosophy and English for getting things done, and similarly for most other such claims (French is better for talking about love, Italian is better for operas, etc. etc.).
miltonlost 11mo ago
Reading any poem that makes use of extensive wordplay within a language shows why there will always be some untranslatable aspect. You can't create all the exact shades of a single pun if all those shades aren't in a different language. Go translate an ee cummings poem and make sure to retain all its meanings.
- mcswell 11mo ago
  Douglas Hofstedter's book The Ton beau de Marot.
KolenCh 11mo ago
Big claim but not much substance. They should try to really understand linear algebra first, and also linguistics a bit. Semantic domain (from linguistics) is a better way to describe it, where using sets (from math) might better convey what they want to say.
- znkr 11mo ago
  The whole article starts with the implicit assumption that all bases are orthonormal and then throws linear algebra out of the window completely.
deleted 11mo ago
[deleted]
tgv 11mo ago
Another physicist who thinks he can solve problems in a domain he knows nothing about with linear approximations. There's an xkcd devoted to this problem, even using computational linguistics as an example, IIRC.
mcswell 11mo ago
The word 'word' is polysemous, and also vague. Polysemous: it can refer to written words or spoken words (or signed words, in sign languages). Vague: does it mean all the inflected forms of a word, or just the stem without inflection? Example: is (are?) 'walk', 'walked', 'walks' and 'walking' four words, or one? What about "stand/stood"? (And languages where the bare root or stem can't appear by itself, like verbs in Spanish.) Derived words, like 'push' and 'pushy'. Do compound nouns count as a word, or do only the parts count? Example: 'doghouse' (or 'dog house'). What about idioms? Example: 'to crane his/her/my/your/our neck(s)'. What about different pronunciations? Is 'roof' pronounced to rhyme with 'aloof' the same word as 'roof' pronounced with the vowel of 'put'? And different spellings but same pronunciation: 'bear' vs. 'bare'. What about words with different grammatical categories, like 'push' as a noun ("I gave her a push") or a verb ("I pushed her"). Or the same word with virtually unrelated meanings, "I pushed her on the swing" vs. "I pushed my ideas."
sevensor 11mo ago
I like this as an analogy but not as an explanation. In fact, if you’re unfamiliar with linear algebra, this might be a nice way to think about projection onto a different set of basis vectors. But even the best human translators can be deeply at odds over what translation is appropriate for a term in its context. There’s never a right translation, let alone a uniquely right translation.
evolextra 11mo ago
[dead]
egl2020 11mo ago
From the title, I thought he was going to explain "eigenvalue".
fnord77 11mo ago
I cannot take seriously an article that presents a 3x3 matrix being multiplied by a 2-vector as an example of linear algebra. Gibberish.
decimalenough 11mo ago
上京 jōkyō is less a regular word and more a form of written shorthand. It would not be used in speech, and even in writing it's ambiguous: not only does 京 kyō simply mean "capital", but there's a district of Kyoto that's also 上京, only read Kamigyō. The closest English equivalent is abbreviations like "PC". They're perfectly usable in context, but if you see one standing alone it's not clear if it's personal computer, politically correct, Peace Corps, etc etc.
throwaway81523 11mo ago
> a gentle, poignant sadness or pathos felt in response to the transient nature of all things, a deep awareness of their impermanence that evokes a subtle, bittersweet sorrow and a profound, quiet empathy for their passing. https://en.wikipedia.org/wiki/Saudade https://en.wikipedia.org/wiki/Saudade ?
deleted 11mo ago
[deleted]

171 points by mrcgnc 11mo ago

zvmaz 11mo ago

> I hope these rather unorthodox leaps between linguistics and mathematics helped make it almost obvious that some words and ideas are untranslatable in practice. I also hope you don't take the analogy too seriously, because it won't go much further than this. Phew! Thanks for clarifying.

d-lisp 11mo ago

Are they multiplying a 3x3 matrix by a 2 component vector ?

tptacek 11mo ago
In that one case, yeah; I don't think they're going for anything more than general illustration here.
- magicalhippo 11mo ago
  The text that follows does take on a new meaning though, for those that know linear algebra: If the mere sight of the above is like a punch in the face for you, don't worry. Almost makes me wonder if it was intentional.
  - tptacek 11mo ago
    Bears. Beets. Battlestar Galactica!
- nyeah 11mo ago
  But what does that illustrate?
  - tptacek 11mo ago
    Like, what a matrix looks like? Seems fine.
    - nyeah 11mo ago
      But, as a few people have pointed out, that particular matrix product is undefined. It's broken. That's really not quite what a matrix-vector product looks like. The author says he has PhD. That's fine. I suffer from the same condition. But it feels off to hear about expertise and see an invalid first example.
seanhunter 11mo ago
Yeah that made me twitch also.
wjholden 11mo ago
It also has mixed square brackets and curved parentheses. I stopped reading the article when I saw this.
- nyeah 11mo ago
  Brackets can be any shape, it's fine. Like (3)*[4] is still 12. But that matrix-vector product is undefined.
moron4hire 11mo ago
Everyone knows you need a 4x4 matrix to do translation, anyway. Now, scale, rotation, and skew...

mannyv 11mo ago

Communication/language depends on shared context. The more context you share the shorter the trigger for evoking that thing and that context. And if you share no context communication becomes very difficult. I wasn't aware that that idea was in dispute.

pixl97 11mo ago
And honestly, without a lot more communication even with a person that speaks your language you have no idea if you actually have a shared context. While an American from NYC and one from some backwater town in Kansas share a lot of context but there is a lot of context they don't, so as communication becomes more detailed between them it's very likely that 'translation' between each other will be somewhat incorrect. This is also why lawyer speak is so particular. Language is fuzzy in most cases. Only language that relates to discrete physical objects gets closer to the binary state of exactness described in the article.

pjsg 11mo ago

The article seems to think that a word is untranslateable if there is no single word in the target language. If I'm not misreading the article, then this is completely obvious -- just consider the number of words in English and the number of words in almost any other language, and you will find that there are more English words than the other language. It is now clear that there exist English words that don't correspond to a single word in the other language.

drivebyhooting 11mo ago
That isn’t a proof. Synonyms can bolster the enumeration sans augmenting novelty.
- manwe150 11mo ago
  That is the crux of the article premise: each synonym conveys similar denotations (principle component is I think what the article called it), but usually with some difference in connotations (the off axis contributions). You can nudge the languages vectors towards each other by adding enough synonyms and modifiers together, but they are always a little bit off even still
  - nomel 11mo ago
    So, really, this can be simplified to the question "can written text fully convey all human concepts", some of which having labels in only some languages, which is an obvious "no".
- James_K 11mo ago
  I think the reasonable reader will conclude it's unlikely for any two languages to share exactly the same vocabulary, accounting for synonyms.
- mannykannot 11mo ago
  True, but many languages now have words that were absent from their earlier vocabularies. Shakespeare did not have the option to use 'telephone', 'semiconductor' or 'entropy'.

behnamoh 11mo ago

Tangent: I really like vornoi diagrams and part of me thinks there's a hidden, precious concept they represent. I didn't get their relation to the article but was wondering if they have applications in engineering/sciences.

HanClinto 11mo ago
A Voronoi diagram is created when you color every point on an image according to which discrete point it is closest to. So in this case, I see the diagrams as representing the boundaries drawn when projecting / quantizing complex ideas into a set of central points that are insufficient for catching all of the nuance of the original. How well can you adapt a nuanced idea to a different space? If Language A has an idea that exists at one point in space, which is the closest word in Language B that might be used to represent it? A Voronoi diagram is one possible way of illustrating it. Tangent on your tangent: this GDC presentation from 2016 is probably my favorite real-world application of Voronoi Diagrams, and uses them for N-player split-screen camera control: https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s https://www.youtube.com/watch?v=tu-Qe66AvtY&t=1594s I have a lingering dream in the back of my mind to make a single-couch Liero-style casual game for N-players with good dynamic camera support using this technique.

rich_sasha 11mo ago

It's a tenuous analogy, but if you along with it, you can take it further. You could consider the "cost" of expressing a word as some kind of metric or norm on the vector. What in one language/basis is a simple Kronecker delta, in another is a very complex vector (of course if it were the same vector in two bases, it would have the same length, but we could rather think of translation as an affine transformation, say). And finally, with two bases, they need not span the same vector space. You can have a three-coordinate vector space all you like, if you have only two basis vectors you ain't spanning it. At best you can hope for an orthogonal projection from one to the other, and lose some nuance. Eventually, with bilinguality, you learn not to translate words. Concepts live in different languages and describe a reality. Usually you can describe that reality in two different languages, but sometimes not.

proteal 11mo ago

I think a succinct way to describe my thoughts on linear algebra/language is that language has high dimensionality (ie many different basis vectors that may not necessarily be orthogonal) and that individual languages use a unique coordinate system to express thought. Each language is a lossy approximation of all conceivable thought and some languages can more efficiently represent the “all thoughts” vector space because they have basis vectors that point in more uncommon directions (like the go to japan example). So while you can more or less point to any thought in any language, some thoughts are easier to express in certain languages, which the post (and me) agree to be untranslatable words. I tried to find the really interesting article about language and color that describes how some cultures use different naming schemes for colors but couldn’t find it. It talked about how back in the day we don’t know orange as a color, we just thought it was red-yellow and only after the fruit was distributed did the word for the color catch on. Here’s the best article I can find that talks about this phenomena https://burnaway.org/magazine/blue-language-visual-perception/ https://burnaway.org/magazine/blue-language-visual-perceptio...

MangoToupe 11mo ago
> Each language is a lossy approximation of all conceivable thought I'm not quite sure I understand this—I do have mental sensations/processes sans language, but I would not characterize them as "thoughts". To me, a thought is inherently linguistic, even if they relate to non-linguistic mental processes. So to me, learning a new language is very literally learning how to think differently.
- proteal 11mo ago
  I think we’re in agreement, but I’m afraid I don’t have the philosophical language to precisely pin my mental model into words (what a meta conundrum lol). I’ll try my best here, but I may come back in a few days with an edit if I can more coherently write my ideas. I take a slightly more narrow definition of “thoughts” that may be more akin to “expressions” - ideas that can be communicated, so excluding non-linguistic mental processes. I think that may be where we disconnect. A lot of my idea about thoughts comes from the Borges story, Funes the memorius (short story about a dude who could not forget - interesting read and really clarifies my feelings on my definition of “all possible thought”). In the story he talks about tree leaves, but instead imagine needing a unique linguistic scheme for every single unique snowflake you ever see. It would be a linguistic nightmare! Therefore language must generalize otherwise it becomes noncommunicable and that generalization to me induces the “lossy approximation” I attribute to language in my prior comment. So, in my head Funes’s mind represent the abstract space of all possible thoughts. When we use language, we are stacking words/sentences/paragraphs/etc together almost like vector addition trying to reach a particular point in the thought vector space. Some languages have really clean ways of getting to certain thoughts while others take a mouthful and still don’t get you exactly there (物の哀れ example from link). I agree with your statement on new languages being different thinking. As you follow that vector addition process to get to the “thought,” different languages will take you on different paths to get to your destination thought because languages encode those vectors differently, even if the destination thought is the same. In my mental model, the act of thinking is putting those language vectors together and tracing their path to get to your thought. And if my comment still makes no sense - I might have to incubate this thought a bit more :) but I do recommend the story- it’s a quick, thought provoking read.

epistasis 11mo ago

> If the mere sight of the above is like a punch in the face for you, don't worry. I'm not going to math you to death in what follows. I will only remind you of a tiny basic part of it that I think relates to languages. Yes, that mathematical expression is like a punch in my face, but not for the reason you think. I am offended that the rank of the matrix does not match the dimension of the matrix, not that I'm seeing a matrix.

tptacek 11mo ago
It's a 3x3 matrix with 3 independent rows. The rank matches the dimension.
- deleted 11mo ago
  [deleted]
- deleted 11mo ago
  [deleted]
- KalMann 11mo ago
  He probably meant to say "vector" the second time he said "matrix".
trostaft 11mo ago
You probably mean that the size of the matrix is incompatible with the size of the vector?
- epistasis 11mo ago
  Yes, and embarrassingly there are two mistakes in the comment! I used "rank" of the matrix rather than dimension too.

triclops200 11mo ago

This article assumes that concepts are somehow precise coordinates within a single language; that's not the case, at best, speakers of a language mutually approximate a relatively consistent representation, but like, look at a word like yeet or whatever: we decided as a society on its meaning while it was being developed, as it were. Furthermore, it never rigorously defines what it means by translation. It claims 上京 is a single basis meaning moving to Tokyo, for example, but that isn't even an accurate translation: the individual components represent superior/greater/above and Tokyo and as an idiomatic phrase it represents the concept of moving to the capital for a better life. Something like "moving on up" or the like in some vernaculars of English, and idioms translating to idioms is a form of translation. It's disingenuous to represent the first concept as a single basis but not the second. Similarly, it claims mono no aware (物の哀れ) is unable to be translated, but, again, more literally "translated" is saying "the sorrow within things" character by character, and, only as an idiom has the full contextual understanding. It's not really a single point even if it's rather accurately located in a hypothetical embedding space by Japanese speakers. Imo, an English translation of the concept is "everything is dust in the wind", only 2 more individual conceptual units than the original Japanese phrase, and 3 of them are mainly just connecting words, but it's understood as a similar idiom/concept, here. Concepts are only usefully distinguished by context and use. By the author's own argumentation: nothing is translatable (or, generally, even communicatable) unless it has a fixed relative configuration to all other concepts that is precisely equivalent. In practice, we handle the fuzziness as part of communication and its useless to try and define a concept as untranslatable unless you're also of the camp that nothing is ever communicated (in which case, this response to the author's post is completely useless as nobody could possibly understand it enough internally for it to be useful. If you've read this far, congrats on squaring the circle somehow)

DFHippie 11mo ago
This. Two speakers of the same language only have approximately the same understanding of the meanings of the words they both use. Communication succeeds because we are constantly seeking and correcting misunderstandings that arise due to no two people speaking exactly the same language. The same process that allows two speakers of the same language to communicate adequately allows one to translate from one language to another. If it were truly impossible to translate from one language to another, we would be unable to perceive this and argue about it. The recognition and correction of errors is part of the process of translation just as it is part of the process of communication in a single language.

raincom 11mo ago

There is a better thesis coming from the late philosopher W.V.O Quine: indeterminacy of translation [1] [1] https://plato.stanford.edu/entries/quine/#IndeTran https://plato.stanford.edu/entries/quine/#IndeTran

scoofy 11mo ago
Ctrl+F: Quine That indeterminacy of translation isn’t mentioned is a huge shortcoming of this article.

nph278 11mo ago

Just read Wittgenstein (The Blue+Brown Books / Philosophical investigations), and this confusion will go away. The difference between translation, definition, and explanation needs to be understood.

mcswell 11mo ago
I'm thinking this is a joke, right? Wittgenstein always seemed to me to be a good way to get confused.

ralferoo 11mo ago

There's one aspect that I think the article starts to hint at, but doesn't quite make the jump to is that words in a language just map to a subset of concepts that don't necessarily have the same subset boundaries in other languages. If you think back to the meme from a decade or two ago about how men and women perceive colour [1], where e.g. "pink" to a man covers a whole range of colours to a woman, then that kind of hints at the idea. One example back in the realm of vocabulary is the English word "happy". This embodies a range of meanings from joy, willingness, pleased, contentment, satiation, etc. There might be some overlap in some of these meanings with other words like "joy" or "excited", that don't have the same overlaps in other languages. E.g. "happy" might be translated to French as "heureuse" for the senses of pleased or content, but not for willingness sense. Similarly, the French word "dommage" can be translated into a whole bunch of English words that aren't normally synonyms of each other - pity, damage, shame, harm. This kind of nuance can lead to two opposite problems when translating - when the meaning is limited to a subset of possible meanings by context, and the wrong one is chosen in the foreign language, and when the author's meaning embodied multiple meanings and the chosen translation doesn't cover all of them. Some of these features can lead to the humour in subtle jokes being lost in translation, e.g. "he'd be late to his own funeral". [1] e.g. https://www.psychologytoday.com/us/blog/brain-babble/201504/when-it-comes-to-color-men-women-arent-seeing-eye-to-eye https://www.psychologytoday.com/us/blog/brain-babble/201504/... or https://digitalsynopsis.com/design/male-vs-female-color-perceptions-preferences/ https://digitalsynopsis.com/design/male-vs-female-color-perc...

voxleone 11mo ago

My personal analogy, useful in my early days: Translating is like finding a vector in another space that points in the same direction or carries a similar magnitude of meaning. In other words: The source sentence is a vector in “language A space.” The target sentence is a vector in “language B space.” A good translation finds a vector that has the same direction (same meaning, intent, tone) even though it lies in a different coordinate system (the new language).

aitchnyu 11mo ago
when did you develop this analogy? Is it well before 2015, when Google demoed a vector model that solved Man:Woman,King:_____ ?
- nyeah 11mo ago
  Yeah, I was hoping the article would say something about word vectors and linear algebra.
energy123 11mo ago
I know what you mean, but semantics is about relative positions of points in a given space. Comparing two points from two different spaces is apples and oranges. I feel like this analogy should be salvageable with a small tweak, however.

midtake 11mo ago

What a trendy article, in tune with our recently linear-algebraic turn in how we see language thanks to LLM's. But I think this exposes an even greater problem, where words thought to be direct translations will always drift in vector value as they are weighted for attention within their respective corpora. Are we on the brink of translation-nihilism? This isn't even limited to complex phenomena or shades of snow. Even "I like" is a different construction in many languages, in an unexpected way to new language learners.

sigbottle 11mo ago

It could be the case that it's not even "effectively", "in practice", etc. N^{any constant} is not bijective with a single R.

yubblegum 11mo ago

Interestingly enough for this morning's walk I was musing over the tension between the hypotheses that: 'LLMs can map between languages in the vector space' (thus languages are ~equivalent); and 'Language affects thoughts' (as in German is good for Philosophy and English for getting things done). If both these thoughts are true, then it would appear that languages have topological characteristics. We can (topologically) map from one to another, 'thoughts' (that is a complex of words) form 'paths on the language manifold' and certain paths may be more 'natural' in one topological form than the other.

andoando 11mo ago
My take is the human brain learns concepts primarily through differentiation. To a newly born child who has no concept of door or a wall, has no reason to see the two as being different parts. Different languages form different differentiations, but one can always compound concepts, and differentiate them differently.
- grokgrok 11mo ago
  To extend: there will also be general alignment tendencies towards those readily mapped and expressed concepts within available language. Hard but useful concepts can get mapped to idioms. Modes of categorization will be influenced by these factors, which in turn influences many processes.
thesz 11mo ago
What is a word in one language is a collocated words in another, possibly context-dependent. We can look no further than English: "man can do something," "man can do not do something" (i.e., can do but does not have to), then pretty straightforward "man can not do something" and, all of sudden, to express that man cannot decline some obligation, we say "man can not help but do." It is not translation per se, but shows that some parts of language were evolved to tiptoe around non-customary things, in this case, double negation. And double negation is very easy in some other languages.
mcswell 11mo ago
"'Language affects thoughts' (as in German is good for Philosophy and English for getting things done). If both these thoughts are true..." Well, the second one isn't true (I omitted the first one in this reply). It is simply not the case that German is good for philosophy and English for getting things done, and similarly for most other such claims (French is better for talking about love, Italian is better for operas, etc. etc.).

miltonlost 11mo ago

Reading any poem that makes use of extensive wordplay within a language shows why there will always be some untranslatable aspect. You can't create all the exact shades of a single pun if all those shades aren't in a different language. Go translate an ee cummings poem and make sure to retain all its meanings.

mcswell 11mo ago
Douglas Hofstedter's book The Ton beau de Marot.

KolenCh 11mo ago

Big claim but not much substance. They should try to really understand linear algebra first, and also linguistics a bit. Semantic domain (from linguistics) is a better way to describe it, where using sets (from math) might better convey what they want to say.

znkr 11mo ago
The whole article starts with the implicit assumption that all bases are orthonormal and then throws linear algebra out of the window completely.

deleted 11mo ago

[deleted]

tgv 11mo ago

Another physicist who thinks he can solve problems in a domain he knows nothing about with linear approximations. There's an xkcd devoted to this problem, even using computational linguistics as an example, IIRC.

mcswell 11mo ago

The word 'word' is polysemous, and also vague. Polysemous: it can refer to written words or spoken words (or signed words, in sign languages). Vague: does it mean all the inflected forms of a word, or just the stem without inflection? Example: is (are?) 'walk', 'walked', 'walks' and 'walking' four words, or one? What about "stand/stood"? (And languages where the bare root or stem can't appear by itself, like verbs in Spanish.) Derived words, like 'push' and 'pushy'. Do compound nouns count as a word, or do only the parts count? Example: 'doghouse' (or 'dog house'). What about idioms? Example: 'to crane his/her/my/your/our neck(s)'. What about different pronunciations? Is 'roof' pronounced to rhyme with 'aloof' the same word as 'roof' pronounced with the vowel of 'put'? And different spellings but same pronunciation: 'bear' vs. 'bare'. What about words with different grammatical categories, like 'push' as a noun ("I gave her a push") or a verb ("I pushed her"). Or the same word with virtually unrelated meanings, "I pushed her on the swing" vs. "I pushed my ideas."

sevensor 11mo ago

I like this as an analogy but not as an explanation. In fact, if you’re unfamiliar with linear algebra, this might be a nice way to think about projection onto a different set of basis vectors. But even the best human translators can be deeply at odds over what translation is appropriate for a term in its context. There’s never a right translation, let alone a uniquely right translation.

evolextra 11mo ago

[dead]

egl2020 11mo ago

From the title, I thought he was going to explain "eigenvalue".

fnord77 11mo ago

I cannot take seriously an article that presents a 3x3 matrix being multiplied by a 2-vector as an example of linear algebra. Gibberish.

decimalenough 11mo ago

上京 jōkyō is less a regular word and more a form of written shorthand. It would not be used in speech, and even in writing it's ambiguous: not only does 京 kyō simply mean "capital", but there's a district of Kyoto that's also 上京, only read Kamigyō. The closest English equivalent is abbreviations like "PC". They're perfectly usable in context, but if you see one standing alone it's not clear if it's personal computer, politically correct, Peace Corps, etc etc.

throwaway81523 11mo ago

> a gentle, poignant sadness or pathos felt in response to the transient nature of all things, a deep awareness of their impermanence that evokes a subtle, bittersweet sorrow and a profound, quiet empathy for their passing. https://en.wikipedia.org/wiki/Saudade https://en.wikipedia.org/wiki/Saudade ?

deleted 11mo ago

[deleted]