3 ms·

AI and the Limits of Language

34 points by sebg 4y ago

a_square_peg 4y ago
Considering that "the tacit convention on which the understanding of everyday language depends are enormously complicated," (Ludwig Wittgenstein) a system only trained on language, a small subset of how we acquire knowledge, will probably never attain the same level of intelligence as we do.
rvense 4y ago
One of the most interesting concepts in the philosophy of natural language is "the cooperative principle"[0] as described by Paul Grice. Basically, to understand what other people are saying requires us to cooperate with them, that is in some ways to put ourselves in their stead and imagine _why_ they're saying what they are and the (shared) situation we find ourselves in, which of course involves not just a knowledge of syntax and lexicon, but also culture and other forms of background knowledge. This is obviously the best method of disambiguation: "can you pass the salt" is ambiguous from the point-of-view of lexical semantics, but a sentence is always said in a context - e.g. either at a dinner table, or by hikers on the edge of a salt flat - and here sentences are only very rarely ambiguous. Many schools of 20th century linguistics were obsessed with ambiguity in language, but if you start with an understanding of language as something that humans do - rather than a language being a set of sentences that can be generated by some grammar, N Chomsky 1958 - ambiguity is actually rather peripheral. The cooperative principle is also relevant to AI. We all know the Eliza effect, and, to me, nothing explains it better than Grice. We cannot help but cooperate with text, because that is how you make text work, so even when we know that it's just a robot reversing our sentences, or AI generated poetry (some of which actually reads remarkably well), we look for whatever meaning might be there, under the instinctive assumption that the author is trying to tell us something. [0] https://en.wikipedia.org/wiki/Cooperative_principle https://en.wikipedia.org/wiki/Cooperative_principle
- deleted 4y ago
  [deleted]
cs702 4y ago
This makes sense to me: > All representational schemas involve a compression of information about something, but what gets left in and left out in the compression varies. The representational schema of language struggles with more concrete information, such as describing irregular shapes, the motion of objects, the functioning of a complex mechanism or the nuanced brushwork of a painting — much less the finicky, context-specific movements needed for surfing a wave. But there are nonlinguistic representational schemes which can express this information in an accessible way: iconic knowledge, which involves things like images, recordings, graphs and maps; and the distributed knowledge found in trained neural networks — what we often call know-how and muscle memory. Each scheme expresses some information easily even while finding other information hard — or even impossible — to represent. That said, we haven't reached the limits of language models yet. No one knows for certain how language models that are 10x or even 100x larger than current state-of-the-art ones will perform at modeling language (e.g., as quantified by perplexity) and at harder cognitive tasks.[a] -- [a] https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis
- learndeeply 4y ago
  > No one knows for certain how language models that are 10x or even 100x larger than current state-of-the-art ones will perform Read the Deepmind Chinchilla paper, it answers this
  - cs702 4y ago
    IIRC, no, it doesn't. It redefines language-model scaling as a function of both training dataset size as well as number of parameters, as opposed to only the latter. Also, all predictions extrapolated from past experimental results are hypothetical, subject to future experimental verification. We don't yet have a body of theory good enough to trust those extrapolations.
  - kordlessagain 4y ago
    Linear improvements with logarithm scaling of parameters would make sense, maybe.
learndeeply 4y ago
Frustratingly it ignores ViT and other image/video transformer models.
rafaelero 4y ago
TLDR: If turtles could speak, would we be able to understand how a turtle think?
- webnrrd2k 4y ago
  Well... we know how they draw [1]. Seriously though, so much of their cognitive hardware is similar to humans that, underneath it all, I'd guess that the way they actually think is very similar to human, although with very poor writing abilities. [1] https://en.wikipedia.org/wiki/Turtle_graphics https://en.wikipedia.org/wiki/Turtle_graphics
AbrahamParangi 4y ago
I am sympathetic to the argument that much of natural intelligence is non-linguistic but it seems to me that people will be saying “this AI stuff can do some pretty cool tricks but it’s not real intelligence” while matrix multiplication surpasses human effort in the fields of art, poetry, rhetoric, literature. At some point “If you can't tell the difference, does it matter?”
- deleted 4y ago
  [deleted]
- smitty1e 4y ago
  > matrix multiplication surpasses human effort in the fields of art, poetry, rhetoric, literature. I sincerely cannot follow the comparison.
xor99 4y ago
This article overly focusses on language derived synthesis. Obv because prompt engineering is a topic atm. H/w style transfer, image to video, etc is not necessarily derived from linguistic inputs if it's unlabelled. We could also hook up a brain to GPU network. Then having direct brain neural synthesis "beyond" language. I agree with the author that the linguistic focus of 20th century philosophy was shown to be fallacious though. Just look at the later Wittgenstein versus the Tractatus. Early Wittgenstein and its many pitfalls should be required reading @ Google (or maybe it is and the engineer in question "rebelled"). Language megalomania is not a good point of evidence for proving the existence of sentient machines.

34 points by sebg 4y ago

Considering that "the tacit convention on which the understanding of everyday language depends are enormously complicated," (Ludwig Wittgenstein) a system only trained on language, a small subset of how we acquire knowledge, will probably never attain the same level of intelligence as we do.

rvense 4y ago

One of the most interesting concepts in the philosophy of natural language is "the cooperative principle"[0] as described by Paul Grice. Basically, to understand what other people are saying requires us to cooperate with them, that is in some ways to put ourselves in their stead and imagine _why_ they're saying what they are and the (shared) situation we find ourselves in, which of course involves not just a knowledge of syntax and lexicon, but also culture and other forms of background knowledge. This is obviously the best method of disambiguation: "can you pass the salt" is ambiguous from the point-of-view of lexical semantics, but a sentence is always said in a context - e.g. either at a dinner table, or by hikers on the edge of a salt flat - and here sentences are only very rarely ambiguous. Many schools of 20th century linguistics were obsessed with ambiguity in language, but if you start with an understanding of language as something that humans do - rather than a language being a set of sentences that can be generated by some grammar, N Chomsky 1958 - ambiguity is actually rather peripheral. The cooperative principle is also relevant to AI. We all know the Eliza effect, and, to me, nothing explains it better than Grice. We cannot help but cooperate with text, because that is how you make text work, so even when we know that it's just a robot reversing our sentences, or AI generated poetry (some of which actually reads remarkably well), we look for whatever meaning might be there, under the instinctive assumption that the author is trying to tell us something. [0] https://en.wikipedia.org/wiki/Cooperative_principle https://en.wikipedia.org/wiki/Cooperative_principle

deleted 4y ago
[deleted]

cs702 4y ago

This makes sense to me: > All representational schemas involve a compression of information about something, but what gets left in and left out in the compression varies. The representational schema of language struggles with more concrete information, such as describing irregular shapes, the motion of objects, the functioning of a complex mechanism or the nuanced brushwork of a painting — much less the finicky, context-specific movements needed for surfing a wave. But there are nonlinguistic representational schemes which can express this information in an accessible way: iconic knowledge, which involves things like images, recordings, graphs and maps; and the distributed knowledge found in trained neural networks — what we often call know-how and muscle memory. Each scheme expresses some information easily even while finding other information hard — or even impossible — to represent. That said, we haven't reached the limits of language models yet. No one knows for certain how language models that are 10x or even 100x larger than current state-of-the-art ones will perform at modeling language (e.g., as quantified by perplexity) and at harder cognitive tasks.[a] -- [a] https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis https://www.gwern.net/Scaling-hypothesis#scaling-hypothesis

learndeeply 4y ago
> No one knows for certain how language models that are 10x or even 100x larger than current state-of-the-art ones will perform Read the Deepmind Chinchilla paper, it answers this
- cs702 4y ago
  IIRC, no, it doesn't. It redefines language-model scaling as a function of both training dataset size as well as number of parameters, as opposed to only the latter. Also, all predictions extrapolated from past experimental results are hypothetical, subject to future experimental verification. We don't yet have a body of theory good enough to trust those extrapolations.
- kordlessagain 4y ago
  Linear improvements with logarithm scaling of parameters would make sense, maybe.

learndeeply 4y ago

Frustratingly it ignores ViT and other image/video transformer models.

rafaelero 4y ago

TLDR: If turtles could speak, would we be able to understand how a turtle think?

webnrrd2k 4y ago
Well... we know how they draw [1]. Seriously though, so much of their cognitive hardware is similar to humans that, underneath it all, I'd guess that the way they actually think is very similar to human, although with very poor writing abilities. [1] https://en.wikipedia.org/wiki/Turtle_graphics https://en.wikipedia.org/wiki/Turtle_graphics

AbrahamParangi 4y ago

I am sympathetic to the argument that much of natural intelligence is non-linguistic but it seems to me that people will be saying “this AI stuff can do some pretty cool tricks but it’s not real intelligence” while matrix multiplication surpasses human effort in the fields of art, poetry, rhetoric, literature. At some point “If you can't tell the difference, does it matter?”

deleted 4y ago
[deleted]
smitty1e 4y ago
> matrix multiplication surpasses human effort in the fields of art, poetry, rhetoric, literature. I sincerely cannot follow the comparison.

xor99 4y ago

This article overly focusses on language derived synthesis. Obv because prompt engineering is a topic atm. H/w style transfer, image to video, etc is not necessarily derived from linguistic inputs if it's unlabelled. We could also hook up a brain to GPU network. Then having direct brain neural synthesis "beyond" language. I agree with the author that the linguistic focus of 20th century philosophy was shown to be fallacious though. Just look at the later Wittgenstein versus the Tractatus. Early Wittgenstein and its many pitfalls should be required reading @ Google (or maybe it is and the engineer in question "rebelled"). Language megalomania is not a good point of evidence for proving the existence of sentient machines.