5 ms·

Language models and linguistic theories beyond words

70 points by Anon84 3y ago

famouswaffles 3y ago
Paraphrasing and summarizing parts of this article, https://hedgehogreview.com/issues/markets-and-the-good/articles/language-machinery https://hedgehogreview.com/issues/markets-and-the-good/artic... Some ~72 years ago in 1951, Claude Shannon released his Paper, "Prediction and Entropy of Printed English", an extremely fascinating read now. It begins with a game. Claude pulls a book down from the shelf, concealing the title in the process. After selecting a passage at random, he challenges his wife, Mary to guess its contents letter by letter. The space between words will count as a twenty-seventh symbol in the set. If Mary fails to guess a letter correctly, Claude promises to supply the right one so that the game can continue. In some cases, a corrected mistake allows her to fill in the remainder of the word; elsewhere a few letters unlock a phrase. All in all, she guesses 89 of 129 possible letters correctly—69 percent accuracy. Discovery 1: It illustrated, in the first place, that a proficient speaker of a language possesses an “enormous” but implicit knowledge of the statistics of that language. Shannon would have us see that we make similar calculations regularly in everyday life—such as when we “fill in missing or incorrect letters in proof-reading” or “complete an unfinished phrase in conversation.” As we speak, read, and write, we are regularly engaged in predication games. Discovery 2: Perhaps the most striking of all, Claude argues that that a complete text and the subsequent “reduced text” consisting of letters and dashes “actually…contain the same information” under certain conditions. How?? (Surely, the first line contains more information!).The answer depends on the peculiar notion about information that Shannon had hatched in his 1948 paper “A Mathematical Theory of Communication” (hereafter “MTC”), the founding charter of information theory. He argues that transfer of a message's components, rather than its "meaning", should be the focus for the engineer. You ought to be agnostic about a message’s “meaning” (or “semantic aspects”). The message could be nonsense, and the engineer’s problem—to transfer its components faithfully—would be the same. a highly predictable message contains less information than an unpredictable one. More information is at stake in (“villapleach, vollapluck”) than in (“Twinkle, twinkle”). Does "Flinkle, fli- - - -" really contain less information than "Flinkle, flinkle" ? Shannon concludes then that the complete text and the "reduced text" are equivalent in information content under certain conditions because predictable letters become redundant in information transfer. Fueled by this, Claude then proposes an illuminating thought experiment: Imagine that Mary has a truly identical twin (call her “Martha”). If we supply Martha with the “reduced text,” she should be able to recreate the entirety of Chandler’s passage, since she possesses the same statistical knowledge of English as Mary. Martha would make Mary’s guesses in reverse. Of course, Shannon admitted, there are no “mathematically identical twins” to be found, but and here's the reveal, “we do have mathematically identical computing machines.” Those machines could be given a model for making informed predictions about letters, words, maybe larger phrases and messages. In one fell swoop, Shannon had demonstrated that language use has a statistical side, that languages are, in turn, predictable, and that computers too can play the prediction game.
- hyeonwho22 3y ago
  There was a fun recent variant on this game using LLMs, asking GPT3 (3.5?) to encode text in a way that it will be able to decode the meaning. Some of the encodings are insane: https://www.piratewires.com/p/compression-prompts-gpt-hidden-dialects https://www.piratewires.com/p/compression-prompts-gpt-hidden...
  - jonahrd 3y ago
    This is super interesting. Are there more examples I can see? The one in the article is a famous song which makes me wonder if it's really "decompressing" the data, or just being hunted towards a very common popular pattern of tokens.
    - famouswaffles 3y ago
      https://aps.arxiv.org/abs/2309.10668 https://aps.arxiv.org/abs/2309.10668
- esafak 3y ago
  If you want to go there, you could say that natural languages are error-correcting codes -- somewhat robust to corruption (typos). https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code
- thomastjeffery 3y ago
  The information wasn't removed: it was moved. Whenever a human remembers something specific, they actually don't. Instead, they remember a few small details, and the patterns that organize them. Then, the brain hallucinates more details that fit the overall pattern of the story. This phenomenon is called "Reconstructive Memory", and is one reason why eyewitness testimony is unreliable. An LLM is similar to memory: you feed its neurons a bunch of data that has meaning encoded into it, and it becomes a model for any patterns present in that data. When an LLM generates a continuation, it is continuing the patterns that it modeled; including the data it was trained on, and whatever prompt it was given. Natural language solved! Right? --- Not so fast! The human mind performs much more than memory reconstruction. How else would we encode meaning into the semantics of language, and write that data into text? There is more to this story. There is more to... story. Remember when I said the information is "moved"? Where did it go? More importantly, how can we use that data? --- Let's consider a human named Dave, reading a book about boats. What is Dave using to read it? The short version: empathy. Story is held together with semantics. Semantics are already-known patterns that define logical relationships between words. When Dave reads the statement, "Sue got in the boat", he interprets that information into meaning. Sue is a person, probably a woman. She entered the top of a vessel that was floating on water. But Dave was wrong! Sue is a cat, and the boat was lying on dry beach. Here's the interesting part: Dave was totally correct until I declared otherwise. His interpretation matched his internal worldview: all of the ambiguity present in what he read was resolved by his assumptions. Making false assumptions is a completely valid result of the process that we call "reading". It's a feature. After Dave overheard me talking to you just now, he learned the truth of the story, and immediately resolved his mistake. In an instant, Dave took the semantic information he had just read, and he reread it with a completely different worldview. But where did he read it from? His worldview. You see, after reading the book, its meaning was added neatly into his worldview. Because of this, he was prepared to interpret what I was telling you: about Sue being a cat and so on. Dave performed the same "reading" process on my new story, and he used the statement he read in the book to do just that. --- Worldview is context. Context is the tool that resolves ambiguity. We use this tool to interpret story, particularly when story is ambiguous. So what is context made of? Story. It's recursive. Everything that we read is added to our internal story. One giant convoluted mess of logic is carved into our neurons. We fold those logical constructs together into a small set of coherent ideas. But where is the base case? What's the smallest part of a story? What are the axioms? This is the part I struggled with the most: there isn't one. Somehow, we manage to perform these recursive algorithms from the middle. We read the story down the ladder of abstraction, as close to its roots as we can find; but we can only read the story as far as we have read it already. We can navigate the logical structure of ideas without ever proving that logic to be sound. We can even navigate logic that is outright false! Constraining this behavior to proven logic has to be intentional. That's why we have a word for it: mathematics. Math is the special story: it's rooted in axioms, and built up exclusively using theorems. Theorems are an optimization: they let us skip from one page of a story to another. They let us fold and compress logical structure into a something more practical. --- LLMs do not use logic at all. The logic of invalidation is missing. When story categorizes an idea into the realm of fiction, the LLM simply can't react accordingly. The LLM has no internal story: only memory.
foobarqux 3y ago
https://www.youtube.com/watch?v=DQuiso493ro&t=3309s https://www.youtube.com/watch?v=DQuiso493ro&t=3309s "Have you seen the paper that's called Modern Language Models Refute Chomsky's Approach to Language? The one by Steven Piantadosi. If so, what do you make of it? Well, I mean, unlike a lot of people who write about this, he does know about large language models, but the article makes absolutely no sense. It has a minor problem and a major problem. The minor problem is that it's beyond absurdity to think that you can learn anything about a two bunch of supercomputers scanning 50 terabytes of data, looking for statistical regularities, and stringing them together. To think that from that you can learn anything about an infant is so beyond absurdity that it's not worth talking about. That's the minor problem. The major problem is that, in principle, you can learn nothing. In principle, make it 100 terabytes, use 20 supercomputers. Just bring out the, in principle, impossibility. Even worse, for more clearly, for a very simple reason, these systems work just as well for impossible languages that children can't acquire as they do for possible languages that children do acquire. It's kind of as if a physicist came along and said, I've got a great new theory. It accounts for a lot of things that happen, a lot of things that can't possibly happen, and I can't make any distinction among them. We would just laugh. I mean, any explanation of anything says, here's why things happen, here's why other things don't happen. You can't make that distinction. You're doing nothing. That's irremediable. It's built into the nature of the systems. The more sophisticated they become at dealing with actual language, the more it's demonstrated that they're telling us nothing, in principle, about language, about learning, or about cognition. So, there's a minor problem with this paper, and a major one. The minor problem is the simple absurdity of thinking that looking at anything of the scale could tell you about what's happening. The major problem is you can't do it in principle. Yes, I heard you describe this as if a physicist came along and said, I have a theory, and it's two words. Anything goes. Well, that's basically the large language models. You give it a system that's designed in order to violate the principles of language, it'll do just as well. Maybe better, because it can use simple algorithms that aren't used by natural language. So, it's basically, like I said, or like I say, suppose some guy comes along with an improvement on the periodic table, says I got a theory that includes all the possible elements, even those that haven't been discovered, and all kinds of impossible ones, and I can't tell any difference. It's not an improvement on the periodic table. That's telling you nothing about chemistry. That's built into the design of the system. It's not remediable."
deleted 3y ago
[deleted]
tkgally 3y ago
The article focuses on linguistic theories that regard natural language in a somewhat narrow sense: language as a cognitive phenomenon, language as a symbolic system, language as something that can be modeled and imitated by a computer. While those aspects are important, language is also a social phenomenon, something used for interaction among people for many different purposes. Both traditional approaches to natural language—the cognitive and the social—face a challenge, because language is no longer used only for communication between human beings. What does it mean for linguistics now that human beings are increasingly using natural language to interact with computers? I don’t know the answer, but it’s a question that deserves attention.
- patcon 3y ago
  > What does it mean for linguistics now that human beings are increasingly using natural language to interact with computers? Yeah, it's going to be strange. So much of our etiquette and norms and basic cultural assumption have to do with this idea: This communication protocol of "language" is for humans. If something else is "on the wire" with us, it must be like us, and worthy of our attention or cycles. But this is increasingly going to be very untrue. Culture is about to be weird.
  - tkgally 3y ago
    Curious about what ChatGPT might have to say on this issue, I asked it: https://www.gally.net/temp/20231116languageandai/index.html https://www.gally.net/temp/20231116languageandai/index.html
    - pharmakom 3y ago
      Why do people keep asking models about models?
      - tkgally 3y ago
        Well, in my case, I was struggling to come up with a coherent understanding for myself about how views of language might need to change when interaction with LLMs is taken into account. ChatGPT’s response helped me think about the issue more clearly. Others might not find the same clarity.
gizajob 3y ago
Get a neural network, same as you'd use at the bottom of an LLM. Don't feed it a corpus of text! Ensure it's data is the opposite of a large language model, i.e. zero language. Let it passively listen to the same sounds and speech as a baby hears in its first five years, over five years of operation. Observe it do absolutely nothing of any use. Get back to me about any useful comparisons with linguistics or philosophy of language.
- famouswaffles 3y ago
  Text is not the important thing here at all. audio language models exist, they are relatively trivial to engineer and they will model audio just as well vs they'll model text or any arbitrary sequence to sequence data. https://google-research.github.io/seanet/audiolm/examples/ https://google-research.github.io/seanet/audiolm/examples/ Linguistics is useful fiction. It doesn't accurately describe the complexity of language anymore than Newton's laws accurately describe gravity. You'd think people would understand that now with symbolic approaches failing this sort of stuff after decades of trying but i guess not. We don't know how we work. We really really don't. The obsession with how we think we work over methods that are actually working is really something to behold.
  - gizajob 3y ago
    https://tmms.co.uk/wp-content/uploads/2023/01/imgpsh_fullsize_anim-5.png https://tmms.co.uk/wp-content/uploads/2023/01/imgpsh_fullsiz...
  - patcon 3y ago
    > We don't know how we work. We really really don't. Amen.
- stevenhuang 3y ago
  You are aware the same transformer architecture that underpins LLMs has shown great success in multimodal applications? Such as successfully modeling sound and music? https://bytedance.github.io/SALMONN/ https://bytedance.github.io/SALMONN/
- 3y ago

70 points by Anon84 3y ago

Paraphrasing and summarizing parts of this article, https://hedgehogreview.com/issues/markets-and-the-good/articles/language-machinery https://hedgehogreview.com/issues/markets-and-the-good/artic... Some ~72 years ago in 1951, Claude Shannon released his Paper, "Prediction and Entropy of Printed English", an extremely fascinating read now. It begins with a game. Claude pulls a book down from the shelf, concealing the title in the process. After selecting a passage at random, he challenges his wife, Mary to guess its contents letter by letter. The space between words will count as a twenty-seventh symbol in the set. If Mary fails to guess a letter correctly, Claude promises to supply the right one so that the game can continue. In some cases, a corrected mistake allows her to fill in the remainder of the word; elsewhere a few letters unlock a phrase. All in all, she guesses 89 of 129 possible letters correctly—69 percent accuracy. Discovery 1: It illustrated, in the first place, that a proficient speaker of a language possesses an “enormous” but implicit knowledge of the statistics of that language. Shannon would have us see that we make similar calculations regularly in everyday life—such as when we “fill in missing or incorrect letters in proof-reading” or “complete an unfinished phrase in conversation.” As we speak, read, and write, we are regularly engaged in predication games. Discovery 2: Perhaps the most striking of all, Claude argues that that a complete text and the subsequent “reduced text” consisting of letters and dashes “actually…contain the same information” under certain conditions. How?? (Surely, the first line contains more information!).The answer depends on the peculiar notion about information that Shannon had hatched in his 1948 paper “A Mathematical Theory of Communication” (hereafter “MTC”), the founding charter of information theory. He argues that transfer of a message's components, rather than its "meaning", should be the focus for the engineer. You ought to be agnostic about a message’s “meaning” (or “semantic aspects”). The message could be nonsense, and the engineer’s problem—to transfer its components faithfully—would be the same. a highly predictable message contains less information than an unpredictable one. More information is at stake in (“villapleach, vollapluck”) than in (“Twinkle, twinkle”). Does "Flinkle, fli- - - -" really contain less information than "Flinkle, flinkle" ? Shannon concludes then that the complete text and the "reduced text" are equivalent in information content under certain conditions because predictable letters become redundant in information transfer. Fueled by this, Claude then proposes an illuminating thought experiment: Imagine that Mary has a truly identical twin (call her “Martha”). If we supply Martha with the “reduced text,” she should be able to recreate the entirety of Chandler’s passage, since she possesses the same statistical knowledge of English as Mary. Martha would make Mary’s guesses in reverse. Of course, Shannon admitted, there are no “mathematically identical twins” to be found, but and here's the reveal, “we do have mathematically identical computing machines.” Those machines could be given a model for making informed predictions about letters, words, maybe larger phrases and messages. In one fell swoop, Shannon had demonstrated that language use has a statistical side, that languages are, in turn, predictable, and that computers too can play the prediction game.

hyeonwho22 3y ago
There was a fun recent variant on this game using LLMs, asking GPT3 (3.5?) to encode text in a way that it will be able to decode the meaning. Some of the encodings are insane: https://www.piratewires.com/p/compression-prompts-gpt-hidden-dialects https://www.piratewires.com/p/compression-prompts-gpt-hidden...
- jonahrd 3y ago
  This is super interesting. Are there more examples I can see? The one in the article is a famous song which makes me wonder if it's really "decompressing" the data, or just being hunted towards a very common popular pattern of tokens.
  - famouswaffles 3y ago
    https://aps.arxiv.org/abs/2309.10668 https://aps.arxiv.org/abs/2309.10668
esafak 3y ago
If you want to go there, you could say that natural languages are error-correcting codes -- somewhat robust to corruption (typos). https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code
thomastjeffery 3y ago
The information wasn't removed: it was moved. Whenever a human remembers something specific, they actually don't. Instead, they remember a few small details, and the patterns that organize them. Then, the brain hallucinates more details that fit the overall pattern of the story. This phenomenon is called "Reconstructive Memory", and is one reason why eyewitness testimony is unreliable. An LLM is similar to memory: you feed its neurons a bunch of data that has meaning encoded into it, and it becomes a model for any patterns present in that data. When an LLM generates a continuation, it is continuing the patterns that it modeled; including the data it was trained on, and whatever prompt it was given. Natural language solved! Right? --- Not so fast! The human mind performs much more than memory reconstruction. How else would we encode meaning into the semantics of language, and write that data into text? There is more to this story. There is more to... story. Remember when I said the information is "moved"? Where did it go? More importantly, how can we use that data? --- Let's consider a human named Dave, reading a book about boats. What is Dave using to read it? The short version: empathy. Story is held together with semantics. Semantics are already-known patterns that define logical relationships between words. When Dave reads the statement, "Sue got in the boat", he interprets that information into meaning. Sue is a person, probably a woman. She entered the top of a vessel that was floating on water. But Dave was wrong! Sue is a cat, and the boat was lying on dry beach. Here's the interesting part: Dave was totally correct until I declared otherwise. His interpretation matched his internal worldview: all of the ambiguity present in what he read was resolved by his assumptions. Making false assumptions is a completely valid result of the process that we call "reading". It's a feature. After Dave overheard me talking to you just now, he learned the truth of the story, and immediately resolved his mistake. In an instant, Dave took the semantic information he had just read, and he reread it with a completely different worldview. But where did he read it from? His worldview. You see, after reading the book, its meaning was added neatly into his worldview. Because of this, he was prepared to interpret what I was telling you: about Sue being a cat and so on. Dave performed the same "reading" process on my new story, and he used the statement he read in the book to do just that. --- Worldview is context. Context is the tool that resolves ambiguity. We use this tool to interpret story, particularly when story is ambiguous. So what is context made of? Story. It's recursive. Everything that we read is added to our internal story. One giant convoluted mess of logic is carved into our neurons. We fold those logical constructs together into a small set of coherent ideas. But where is the base case? What's the smallest part of a story? What are the axioms? This is the part I struggled with the most: there isn't one. Somehow, we manage to perform these recursive algorithms from the middle. We read the story down the ladder of abstraction, as close to its roots as we can find; but we can only read the story as far as we have read it already. We can navigate the logical structure of ideas without ever proving that logic to be sound. We can even navigate logic that is outright false! Constraining this behavior to proven logic has to be intentional. That's why we have a word for it: mathematics. Math is the special story: it's rooted in axioms, and built up exclusively using theorems. Theorems are an optimization: they let us skip from one page of a story to another. They let us fold and compress logical structure into a something more practical. --- LLMs do not use logic at all. The logic of invalidation is missing. When story categorizes an idea into the realm of fiction, the LLM simply can't react accordingly. The LLM has no internal story: only memory.

foobarqux 3y ago

https://www.youtube.com/watch?v=DQuiso493ro&t=3309s https://www.youtube.com/watch?v=DQuiso493ro&t=3309s "Have you seen the paper that's called Modern Language Models Refute Chomsky's Approach to Language? The one by Steven Piantadosi. If so, what do you make of it? Well, I mean, unlike a lot of people who write about this, he does know about large language models, but the article makes absolutely no sense. It has a minor problem and a major problem. The minor problem is that it's beyond absurdity to think that you can learn anything about a two bunch of supercomputers scanning 50 terabytes of data, looking for statistical regularities, and stringing them together. To think that from that you can learn anything about an infant is so beyond absurdity that it's not worth talking about. That's the minor problem. The major problem is that, in principle, you can learn nothing. In principle, make it 100 terabytes, use 20 supercomputers. Just bring out the, in principle, impossibility. Even worse, for more clearly, for a very simple reason, these systems work just as well for impossible languages that children can't acquire as they do for possible languages that children do acquire. It's kind of as if a physicist came along and said, I've got a great new theory. It accounts for a lot of things that happen, a lot of things that can't possibly happen, and I can't make any distinction among them. We would just laugh. I mean, any explanation of anything says, here's why things happen, here's why other things don't happen. You can't make that distinction. You're doing nothing. That's irremediable. It's built into the nature of the systems. The more sophisticated they become at dealing with actual language, the more it's demonstrated that they're telling us nothing, in principle, about language, about learning, or about cognition. So, there's a minor problem with this paper, and a major one. The minor problem is the simple absurdity of thinking that looking at anything of the scale could tell you about what's happening. The major problem is you can't do it in principle. Yes, I heard you describe this as if a physicist came along and said, I have a theory, and it's two words. Anything goes. Well, that's basically the large language models. You give it a system that's designed in order to violate the principles of language, it'll do just as well. Maybe better, because it can use simple algorithms that aren't used by natural language. So, it's basically, like I said, or like I say, suppose some guy comes along with an improvement on the periodic table, says I got a theory that includes all the possible elements, even those that haven't been discovered, and all kinds of impossible ones, and I can't tell any difference. It's not an improvement on the periodic table. That's telling you nothing about chemistry. That's built into the design of the system. It's not remediable."

deleted 3y ago

[deleted]

tkgally 3y ago

The article focuses on linguistic theories that regard natural language in a somewhat narrow sense: language as a cognitive phenomenon, language as a symbolic system, language as something that can be modeled and imitated by a computer. While those aspects are important, language is also a social phenomenon, something used for interaction among people for many different purposes. Both traditional approaches to natural language—the cognitive and the social—face a challenge, because language is no longer used only for communication between human beings. What does it mean for linguistics now that human beings are increasingly using natural language to interact with computers? I don’t know the answer, but it’s a question that deserves attention.

patcon 3y ago
> What does it mean for linguistics now that human beings are increasingly using natural language to interact with computers? Yeah, it's going to be strange. So much of our etiquette and norms and basic cultural assumption have to do with this idea: This communication protocol of "language" is for humans. If something else is "on the wire" with us, it must be like us, and worthy of our attention or cycles. But this is increasingly going to be very untrue. Culture is about to be weird.
- tkgally 3y ago
  Curious about what ChatGPT might have to say on this issue, I asked it: https://www.gally.net/temp/20231116languageandai/index.html https://www.gally.net/temp/20231116languageandai/index.html
  - pharmakom 3y ago
    Why do people keep asking models about models?
    - tkgally 3y ago
      Well, in my case, I was struggling to come up with a coherent understanding for myself about how views of language might need to change when interaction with LLMs is taken into account. ChatGPT’s response helped me think about the issue more clearly. Others might not find the same clarity.

gizajob 3y ago

Get a neural network, same as you'd use at the bottom of an LLM. Don't feed it a corpus of text! Ensure it's data is the opposite of a large language model, i.e. zero language. Let it passively listen to the same sounds and speech as a baby hears in its first five years, over five years of operation. Observe it do absolutely nothing of any use. Get back to me about any useful comparisons with linguistics or philosophy of language.

famouswaffles 3y ago
Text is not the important thing here at all. audio language models exist, they are relatively trivial to engineer and they will model audio just as well vs they'll model text or any arbitrary sequence to sequence data. https://google-research.github.io/seanet/audiolm/examples/ https://google-research.github.io/seanet/audiolm/examples/ Linguistics is useful fiction. It doesn't accurately describe the complexity of language anymore than Newton's laws accurately describe gravity. You'd think people would understand that now with symbolic approaches failing this sort of stuff after decades of trying but i guess not. We don't know how we work. We really really don't. The obsession with how we think we work over methods that are actually working is really something to behold.
- gizajob 3y ago
  https://tmms.co.uk/wp-content/uploads/2023/01/imgpsh_fullsize_anim-5.png https://tmms.co.uk/wp-content/uploads/2023/01/imgpsh_fullsiz...
- patcon 3y ago
  > We don't know how we work. We really really don't. Amen.
stevenhuang 3y ago
You are aware the same transformer architecture that underpins LLMs has shown great success in multimodal applications? Such as successfully modeling sound and music? https://bytedance.github.io/SALMONN/ https://bytedance.github.io/SALMONN/
3y ago