5 ms·

Let's say you don't speak french. Imagine I lock you in a room filled with french-language magazines and movies and books. Having nothing better to do, you wat

by dilap 4y ago

Let's say you don't speak french. Imagine I lock you in a room filled with french-language magazines and movies and books. Having nothing better to do, you watch and read them (as well as you can). Periodically, I have my confederate, a french man, enter the room and speak a french sentence with one word missing. He asks you to fill in the missing word. We keep you locked in this room until you've achieved some target benchmark of correctly saying the missing word, say 20% of the time. Are you then, just a "next-french-word-predicting" robot? No, obviously not. To have achieved the task and escaped the room, you will have had to have gained a significant understanding of French. It is the same with ChatGPT.

jliptzin 4y ago
So does my pocket calculator understand what I’m doing when I multiply restaurant bills by 0.15?
counters 4y ago
No, all you need to do is memorize enough vocabulary and have enough chances to get lucky on occasion. You can have better odds if you learn the conditional probability of that vocabulary given surrounding words. And this is all that GPT is doing.
- dilap 4y ago
  I don't think so at all. The number of possible sentences is far too great to ever hope to accomplish the task (at some threshold level of difficulty) simply by having a "conditional probability table". Nor would it ever be reasonable to store or generate such a table. To do it it, you need a form of compression: reasoning, intelligence, knowledge.
  - pwinnski 4y ago
    Right, for a human, presumably it would be easier to go ahead and learn the language. But that begs the question. We already know LLMs tokenize words, so they have definitely generated and stored such a table. The marvelous step forward from previous word-based solutions is that they store not just single words, not just pairs of words, but larger sequences of words, even up to short sentence length. It makes the table pretty large, as you suggest, but that's exactly we've been seeing, and each generation of GPT is that much larger than the one before.
ghostwriter 4y ago
> Imagine I lock you in a room If you lock a reasoning being in a room, then regardless of your further actions with the French language, you'll be dealing with a reasoning being. Your sequence "you will have had to have gained a significant understanding of French" is deprecated on the fact that "understanding" is attributed to the reasoning being capable of it, not to the fact that they were locked in the room and forced to repeat the words.
BeefWellington 4y ago
> To have achieved the task and escaped the room, you will have had to have gained a significant understanding of French. No, in this scenario the posited person has only gained a significant understanding of patterns of French language, not of the language itself. You may believe that difference to be meaningless but tangibly it is the difference between identifying the next shape in a sequence and understanding that there is some larger purpose at play (context, if nothing else). What actual understanding of the language they may have gained comes from second order effects -- seeing the language used in a movie for example is going to drastically influence and impart meaning that isn't present in the words alone. This is somewhat born out in how ChatGPT and other LLMs being hyped are often really aggressively wrong about things. They don't get the correct answer in many circumstances, even when it's obvious to a human. If it truly understood everything it could from what it has been trained on, those gaps would not occur.
- dilap 4y ago
  I guess my analogy didn't help because the same difference of perspective manifests itself there. To me it seems quite obvious you could never escape the french room without actually gaining an understanding of french; it's simply not possible to "identify the patterns" without understanding the patterns.
  - pwinnski 4y ago
    > it's simply not possible to "identify the patterns" without understanding the patterns I don't think that's true at all. To explain my thinking, let's assume that I am an American, and therefore according to popular understanding, completely incapable of learning any language other than English. I despair! But then I think no, wait, I happen to have a gift for numbers. Like, my mental arithmetic stunned teachers from a young age, and I currently hold the world record for reciting digits of pi from memory. So what I will do, I think, is assign a number to every word I read or hear and where possible, visualize where that chain of numbers exists among the decimal digits of pi. At some point the person would come in and utter some words I couldn't comprehend, but I would think, aha! That was almost 498153217845632754, except it was missing the second seven, which mapped to "contraire." And I'd do that at least 20% of the time, escaping the room with no understanding of French at all. If I could map between memorized numbers and words quickly enough, I'd even be able to converse in French given suitable trigger words without a shred of understanding.
guhcampos 4y ago
Your analogy is perfect exactly because it shows how GPT is not reasoning. In your analogy, the person never learned the meanings of the French words. Just how they usually go together.

by dilap 4y ago

jliptzin 4y ago

So does my pocket calculator understand what I’m doing when I multiply restaurant bills by 0.15?

counters 4y ago

No, all you need to do is memorize enough vocabulary and have enough chances to get lucky on occasion. You can have better odds if you learn the conditional probability of that vocabulary given surrounding words. And this is all that GPT is doing.

dilap 4y ago
I don't think so at all. The number of possible sentences is far too great to ever hope to accomplish the task (at some threshold level of difficulty) simply by having a "conditional probability table". Nor would it ever be reasonable to store or generate such a table. To do it it, you need a form of compression: reasoning, intelligence, knowledge.
- pwinnski 4y ago
  Right, for a human, presumably it would be easier to go ahead and learn the language. But that begs the question. We already know LLMs tokenize words, so they have definitely generated and stored such a table. The marvelous step forward from previous word-based solutions is that they store not just single words, not just pairs of words, but larger sequences of words, even up to short sentence length. It makes the table pretty large, as you suggest, but that's exactly we've been seeing, and each generation of GPT is that much larger than the one before.

ghostwriter 4y ago

> Imagine I lock you in a room If you lock a reasoning being in a room, then regardless of your further actions with the French language, you'll be dealing with a reasoning being. Your sequence "you will have had to have gained a significant understanding of French" is deprecated on the fact that "understanding" is attributed to the reasoning being capable of it, not to the fact that they were locked in the room and forced to repeat the words.

BeefWellington 4y ago

> To have achieved the task and escaped the room, you will have had to have gained a significant understanding of French. No, in this scenario the posited person has only gained a significant understanding of patterns of French language, not of the language itself. You may believe that difference to be meaningless but tangibly it is the difference between identifying the next shape in a sequence and understanding that there is some larger purpose at play (context, if nothing else). What actual understanding of the language they may have gained comes from second order effects -- seeing the language used in a movie for example is going to drastically influence and impart meaning that isn't present in the words alone. This is somewhat born out in how ChatGPT and other LLMs being hyped are often really aggressively wrong about things. They don't get the correct answer in many circumstances, even when it's obvious to a human. If it truly understood everything it could from what it has been trained on, those gaps would not occur.

dilap 4y ago
I guess my analogy didn't help because the same difference of perspective manifests itself there. To me it seems quite obvious you could never escape the french room without actually gaining an understanding of french; it's simply not possible to "identify the patterns" without understanding the patterns.
- pwinnski 4y ago
  > it's simply not possible to "identify the patterns" without understanding the patterns I don't think that's true at all. To explain my thinking, let's assume that I am an American, and therefore according to popular understanding, completely incapable of learning any language other than English. I despair! But then I think no, wait, I happen to have a gift for numbers. Like, my mental arithmetic stunned teachers from a young age, and I currently hold the world record for reciting digits of pi from memory. So what I will do, I think, is assign a number to every word I read or hear and where possible, visualize where that chain of numbers exists among the decimal digits of pi. At some point the person would come in and utter some words I couldn't comprehend, but I would think, aha! That was almost 498153217845632754, except it was missing the second seven, which mapped to "contraire." And I'd do that at least 20% of the time, escaping the room with no understanding of French at all. If I could map between memorized numbers and words quickly enough, I'd even be able to converse in French given suitable trigger words without a shred of understanding.

guhcampos 4y ago

Your analogy is perfect exactly because it shows how GPT is not reasoning. In your analogy, the person never learned the meanings of the French words. Just how they usually go together.