4 ms·

Obligatory mention: GPT-3 cannot learn to rhyme, it can only brute-force memorize rhyme-pairs, because the BPE encoding erases phonetic information from the enc

by gwern 5y ago

Obligatory mention: GPT-3 cannot learn to rhyme, it can only brute-force memorize rhyme-pairs, because the BPE encoding erases phonetic information from the encoding of words that a GPT-3 (and similar models like Gopher or GPT-J or Jurassic-1) is trained on. https://www.gwern.net/GPT-3#bpes https://www.gwern.net/GPT-3#bpes This is why that rap looks so weird: so high-quality in general, yet so totally lacking a critical element we expect from rap. GPT-3 has a pretty good idea how to write rap, but it just doesn't know what words sound like. It is blind to that. It can't see that difference between rap and poems. To it, rap is just a sort of poetry, and some words are chosen for reasons it can't understand, no matter how many rap lyrics or poems it looks at. The better the models get at writing, the more this difference throws us for a loop: "how can it be so good at ABC, and yet fail so totally at D?"

TaylorAlexander 5y ago
Interesting! So what I am hearing from this is that some day we will overcome this and have programs that can rap. And then companies with customer support bots will have a little check box if you want the bot to rap all responses to you.
- eru 5y ago
  Technically, we know exactly what we have to do to GPT-3 to give it a chance to learn how to rhyme: get rid of BPES and just use raw UTF-8 (or even just ASCII) as the encoding. Enlarge the model a bit, as necessary. At least that's what I am getting from Gwern's write-up. I might have misunderstood Gwern, or Gwern might be wrong, of course.
  - PeterisP 5y ago
    Replacing BPEs not with characters but with a sylabbary (is that a word? a vocabulary made of possible syllables) would be even more powerful, and you could also run preprocessing to convert text to phonetic transcription, to enforce a concept that spelling is irrelevant but the pronunciation matters.
    - eru 5y ago
      That would make the system more complicated by baking in extra assumptions. And eg it would probably perform worse at translating from English to French than a naive system. (Unless you preprocess your corpus extensively to figure out when you get English and when you get a snippet of French.) GPT-3 is surprisingly good at translating English to French. Another problem is that English has not just one text-to-phonetic transcription: different accents pronounce words differently. For just one example: > n most non-rhotic accents, if a word ending in written "r" is followed immediately by a word beginning with a vowel, the /r/ is pronounced, as in water ice. That phenomenon is referred to as "linking R". Many non-rhotic speakers also insert an epenthetic /r/ between vowels when the first vowel is one that can occur before syllable-final r (drawring for drawing). The so-called "intrusive R" has been stigmatized, but many speakers of Received Pronunciation (RP) now frequently "intrude" an epenthetic /r/ at word boundaries, especially if one or both vowels is schwa. For example, the idea of it becomes the idea-r-of it, Australia and New Zealand becomes Australia-r-and New Zealand, the formerly well-known India-r-Office and "Laura Norder" (Law and Order). The typical alternative used by RP speakers (and some rhotic speakers as well) is to insert a glottal stop wherever an intrusive R would otherwise have been placed. https://en.wikipedia.org/wiki/Rhoticity_in_English https://en.wikipedia.org/wiki/Rhoticity_in_English Btw, even without any explicit pronunciation data, I would expect the system to get a pretty good idea of how pronunciation works by essentially using the same technique human linguists use to reconstruct old pronunciations: They observe what kinds of mistakes people make. Eg when you see people mixing up "there" and "their" and "they're" in the corpus, that tells you that in modern English these three are pronounced almost the same. From spelling mistakes in words like carburetor you can figure out that unstressed vowels in English are pretty much all pronounced the same: as a schwa. https://en.wikipedia.org/wiki/Schwa https://en.wikipedia.org/wiki/Schwa You can also learn a lot from observing rhymes.
      - PeterisP 5y ago
        But you have to make those assumptions if you want to make a model for rhyming because large quantities of raw prose does not contain any information whatsoever about which words rhyme, how they are pronounced or where the accents lie. And of course that would be worse for other tasks, that's kind of the whole point - discarding irrelevant information is the key part that distinguishes learning from memorizing; for most use cases you do want to discard the pronunciation part and other nuances of representation to focus on the semantics, but for some (like this) you may want to discard some other parts in order to focus on how the language sounds like. The 'no free lunch theorem' is conceptually relevant even if it doesn't literally apply here. Your example of their/they're/there has some data because the whole words can be mistaken for each other, but even if you take a billion pages of prose, you won't get data to deduce that 'there' rhymes with 'pear' but not 'here', that 'great' rhymes with 'straight' but 'cough', 'dough' and 'through' do not rhyme. A model can't learn something that's simply not represented in the training data. So you either have to bring in external information (i.e. pronunciation models, or dictionary data with pronunciation guides, or audio recordings) or you have to have sufficient rhyming text inside your training data - i.e. train it on corpora of poetry instead of prose. Also, I'm not sure if the variation of pronunciation is critical - any accent variations that affect every instance of some sound similarly would preserve rhyming. There are certain differences e.g. I recall a discussion about some parts of Shakespeare which rhyme perfectly in pronunciation of that period but do not in modern English, but I think that the variations of modern English accents should be mostly OK from that perspective.
oofbey 5y ago
Totally disagree. The encoding scheme could give the model an inductive bias which would make rhyming easier. But the lack of such an inductive bias in no way prevents it from understanding rhymes. This model has 10^11 parameters! The embeddings for the encoding are maybe 0.1% of that - probably less I’d guess without looking it up. Converting to a different encoding would use a very small fraction of the model’s computational power.
- gwern 5y ago
  I'm not talking about inductive bias. I am talking about it being erased from the data before inductive biases or parameters ever have a chance to matter. What you are proposing is tantamount to expecting a model to be able to generate colorful Impressionist masterpieces while trained on solely monochrome bitmaps. Yeah, doesn't work like that. Doesn't matter how many parameters it has; and as a matter of fact, the GPT models have not improved their rhyming noticeably even as they go from a hundred million parameters to over a hundred billion parameters. (And I have extensive evidence consistent with my interpretation at the link, ranging from several years of failure by many people to prove me wrong by eliciting rhymes even a fraction as good as its non-rhyme poetry to fixing GPT-3 performance on tasks by working around BPEs to different models by different groups but the same data encoding also showing the same utter failure to rhyme.)
  - oofbey 5y ago
    I appreciate that you've analyzed GPT3's inability to rhyme rhyme well. And maybe some of that comes from BPE. But saying that "GPT3 _cannot_ rhyme" is a strong and unsupportable statement. What, really, would the difference be between memorizing a rhyming dictionary and being able to apply that vs "actually" learning to rhyme? Because GPT3 can certainly do the former, so why can't it do the latter? Now if you ran an experiment comparing MLM (or any LM) on rhyming tasks with different encodings, then you could certainly make a statement like "BPE is worse at rhyming than other encodings" and it would be scientifically supportable. And that very well might be true. But your extreme conclusion is not supportable.
    - 5y ago

by gwern 5y ago

TaylorAlexander 5y ago

Interesting! So what I am hearing from this is that some day we will overcome this and have programs that can rap. And then companies with customer support bots will have a little check box if you want the bot to rap all responses to you.

eru 5y ago
Technically, we know exactly what we have to do to GPT-3 to give it a chance to learn how to rhyme: get rid of BPES and just use raw UTF-8 (or even just ASCII) as the encoding. Enlarge the model a bit, as necessary. At least that's what I am getting from Gwern's write-up. I might have misunderstood Gwern, or Gwern might be wrong, of course.
- PeterisP 5y ago
  Replacing BPEs not with characters but with a sylabbary (is that a word? a vocabulary made of possible syllables) would be even more powerful, and you could also run preprocessing to convert text to phonetic transcription, to enforce a concept that spelling is irrelevant but the pronunciation matters.
  - eru 5y ago
    That would make the system more complicated by baking in extra assumptions. And eg it would probably perform worse at translating from English to French than a naive system. (Unless you preprocess your corpus extensively to figure out when you get English and when you get a snippet of French.) GPT-3 is surprisingly good at translating English to French. Another problem is that English has not just one text-to-phonetic transcription: different accents pronounce words differently. For just one example: > n most non-rhotic accents, if a word ending in written "r" is followed immediately by a word beginning with a vowel, the /r/ is pronounced, as in water ice. That phenomenon is referred to as "linking R". Many non-rhotic speakers also insert an epenthetic /r/ between vowels when the first vowel is one that can occur before syllable-final r (drawring for drawing). The so-called "intrusive R" has been stigmatized, but many speakers of Received Pronunciation (RP) now frequently "intrude" an epenthetic /r/ at word boundaries, especially if one or both vowels is schwa. For example, the idea of it becomes the idea-r-of it, Australia and New Zealand becomes Australia-r-and New Zealand, the formerly well-known India-r-Office and "Laura Norder" (Law and Order). The typical alternative used by RP speakers (and some rhotic speakers as well) is to insert a glottal stop wherever an intrusive R would otherwise have been placed. https://en.wikipedia.org/wiki/Rhoticity_in_English https://en.wikipedia.org/wiki/Rhoticity_in_English Btw, even without any explicit pronunciation data, I would expect the system to get a pretty good idea of how pronunciation works by essentially using the same technique human linguists use to reconstruct old pronunciations: They observe what kinds of mistakes people make. Eg when you see people mixing up "there" and "their" and "they're" in the corpus, that tells you that in modern English these three are pronounced almost the same. From spelling mistakes in words like carburetor you can figure out that unstressed vowels in English are pretty much all pronounced the same: as a schwa. https://en.wikipedia.org/wiki/Schwa https://en.wikipedia.org/wiki/Schwa You can also learn a lot from observing rhymes.
    - PeterisP 5y ago
      But you have to make those assumptions if you want to make a model for rhyming because large quantities of raw prose does not contain any information whatsoever about which words rhyme, how they are pronounced or where the accents lie. And of course that would be worse for other tasks, that's kind of the whole point - discarding irrelevant information is the key part that distinguishes learning from memorizing; for most use cases you do want to discard the pronunciation part and other nuances of representation to focus on the semantics, but for some (like this) you may want to discard some other parts in order to focus on how the language sounds like. The 'no free lunch theorem' is conceptually relevant even if it doesn't literally apply here. Your example of their/they're/there has some data because the whole words can be mistaken for each other, but even if you take a billion pages of prose, you won't get data to deduce that 'there' rhymes with 'pear' but not 'here', that 'great' rhymes with 'straight' but 'cough', 'dough' and 'through' do not rhyme. A model can't learn something that's simply not represented in the training data. So you either have to bring in external information (i.e. pronunciation models, or dictionary data with pronunciation guides, or audio recordings) or you have to have sufficient rhyming text inside your training data - i.e. train it on corpora of poetry instead of prose. Also, I'm not sure if the variation of pronunciation is critical - any accent variations that affect every instance of some sound similarly would preserve rhyming. There are certain differences e.g. I recall a discussion about some parts of Shakespeare which rhyme perfectly in pronunciation of that period but do not in modern English, but I think that the variations of modern English accents should be mostly OK from that perspective.

oofbey 5y ago

Totally disagree. The encoding scheme could give the model an inductive bias which would make rhyming easier. But the lack of such an inductive bias in no way prevents it from understanding rhymes. This model has 10^11 parameters! The embeddings for the encoding are maybe 0.1% of that - probably less I’d guess without looking it up. Converting to a different encoding would use a very small fraction of the model’s computational power.

gwern 5y ago
I'm not talking about inductive bias. I am talking about it being erased from the data before inductive biases or parameters ever have a chance to matter. What you are proposing is tantamount to expecting a model to be able to generate colorful Impressionist masterpieces while trained on solely monochrome bitmaps. Yeah, doesn't work like that. Doesn't matter how many parameters it has; and as a matter of fact, the GPT models have not improved their rhyming noticeably even as they go from a hundred million parameters to over a hundred billion parameters. (And I have extensive evidence consistent with my interpretation at the link, ranging from several years of failure by many people to prove me wrong by eliciting rhymes even a fraction as good as its non-rhyme poetry to fixing GPT-3 performance on tasks by working around BPEs to different models by different groups but the same data encoding also showing the same utter failure to rhyme.)
- oofbey 5y ago
  I appreciate that you've analyzed GPT3's inability to rhyme rhyme well. And maybe some of that comes from BPE. But saying that "GPT3 _cannot_ rhyme" is a strong and unsupportable statement. What, really, would the difference be between memorizing a rhyming dictionary and being able to apply that vs "actually" learning to rhyme? Because GPT3 can certainly do the former, so why can't it do the latter? Now if you ran an experiment comparing MLM (or any LM) on rhyming tasks with different encodings, then you could certainly make a statement like "BPE is worse at rhyming than other encodings" and it would be scientifically supportable. And that very well might be true. But your extreme conclusion is not supportable.
  - 5y ago