2 ms·

> This would explain why even the largest language models can't generalize beyound the training set. Why do you say this, they do. I can teach gpt about new ob

> This would explain why even the largest language models can't generalize beyound the training set. Why do you say this, they do. I can teach gpt about new objects I have made up and it’s physical properties and it can understand them. That seems a clear example of generalizing beyond training set, or do you not think so?

euphetar 5y ago
It's repeating the same patterns it learned from text. Maybe with different specific objects, but still. One good way to test this is to ask it to count. It will break very soon. A person is able to build a rule in their head: "one apple is 1, two apples are 2, three apples are... 1+2 = 3". Let's test GPT-2: https://huggingface.co/gpt2?text=One+apple+is+1%2C+two+apples+are+2%2C+three+apples+are https://huggingface.co/gpt2?text=One+apple+is+1%2C+two+apple... Prompt: "One apple is 1, two apples are 2, three apples are" Model output: "One apple is 1, two apples are 2, three apples are 4, four apples are 5, six apples are 7, seven apples are 8, nine apples are 10, ten apples are 11 (for apples being the perfect length of life)." Even if you use a special dataset to each it to count, it won't be able to count beyound the examples in the training set. So it's spewing plausible-sounding gibberish at you (i.e. approximating the training set distribution) It doesn't generalize. Not in the sense that it can't give you a phrase that didn't exist in the training set. It can. But it can't give you a new kind of phrase, of a "kind" that didn't exist in the training set.
- VirusNewbie 5y ago
  >But it can't give you a new kind of phrase, of a "kind" that didn't exist in the training set. Can you give a concrete example that doesn't involve math (which GPT is a bit handicapped at, because of the way its encoded)? I feel this is a bit like 'moving the goalpost'. It seems to me plenty of humans only ever repeat things they've heard and aren't coming up with novel, complex abstractions or ideas...

euphetar 5y ago

It's repeating the same patterns it learned from text. Maybe with different specific objects, but still. One good way to test this is to ask it to count. It will break very soon. A person is able to build a rule in their head: "one apple is 1, two apples are 2, three apples are... 1+2 = 3". Let's test GPT-2: https://huggingface.co/gpt2?text=One+apple+is+1%2C+two+apples+are+2%2C+three+apples+are https://huggingface.co/gpt2?text=One+apple+is+1%2C+two+apple... Prompt: "One apple is 1, two apples are 2, three apples are" Model output: "One apple is 1, two apples are 2, three apples are 4, four apples are 5, six apples are 7, seven apples are 8, nine apples are 10, ten apples are 11 (for apples being the perfect length of life)." Even if you use a special dataset to each it to count, it won't be able to count beyound the examples in the training set. So it's spewing plausible-sounding gibberish at you (i.e. approximating the training set distribution) It doesn't generalize. Not in the sense that it can't give you a phrase that didn't exist in the training set. It can. But it can't give you a new kind of phrase, of a "kind" that didn't exist in the training set.

VirusNewbie 5y ago
>But it can't give you a new kind of phrase, of a "kind" that didn't exist in the training set. Can you give a concrete example that doesn't involve math (which GPT is a bit handicapped at, because of the way its encoded)? I feel this is a bit like 'moving the goalpost'. It seems to me plenty of humans only ever repeat things they've heard and aren't coming up with novel, complex abstractions or ideas...