3 ms·

I'll wait for a working interactive model before blindly believing these statements. GPT-2 was hyped through the roof, but when inspected with a bit of critical

I'll wait for a working interactive model before blindly believing these statements. GPT-2 was hyped through the roof, but when inspected with a bit of criticality it demonstrated glitches that told us more about how it actually works than "good" examples: https://medium.com/@VictorBanev/interrogating-full-gpt-2-10ae1a9179f6 https://medium.com/@VictorBanev/interrogating-full-gpt-2-10a... ML models should be pushed to their limit, because that's where you gather most useful information about what they actually do. Their results need to be critically examined with both exploratory and hypothesis-driven testing. And yet this is never done in initial papers and rarely done afterwards. What was the last AI paper you've read that that said "and here is a list of things out model failed at"?

gwern 6y ago
That's a very sloppy post. He does a single example, not even running locally or changing sampling parameters, and then concludes that GPT-2 is doing nothing but pattern-matching? A lot of people underestimate NNs because the sampling from them (top-k! how much dumber and cruder can you get? nucleus works better, but is still obviously suboptimal) destroys a lot of dark knowledge. I noticed this with Gary Marcus's claims about GPT-2 too: he would try once, without changing any sampling settings, and conclude that it wasn't doing anything, but if you tried, you would get different results. I'm not the only one to notice that: https://www.quantamagazine.org/common-sense-comes-to-computers-20200430/ https://www.quantamagazine.org/common-sense-comes-to-compute... Such tests can prove the presence of knowledge, but not the absence... And of course, GPT-3 does extensive arithmetic tricks: https://arxiv.org/pdf/2005.14165.pdf#page=22 https://arxiv.org/pdf/2005.14165.pdf#page=22
- gambler 6y ago
  The article I linked to makes claims only about the model it tests and since it actually links to an online implementation, anyone can try to reproduce the results and see for themselves. This is more than I can say about most chatter about ML. >Such tests can prove the presence of knowledge, but not the absence... This sounds like a setup for non-falsifiable beliefs.
  - gwern 6y ago
    > The article I linked to makes claims only about the model it tests and since it actually links to an online implementation, anyone can try to reproduce the results and see for themselves. And I did (using my own local GPT-2-1.5b install which let me set the hyperparameters rather than restricting it to inappropriate hardwired ones of an online service), I linked to another person demonstrating the same thing, I pointed out the extensive GPT-3 evaluation OA did, and here, have another link about how bad querying of language models leads to highly misleading results about how much they know: https://arxiv.org/abs/1911.12543 https://arxiv.org/abs/1911.12543 Measurement error in general biases estimates towards zero. > This sounds like a setup for non-falsifiable beliefs. It's just as non-falsifiable as, say, concepts like 'lower bounds' or 'bugs'.
    - YeGoblynQueenne 6y ago
      The paper you link to claims that hand-crafted queries used to evaluate the knowledge and understanding of language models are "sub-optimal" because they do not take into account the context in which a LM was trained. For example: These manually created prompts (e.g. “Barack Obama was born in _”) might be sub-optimal because LMs might have learned target knowledge from substantially different contexts (e.g. “The birth place of BarackObama is Honolulu, Hawaii.”) during their training. In other words, the paper considers hand-crafted prompts like in the example to be "sub-optimal" because they are not in the right format. To paraphrase them a bit, such prompts are like making a mis-formed query to a database. It is difficult to see how this is an argument for the ability of LMs to demonstrate "understanding". Imagine asking a child: "how much is 4+2?" and getting a correct answer; then asking "how much is 2+4?" and getting a wrong answer. Most people would probably not take that as evidence that the second question was "wrong". They would instead conclude that the child does not "understand" addition and has only learned to reproduce specific answers to specific questions. To be fair the ability to return a correct answer given a question in the right format is not without use. That, indeed, is how databases work. But it shows none of the "understanding" or "knowledge" the paper claims is acquired by Language Models.
      - 6y ago
Imnimo 6y ago
The last AI paper I read that has a list of things the model failed at is this one: https://news.ycombinator.com/item?id=23345379 https://news.ycombinator.com/item?id=23345379 See Section 5, titled "Limitations"

gwern 6y ago

That's a very sloppy post. He does a single example, not even running locally or changing sampling parameters, and then concludes that GPT-2 is doing nothing but pattern-matching? A lot of people underestimate NNs because the sampling from them (top-k! how much dumber and cruder can you get? nucleus works better, but is still obviously suboptimal) destroys a lot of dark knowledge. I noticed this with Gary Marcus's claims about GPT-2 too: he would try once, without changing any sampling settings, and conclude that it wasn't doing anything, but if you tried, you would get different results. I'm not the only one to notice that: https://www.quantamagazine.org/common-sense-comes-to-computers-20200430/ https://www.quantamagazine.org/common-sense-comes-to-compute... Such tests can prove the presence of knowledge, but not the absence... And of course, GPT-3 does extensive arithmetic tricks: https://arxiv.org/pdf/2005.14165.pdf#page=22 https://arxiv.org/pdf/2005.14165.pdf#page=22

gambler 6y ago
The article I linked to makes claims only about the model it tests and since it actually links to an online implementation, anyone can try to reproduce the results and see for themselves. This is more than I can say about most chatter about ML. >Such tests can prove the presence of knowledge, but not the absence... This sounds like a setup for non-falsifiable beliefs.
- gwern 6y ago
  > The article I linked to makes claims only about the model it tests and since it actually links to an online implementation, anyone can try to reproduce the results and see for themselves. And I did (using my own local GPT-2-1.5b install which let me set the hyperparameters rather than restricting it to inappropriate hardwired ones of an online service), I linked to another person demonstrating the same thing, I pointed out the extensive GPT-3 evaluation OA did, and here, have another link about how bad querying of language models leads to highly misleading results about how much they know: https://arxiv.org/abs/1911.12543 https://arxiv.org/abs/1911.12543 Measurement error in general biases estimates towards zero. > This sounds like a setup for non-falsifiable beliefs. It's just as non-falsifiable as, say, concepts like 'lower bounds' or 'bugs'.
  - YeGoblynQueenne 6y ago
    The paper you link to claims that hand-crafted queries used to evaluate the knowledge and understanding of language models are "sub-optimal" because they do not take into account the context in which a LM was trained. For example: These manually created prompts (e.g. “Barack Obama was born in _”) might be sub-optimal because LMs might have learned target knowledge from substantially different contexts (e.g. “The birth place of BarackObama is Honolulu, Hawaii.”) during their training. In other words, the paper considers hand-crafted prompts like in the example to be "sub-optimal" because they are not in the right format. To paraphrase them a bit, such prompts are like making a mis-formed query to a database. It is difficult to see how this is an argument for the ability of LMs to demonstrate "understanding". Imagine asking a child: "how much is 4+2?" and getting a correct answer; then asking "how much is 2+4?" and getting a wrong answer. Most people would probably not take that as evidence that the second question was "wrong". They would instead conclude that the child does not "understand" addition and has only learned to reproduce specific answers to specific questions. To be fair the ability to return a correct answer given a question in the right format is not without use. That, indeed, is how databases work. But it shows none of the "understanding" or "knowledge" the paper claims is acquired by Language Models.
    - 6y ago

Imnimo 6y ago

The last AI paper I read that has a list of things the model failed at is this one: https://news.ycombinator.com/item?id=23345379 https://news.ycombinator.com/item?id=23345379 See Section 5, titled "Limitations"