4 ms·

The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obv

The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obvious things, it is because it is an alien form of intellingece. RLHF and the tokenizer together explain many of the more common failure modes.

civilized 4y ago
Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it definitely hasn't seen is itself now a hard problem. What's been shown so far: OpenAI's benchmarking is not always rigorous or appropriate, and GPT's performance is brittle and highly sensitive to the problem phrasing [1]. I agree with the article that GPT's training enables it to access meaningful conceptual abstractions. But there is clearly quite a lot that's still missing. For now, people are too excited to care. But when they try to deliver on their promises, the gaps will still be there. Hopefully at that point we will embark towards a new level of understanding. [1] https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
- wizofaus 4y ago
  It's not that hard to give it problems it hasn't seen - you can take a classic description of a logical thinking exercise the text for which does occur online, then mix it up in ways that doesn't change the underlying pattern of reasoning necessary to solve it, and at least from the tests I've done it will confidently tell you the incorrect answer (along with some semi-plausible but fatally flawed description of the reasoning it used to come up with the answer). In at least one case it was certain the answer was exactly the answer given in the common online statement of the problem, despite the fact I'd changed all the inputs such that the given answer was obviously not an option (it was a guess so-and-so's birthday type problem). Even for simple arithmetic just choosing sufficiently large numbers will bring it unstuck.
  - civilized 4y ago
    My experience as well. I produced another instance of this phenomenon in a comment just now: https://news.ycombinator.com/item?id=35313768 https://news.ycombinator.com/item?id=35313768 If a human mathematician said the things ChatGPT said in this dialogue, you would wonder if the person had recently suffered a severe traumatic brain injury.
  - famouswaffles 4y ago
    This is a failure mode of people as well. Rewriting so it doesn't bias common priors or at least in the case of Bing, telling it it's making a wrong assumption works.

civilized 4y ago

Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it definitely hasn't seen is itself now a hard problem. What's been shown so far: OpenAI's benchmarking is not always rigorous or appropriate, and GPT's performance is brittle and highly sensitive to the problem phrasing [1]. I agree with the article that GPT's training enables it to access meaningful conceptual abstractions. But there is clearly quite a lot that's still missing. For now, people are too excited to care. But when they try to deliver on their promises, the gaps will still be there. Hopefully at that point we will embark towards a new level of understanding. [1] https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...

wizofaus 4y ago
It's not that hard to give it problems it hasn't seen - you can take a classic description of a logical thinking exercise the text for which does occur online, then mix it up in ways that doesn't change the underlying pattern of reasoning necessary to solve it, and at least from the tests I've done it will confidently tell you the incorrect answer (along with some semi-plausible but fatally flawed description of the reasoning it used to come up with the answer). In at least one case it was certain the answer was exactly the answer given in the common online statement of the problem, despite the fact I'd changed all the inputs such that the given answer was obviously not an option (it was a guess so-and-so's birthday type problem). Even for simple arithmetic just choosing sufficiently large numbers will bring it unstuck.
- civilized 4y ago
  My experience as well. I produced another instance of this phenomenon in a comment just now: https://news.ycombinator.com/item?id=35313768 https://news.ycombinator.com/item?id=35313768 If a human mathematician said the things ChatGPT said in this dialogue, you would wonder if the person had recently suffered a severe traumatic brain injury.
- famouswaffles 4y ago
  This is a failure mode of people as well. Rewriting so it doesn't bias common priors or at least in the case of Bing, telling it it's making a wrong assumption works.