2 ms·

I'm not sure why you interpreted my comment as "antagonistic" or "unfriendly". Yes I do disagree with you (at least assuming I'm understanding you correctly), b

I'm not sure why you interpreted my comment as "antagonistic" or "unfriendly". Yes I do disagree with you (at least assuming I'm understanding you correctly), but I'm not attacking you personally, just your argument. I guess it's fair to say my examples are cherry-picked too (though I didn't really give specific examples in my comment so much as entire general categories of problems ChatGPT is known to be proficient at solving). But they aren't so cherry-picked as for "random chance" or "that example was in the training data verbatim" to be possible explanations. It's not like I had ChatGPT answer 100 billion questions and am only showing you the top 0.1%. It's very common for ChatGPT to be perfectly correct even when answering logic puzzles or coding problems not in its training set. So if not those then what's your explanation other than "understanding"? On the flip side, I don't think it's possible to prove ChatGPT does not posses understanding with individual examples. (And it seems you agree?) Those are easily dismissable as just "ChatGPT isn't great at that particular task". Particularly given how many other tasks ChatGPT is great at. Scott Alexander figured this out back in 2019 before GPT-3 was even a thing (let alone ChatGPT), I think it's worth a read: https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-general-intelligence/ https://slatestarcodex.com/2019/02/19/gpt-2-as-step-toward-g... > The example was shown because it demonstrates a simple question that we'd expect any reasonable understanding creature, that knows what a pound and kilogram are, to be able to answer. Not really though. The "pound of bricks or pound of feathers" question is specifically designed to trip up humans, and the specific formulation you used seems specifically designed to trip up LLMs (by playing off their tenancy to pattern match common sayings), yet despite those disadvantages GPT-4 succeeds. Further, I don't think "I'd expect even a child to be able to answer this" is a good metric. ChatGPT isn't human, so we shouldn't expect it to be good at everything humans are good at just because it's good at some things humans are good at. Again, I'm not trying to argue ChatGPT is a human-like AGI.

godelski 3y ago
> I'm not sure why you interpreted my comment as "antagonistic" It was where you said cherry picking. I'll admit it was an over correction given your follow-up, but internet comments (especially when disagreement exists) end up being combative. Many times unintentionally. The point of the specific example is not that it is a trick question. It is more that the answer is self contradictory. That's the key part and what demonstrates that it doesn't understand. The second example is that follow-up questions do not result in a convergence. This is related to cognition but not as strong. A bias to pattern match is an issue but you're right that it doesn't disprove sentience or even understanding. But a self contradiction does demonstrate a lack of understanding. If a human gave that exact answer, you would be very confused and how someone could be so stupid. Remember, it does currently identify the relationship between pounds and kilograms, fully explaining even the true base comparison through common units. But it still gets it wrong. That's the critical part. That's overfitting to the statistical nature and that this takes far more priority than meaning. Getting it wrong is one thing. Getting it wrong and using the right answer to justify it's incorrect answer is another thing. A decent example between knowledge and understanding. It's not that a child would get the answer right, it's that the child would easily identify it's self inconsistency were it to give the same explanation. As for gpt3, we have to remember that this is quite a different model than any of the chat versions (typically including gpt4). It was never trained through RLHF, which introducers many more biases as it dramatically changes the latent distributions. GPT base is often jokingly called a babbler, as these are more word prediction models. The chat aspect changes things. But I wouldn't expect anyone not deep in the literature to understand why these are extremely important differences just the same way I wouldn't expect an average person to understand why there's a big difference between a flat head screw and a slotted screw drive. I don't want to shut down a conversation through authority (it doesn't exist on HN) but I have to state that it's difficult to go down this path without bringing up a lot more background material. We have to really dig into theory of mind, cognition, as well as get nuanced about NLP and LLMs in general. That's far too cumbersome than I'm willing to write in comments. But these things are essential priors to make the arguments we are discussing here. And I truly mean that this is not something that can be learned from YouTube and quite difficult to learn on the Internet. These are difficult subjects to learn in even the best settings with lots of nuances that are brushed away in introductory materials but critical when we discuss the meat and potatoes.

godelski 3y ago

> I'm not sure why you interpreted my comment as "antagonistic" It was where you said cherry picking. I'll admit it was an over correction given your follow-up, but internet comments (especially when disagreement exists) end up being combative. Many times unintentionally. The point of the specific example is not that it is a trick question. It is more that the answer is self contradictory. That's the key part and what demonstrates that it doesn't understand. The second example is that follow-up questions do not result in a convergence. This is related to cognition but not as strong. A bias to pattern match is an issue but you're right that it doesn't disprove sentience or even understanding. But a self contradiction does demonstrate a lack of understanding. If a human gave that exact answer, you would be very confused and how someone could be so stupid. Remember, it does currently identify the relationship between pounds and kilograms, fully explaining even the true base comparison through common units. But it still gets it wrong. That's the critical part. That's overfitting to the statistical nature and that this takes far more priority than meaning. Getting it wrong is one thing. Getting it wrong and using the right answer to justify it's incorrect answer is another thing. A decent example between knowledge and understanding. It's not that a child would get the answer right, it's that the child would easily identify it's self inconsistency were it to give the same explanation. As for gpt3, we have to remember that this is quite a different model than any of the chat versions (typically including gpt4). It was never trained through RLHF, which introducers many more biases as it dramatically changes the latent distributions. GPT base is often jokingly called a babbler, as these are more word prediction models. The chat aspect changes things. But I wouldn't expect anyone not deep in the literature to understand why these are extremely important differences just the same way I wouldn't expect an average person to understand why there's a big difference between a flat head screw and a slotted screw drive. I don't want to shut down a conversation through authority (it doesn't exist on HN) but I have to state that it's difficult to go down this path without bringing up a lot more background material. We have to really dig into theory of mind, cognition, as well as get nuanced about NLP and LLMs in general. That's far too cumbersome than I'm willing to write in comments. But these things are essential priors to make the arguments we are discussing here. And I truly mean that this is not something that can be learned from YouTube and quite difficult to learn on the Internet. These are difficult subjects to learn in even the best settings with lots of nuances that are brushed away in introductory materials but critical when we discuss the meat and potatoes.