Feels like what you're referring to is just "understanding". GPT3 fails at this as well. It seems that gpt3 can string together sentences and whole paragraphs t

Feels like what you're referring to is just "understanding". GPT3 fails at this as well. It seems that gpt3 can string together sentences and whole paragraphs that seem to sort of make sense together but it seems that it never really puts hard constraints on any concepts. It doesn't try to build a comprehensive world with rules. Everything is just a sort of a fuzzy continuum. If you train it on enough data this fuzzyness generate a lot of plausibly sounding output but it will have a hard time coming up with the right answer in a new situation. I feel the missing ingredient is to be able to combine this fuzzy or intuitive understanding with some kind of rule engine. It's like in the book "Thinking Fast and Slow". It feels like GPT and the best image recognition algorithms we have today are good at the fast kind of thinking but doesn't have the slow and logical part.