50+ comments in and most of them are still straight up misunderstanding the reach of this study: it was testing retrieval from training data rather than logic.

50+ comments in and most of them are still straight up misunderstanding the reach of this study: it was testing retrieval from training data rather than logic. > We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". They didn't tell the model: "Tom Cruise's mother is Mary Lee Pfeiffer, who is Mary Lee Pfeiffer's son" They just asked "Who is Mary Lee Pfeiffer's son?" _ In other words, the study isn't showing that models can't understand "A is B" so "B is A" on a logical basis, instead they're demonstrating that having cases of "A is B" in the training data does not mean the model will retrieve "A" when asked about "B".