the examples are about relations in natural language rather than formal logic though. Mary Lee Pfeiffer being Tom Cruise's mother definitely does imply that Tom

the examples are about relations in natural language rather than formal logic though. Mary Lee Pfeiffer being Tom Cruise's mother definitely does imply that Tom Cruise is a valid answer to questions about who Mary Lee Pfeiffer's son is (it doesn't necessarily imply she doesn't have other sons or there isn't another lady of that name who is childless, but that isn't what's tripping the model up). And there is absolutely nothing ambiguous about the failures of the fine-tuned model where the author gave it specific phrases exclusively associated with a fake name in the fine-tuning training set and when prompted for "who is $specific_phrase", supplied different names exclusively associated with completely different phrases (unlike the author, I don't see [serendipitously or otherwise] picking name words from the right training set as "B kinda-has-something-to-do-with A generalization", not when it's lost so much information that the combination of name words which exclusively appears in sentences with that phrase isn't treated as a more probable response. Never mind emergent understanding of syntax, it isn't even making obvious inferences from proximity here) If GPT4 rarely makes those errors it's clearly not an insurmountable problem at this level of training, but it does imply a lot more difficulty fine-tuning models to reliably retrieve correct information from specific text.