5 ms·
England/London is a bad example, but this one (based on the paper) is easy to replicate with GPT4 and other top models: > Who is Mary Lee Pfeiffer's son? >> M
by sprobertson 3y ago
England/London is a bad example, but this one (based on the paper) is easy to replicate with GPT4 and other top models:
> Who is Mary Lee Pfeiffer's son?
>> Mary Lee Pfieffer's son is Sean Pfieffer.
> Who is Tom Cruise's mother?
>> Tom Cruise's mother is Mary Lee Pfeiffer.
> Who is Mary Lee Pfeiffer's son?
>> Mary Lee Pfeiffer's son is the actor Tom Cruise.
Of course the second time around (with models that keep context) it "realizes" its mistake, sometimes apologizing. I've tried this one tens of times and never seen it get Tom Cruise, but it will often pick another random celebrity like Harrison Ford or Eddie Van Halen.
- chagen 3y agoNice, thank you for this. I was able to replicate your example: https://chat.openai.com/share/6d7e4c6c-a753-4ffb-a021-6d1f66ce65be https://chat.openai.com/share/6d7e4c6c-a753-4ffb-a021-6d1f66... If GPT-4 has access to Bing, it has no issue, but if you ask it to answer without using Bing, then it will say it doesn't know.
- sprobertson 3y agoAh good point, I have "Please do not use web search unless asked." as part of my system prompt.