4 ms·
Models don't know what their cutoff dates are unless told via a system prompt. The proper way to figure out the real cutoff date is to ask the model about thin
by bakugo 6mo ago
Models don't know what their cutoff dates are unless told via a system prompt.
The proper way to figure out the real cutoff date is to ask the model about things that did not exist or did not happen before the date in question.
A few quick tests suggest 5.5's general knowledge cutoff is still around early 2025.
- czk 6mo agoi wonder if they put an older cutoff date into the prompt intentionally so that when asked on more current events it leans towards tool calls / web searches for tuning
- ssl-3 6mo agoI wonder if the cutoff date is the result of so many people posting about the date over time and poisoning the data. "Dead cutoff date theory," perhaps. Whatever it is, the cutoff date reporting discrepancy isn't new. Back when Musk was making headlines about buying/not buying Twitter, I was able to find recent-ish related news that was published well after the bot's stated cutoff date. ChatGPT was not yet browsing/searching/using the web at that point. That tool didn't come for another year or so.
- soco 6mo agoStupid question: wouldn't it then search the web for that event?
- bakugo 6mo agoIf you have web search enabled, sure. But if you're testing on the API, you can just not enable it.
- MallocVoidstar 6mo agoOpenAI does tell the model the current date via API, so it's odd for them not to also tell the model its cutoff