3 ms·
It's not strictly compute, because this has noticeably improved in open-weight models too, such as Gemma and Qwen. I suspect they noticed this issue and adjuste
by Catloafdev 17d ago
It's not strictly compute, because this has noticeably improved in open-weight models too, such as Gemma and Qwen. I suspect they noticed this issue and adjusted their training to be better about it over time.
- ahmedfromtunis 17d agoI built a toy news-summarizing agent with Gemma 4, and it was so frustrating, actually, because of the cut-off date. The model wasted over half the token budget, each time, on internal debates over the current date. When generating a World Cup summary, for example, it refused to believe qualification rounds were over and refused to even call the web searching tool to collect the data. I injected the current datetime at the very beginning of the system prompt, but Gemma refused to believe it! The m-effer insisted the timestamp was fake and hypothesized it was being evaluated in a synthetic lab test with simulated future dates! No amount of system prompting could convince it to trust the clock. That was the most frustrating and bizarre "bug" I ever faced!
- ThunderSizzle 17d agoDid you try other models (e.g. Qwen3.6 or others)? I'm curious how others fare. I've noticed Qwen3.6 struggles a bit with today/date based logic.
- carlos_rpn 17d agoQwen 3.6 did the same thing for me. Only after some cajolling it finally went to check the history I asked it to (I had been testing KoboldCPP's web search).