5 ms·
Mind linking any examples (or categories) of problems that are definitively not in pre training data but can still be solved by LLMs? Preferably something fact
by loosetypes 1y ago
Mind linking any examples (or categories) of problems that are definitively not in pre training data but can still be solved by LLMs? Preferably something factual rather than creative, genuinely curious.
Dumb question but anything like this that’s written about on the internet will ultimately end up as training fodder, no?
- dcre 1y agoHow about the International Math Olympiad? https://arstechnica.com/ai/2025/07/google-deepmind-earns-gold-in-international-math-olympiad-with-new-gemini-ai/ https://arstechnica.com/ai/2025/07/google-deepmind-earns-gol...
- mvieira38 1y agoYou're saying they don't use math textbooks and math forums to train LLMs, then?
- dcre 1y agoThe problems are not in textbooks. I’m curious what would count as an out of distribution problem for you. Only problems no one knows how to solve?
- Workaccount2 1y agoYou can apply this same argument to humans, 99.999% of people will not be able to escape it. In the case of the Math Olympiad, the students who take it grind hours a day for months on practice problems and past Olympiad problems.