I don't think it's crazy to talk about plateaus, it just depends on what domain we're talking about. Performance on olympiad-style problems doesn't necessarily

I don't think it's crazy to talk about plateaus, it just depends on what domain we're talking about. Performance on olympiad-style problems doesn't necessarily translate into success in research, or industry, or creative pursuits. We know this is true for humans, then add on to that all the usual problems with LLMs like hallucinations and you can see why some people are still skeptical. I'm still in the "wait and see" stage. Maybe throwing more compute at the problem will solve it, but maybe not. I would like to see benchmarks that take a more project-based approach, e.g. tell the LLM to go work on something complicated and ambiguous for a week and see what it comes up with.