What sort of nonsense chart is that? I can trivially come up with tasks that a competent human can complete in a minute, but that LLMs will absolutely face-plan

What sort of nonsense chart is that? I can trivially come up with tasks that a competent human can complete in a minute, but that LLMs will absolutely face-plant on. In fact I could probably make that line go any direction I wanted to. Does that tell us anything useful? No. They're LLMs, not chess engines, "word count" software, or a game of hangman*. You might as well add "make a sandwich" to the list of tasks. Also 50% is the bar? In most jobs trainees only start actually being worth their wage once they reach about 99%, anything below wastes the time of someone more competent. I wonder how much money is being collectively wasted on trying to shove LLMs into areas where you'd really need AGI, rather than focusing resources on improving LLMs for those areas where they're actually useful. * Though I do recommend attempting to play hangman with an LLM. It's highly entertaining.