4 ms·
These predictions seem wildly reductive in any case and it seems like extrapolating AI's ability to complete task that would take a human 30 seconds -> 10 minut
by lubujackson 1y ago
These predictions seem wildly reductive in any case and it seems like extrapolating AI's ability to complete task that would take a human 30 seconds -> 10 minutes is far different than going from 10 minutes to 5 years. For one reason, a 5 year task generally requires much more input and intent than a 10 minute task. Already we have ramped up from "enter a paragraph" to complicated Cursor rules and rich context prompts to get to where we are today. This is completely overlooked in these simple "graphs go up" predictions.
- echelon 1y agoI'm also interested in error rates multiplying for simple tasks. A human can do a long sequence of easy tasks without error - or easily correct. Can a model do the same?
- kingstnap 1y agoThe recent Apple "LLMs can't reason yet" paper was exactly this. They just tested if models could run an exponential number of steps. Of course, they gave it a terrible clickbait title and framed the question and graphs incorrectly. But if they did the study better it would have been "How long of a sequence of algorithmic steps can LLMs execute before making a mistake or giving up?"