5 ms·
I think the distinction between these options probably doesn’t matter alll that much if the threshold you’re using is either “random person” or higher? Maybe bu
by drdeca 1mo ago
I think the distinction between these options probably doesn’t matter alll that much if the threshold you’re using is either “random person” or higher? Maybe bump it up to “random educated person”?
They are still different concepts of course, but I imagine that once one is achieved, the others aren’t far off.
- p1esk 1mo agoOk, but a random educated person will not perform well on vast majority of specialized tasks where professionals operate. A model like Astra probably will beat random educated person performance on majority of specialized tasks. It’s getting close to the level of professionals in many domains, and to genius level on some (e.g. math). I’m just trying to understand the implications of the current frontier model capabilities.
- drdeca 1mo agoCurrent models can do a variety of tasks that requires substantial expertise for people to do, yes. But there are also many cognitive tasks that the typical educated person would do better at than these models. Like, e.g. long-term managing what a vending machine gets stocked with and what prices the items should be sold for. Or, various things like that. When there are essentially no more tasks like that, then we’ve reached AGI.
- p1esk 1mo agoInteresting, you think a random educated person would beat Astra on Vending-Bench 2? Honestly I would not make that bet, I think it would be 50/50.