It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant

It's the wrong question. When this comes up in common conversations like this, it always turns to what is understanding, is this real intelligence? Okay, grant that is. GPT-4 is a human-level intelligence. That doesn't make it the kind of existential threat these kinds of pieces are worrying over. We can operationalize that capability far better than we can vague goals we can barely articulate like "intelligence." What it needs to achieve recursive self-improvement, at bare minimum, is the ability to conduct original machine learning research, better than people like Geoff Hinton and Yoshua Bengio. I have no clue what kinds of related abilities we should be looking for in terms of text it generates to start believing it can possibly do that, but I don't think anyone is suggesting that's a realistic expectation for GPT-4. Even memorizing all of LeetCode and taking ten billion practice LSATs until you've become the top idiot savant in history doesn't mean you can replace all of science. The projection that we'll get there sooner and sooner than previously thought is clearly based on extrapolation from GPT-4 and its generation performing better than was expected on the tasks it is good at, but the point seems to stand that this is a dubious extrapolation. Without any clear idea of new directions to take, it entirely stands on how realistic we believe the strategy of "just keep scaling forever" to be. At what point do we run out of more compute? At what point do we run out of energy? At what point does it simply become uneconomical? At what point do we ru out of useful training data? Those are the far more concrete questions that need to be answered if if we're to believe the current course is going to get us a planet-killing god, not "is this thing actually intelligent?"