3 ms·
This roadmap does not allay my fears. People like Eliezer Yudkowsky have been loudly proclaiming since 2007 (1) that there is an infinite variety of possible m
by hollerith 4y ago
This roadmap does not allay my fears.
People like Eliezer Yudkowsky have been loudly proclaiming since 2007 (1) that there is an infinite variety of possible minds, (2) that among the powerful minds, there is an infinite variety of goals -- or "regions in the space of possible outcomes" -- they can be aimed at and (3) that a superintelligence would have to be aimed at a very small and very particular region in the space of possible outcomes for there to be any hope for the survival of the human race. I have seen no argument to the effect that that is either untrue or irrelevant, but there is no sign in this roadmap that the author understands that basic point or that he understands that no one has a credible plan for how to aim a superintelligence at anything let alone at a goal compatible with human survival. There is also no acknowledgement (explicit or implied) of the existence of an organization (namely, MIRI) started 15 years before the start of OpenAI that has been focused from its start on the problem of how to aim a superintelligence. There is no acknowledgement of the fact that MIRI is currently saying that they do not see any plan for how to aim a superintelligence that will not take decades -- and we don't have decades before some reckless actor creates a superintelligence that will kill us all.
Expanding on my (3) above: the explanation for that is that the vast majority of possible minds neither hate nor love humans, but humans (and things they rely on like cities and farms) are made of atoms that can be used for something else. Also, before the superintelligence has consolidated control over Earth, the humans represent a threat to it: they might figure out a way to turn the SI off or (having already succeeded in creating one) they might create a second superintelligence, which would almost certainly compete with the first SI. The easiest way to minimize that threat is to kill all the humans.
Definition: a superintelligence is a generally-intelligent AI better than every human at every endeavor. A chess engine is better than every human at chess, but has no knowledge of or skill at anything else. ChatGPT is generally-intelligent, but inferior in skill and knowledge to most humans. The only thing stopping ChatGPT and all other current generally-intelligent AIs that I know of from killing us all is that they lack the cognitive power to do so, but of course AI researchers are rushing to increase the cognitive power of the next generation of generally-intelligent AIs, which is the primary danger facing humankind.
Also, the author shows no signs of having considered the possibility that gradient descent is simply the wrong approach and that is it simply not humanly possible to use that approach to create a superintelligence that is not fatal to the human race.
Sam Altman continues to emphasize and to focus on the possibility that the power conferred by AI will be unequally distributed among the humans. In reality, what would happen if Vladimir Putin and his allies obtained a decisive advantage in AI is that the AI Putin mistakenly believed is under his control would kill him, his allies and the rest of humankind instead of humankind's being ended by an AI created by OpenAI or DeepMind. The situation is so dire -- the probability of human survival is currently so low (unless by some miracle all AI research is stopped) that Putin's ending up as the only human who can control a superintelligent AI should be considered a very good outcome -- a much better outcome than we can currently realistically hope for (good because at least some humans would survive)! The mere fact that Altman continues to emphasize the unequal-distribution possibility as heavily as he does is strong evidence that he is still not taking the main danger seriously -- even though at least one book (e.g., Nick Bostrom's 2014 book Superintelligence) had been published about the main danger before Altman entered the AI game 8 years ago. (The arguments were available by 2009 in the form of linked web pages, but 2014 is the first time I am aware of that the arguments were made available in convenient, compact book form.)
I'm not asking that Altman and the other leader of the AI research enterprise subscribe to or believe the arguments, I am asking that they acknowledge they even exist, and maybe explain why they do not find them convincing. I certainly find them convincing as do many others.