5 ms·
'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws g
by jimmcslim 10y ago
'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.'
Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into.
A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible.
More relevant, a superintelligent being isn't going to be able to enslave us all with robots if the laws of chemistry don't permit a quantum leap in battery chemistry.
- idlewords 10y agoThe place where AI alarmists seem to forget this most often is in computational complexity, and particularly in the power of super-hyper-computers to simulate reality. Bostrom in particular doesn't seem to appreciate the constraints complexity theory puts on what an AI could calculate (unless he thinks P=NP).
- eemax 10y agoHere's a response about why complexity theory may not constrain AI that much: https://www.gwern.net/Complexity%20vs%20AI https://www.gwern.net/Complexity%20vs%20AI It is something "AI alarmists" think about.
- mrfusion 10y agoWhat's his reason? I got halfway through that and realized I didn't understand.
- paulbaumgart 10y agoThat even if you see diminishing returns to what you can accomplish with greater intelligence, a small advantage will still compound into a large one over the theoretically infinite lifespan of an AI.
- swolchok 10y ago> the theoretically infinite lifespan of an AI Infinite? Why isn't AI subject to the heat death of the universe?
- paulbaumgart 10y agoIt just might be smart enough to figure out a way around that.
- fjdlwlv 10y agoNaturally , Asimov already wrote about this. _The Last Question_ and _The Last Answer_ Really,Asimov is more on point and more fun to read than all these Superintenlligence pundits
- Twisell 10y agoHe actually even wrote about the current debate. In his multiverse robots were actually banned from earth by bigots at some point. I miss him I only started to read his book after his death. RIP Good doctor
- erikpukinskis 10y agoOoh thanks for this. I've read THQ at least once before, but am really enjoying reading it again. I really liked this bit, emphasis mine as it relates to our current spectating of a dying ecosystem: "Unhappily, Zee Prime began collecting interstellar hydrogen out of which to build a small star of his own. If the stars must someday die, at least some could yet be built."
- TeMPOraL 10y agos/theoretically/practically/, and I doubt it changes the conclusion that much (in practice).
- paulbaumgart 10y ago
- drakenot 10y agoWhy would they be any more constrained than our biological brains? Humans are able to calculate plenty with a relatively small amount of mass and energy.
- idlewords 10y agoI mean that for a lot of things, doubling our brainpower would would only extend our predictive horizon by a tiny amount. There is a large class of problems for which you have fairly good solutions up front, but where even vast amounts of additional computing power don't help you do much better.
- marcosdumay 10y ago> I mean that for a lot of things, doubling our brainpower would would only extend our predictive horizon by a tiny amount. Unless you come out with some algorithm that is sub-exponential for the important cases, and only gets slower for stuff you don't care about, or happen very rarely in practice. People invent those algorithms all the time, for lots of kinds of problems.
- trishume 10y agoThe thing is it doesn't need a huge leap in any specific thing, it needs a quantum leap in anything at all. Complexity only constrains you from solving all problems of a given type in reasonable time, not from solving any instance. There are even many things that we already suspect are just engineering challenges. Like rocketry, hacking and robotics. Even if an AI can just act like a million engineers and scientists working for a decade per second, without any higher level of thinking, it could do some incredible things.
- krashnburn200 10y agoThe laws of the universe forbid steam locomotives in space, and yet we reached mars regardless. And here we are, barely even regular Inteligences. The laws of computational complexity seem like a stronger candidate for a hard limit, but that only means that to get smarter they will require more matter to distribute comoutations across. The battery limit proposal is just silly.
- mcguire 10y agoTherefore, to paraphrase Cato, AI research must be destroyed.
- darawk 10y agoAll matter is, in principle, a battery. There are no physical laws that prevent near limitless energy storage.
- Retra 10y agoThere are if you want to have a functioning computer in the middle of all that energy. There are laws governing heat dispersal (speed of light in only 3 dimensions) and laws governing the collapse of information processing structures into black holes.
- darawk 10y agoSure, but none of those laws would prevent a superintelligence from generating more than enough electricity to power itself, in principle.
- TeMPOraL 10y agoIndeed, this seems like pointing to the speed of light as a limit in a discussion about building faster cars.
- mcguire 10y agoTherefore, to paraphrase Cato, AI research must be destroyed.
- Retra 10y agoAI's powering themselves isn't the objection. Exponential increase in capability is. And that requires you to deal with things like heat dissipation, which are real problems for our current "dumb" computers.
- darawk 10y agoIt's not a huge problem for humans. I see no reason to think that a computer say, 10x as powerful as a human would have major problems with heat dissipation.
- ewjordan 10y agoThat "quantum leap in battery chemistry" is obviously physically possible, given that animals exist...
- TeMPOraL 10y agoOr even forgetting biology for a second: - gasoline is a liquid battery with much greater energy density than our solid ones - nuclear batteries are a thing, too
- Florin_Andrei 10y agoI think a lot of people forget one thing: intelligence is basically as good as the info you're putting into it. So even if you have a tremendous AI, it can only be as smart as the info it can access. It could only have god-like powers in realms from which it could extract some information. It's still very much subject to being blinded by ignorance (in the most literal sense of the word). But I do agree that, if it has complete information, an AI could potentially beat us at anything that requires thinking.
- TeMPOraL 10y ago> A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. It isn't, but if it wants one then it will likely go and discover the actual physical constraints. > More relevant, a superintelligent being isn't going to be able to enslave us all with robots if the laws of chemistry don't permit a quantum leap in battery chemistry. That's what differentiates even a somewhat intelligent AI from technologies we have today. Enslaving us with battery-powered robots is but one way to reach the goal. There are many others. If we stick to robots, you have gasoline-powered robots (gasoline is a liquid battery that's still 30x as dense as our solid batteries). Then you have nuclear batteries, which we know are feasible, but don't exist because of political reasons. And then again, robots are but one way to reach a goal...
- red75prime 10y agoRegarding enslavement with robots. Superintelligent means that if it is not feasible to enslave humanity with robots, then SAI will find feasible way: human collaborators, genetically engineered fungi, backing of puppet president candidate, fabricating external threat and convincing people that only it can handle it, and so on and so forth.