2 ms·

Do you think it's a strike against these points to call them unfalsifiable? They seem no more so than any claim of technological possibility, such as "nuclear f

Do you think it's a strike against these points to call them unfalsifiable? They seem no more so than any claim of technological possibility, such as "nuclear fission is possible", "fission chain reactions are possible", "nuclear weapons are possible", "nuclear fusion is possible", "nuclear fusion can be made economical", etc. Unless a technology violates laws of physics it's very hard to prove through math or experiment that something can't be done. But the AI safety groups don't just assume these without any justification. The existence of the human brain itself is either a proof and very strong launching point for most of these. In 1930, maybe you could have convinced someone that a nuclear bomb is impossible or at least an unfalsifiable worry, because one had never yet been built, but your reasoning is like trying to cast doubt on the possibility of artificial digestion, when there are already billions of stomachs roaming the earth. The idea that a computer can't possibly be made to accomplish whatever a brain can is losing plausibility with every passing day. And as for surpassing it: a human brain is powerful, but has so many surmountable limitations, like: requiring 20 years of education to get up to speed with existing experts; dying after 70-90 years; not being able to run copies of oneself in parallel; etc. We only need to imagine removing these limitations, and doing so violates no known scientific principles. It's like having a 50-megaton warhead in your lab, and meanwhile the rocket scientists are gradually figuring out how to make rockets fly, and you're saying "yes this is a big bomb, but the idea that one of these could be more dangerous if mounted on a missile is an unfalsifiable assumption!"

danaris 4y ago
Nuclear fission, nuclear fusion, digestion—all of these are reasonably well-understood phenomena. The physics and chemistry behind them are clear. Making nuclear fusion energy-positive and economically viable is an engineering problem. Consciousness, intelligence, sapience—these are not well-understood phenomena. We don't know what makes us conscious. We don't know if other animals are, or to what degree. It's not even possible to determine with any scientific certainty that another human being is conscious. As things stand, "we can build an AGI" is not a scientific statement. It is not grounded on a foundation that allows clear reasoning about it, one way or the other. Your arguments are not incorrect; however, they do not bridge the gap to "and so we can definitely make a conscious, sapient, intelligent computer." Being able to replicate particular capabilities of the brain is not the same thing. And, again, that's only the first unfalsifiable proposition that must be satisfied in order for the purported AI threat to be real. They also have to be capable of breaking free of our control, decide we're a threat to them for whatever reason, and have the means to carry it out.
- mitthrowaway2 4y ago
  Artificially sustained nuclear fission was, in 1923, not-well-understood and thought impossible; radioactive decay had been observed but the idea of chain reaction had not yet been conceived (it would take another 10 years). Natural nuclear fusion we currently know to be possible in the sun, but we don't yet know how to sustain it artificially today. Digestion was not-well-understood in 1823; although they knew that it was somehow similar in input-and-output to combustion, which could be imitated artificially, they did not know how to replicate digestion on the chemical level. Now we do. Consciousness is irrelevant, as it's clearly not necessary for AGI nor even human intelligence; otherwise you wouldn't say "it's not even possible to determine with any scientific certainty that another human being is conscious." On what grounds do you believe that there are capabilities of the brain that cannot be replicated by a computer? When I look at the 1.5 kgs of matter in a typical human brain, I don't see anything that jumps out and says "my operation is not computable!" At least with fusion, the high temperatures and pressures are a clear barrier. Our brains don't need to be held at 3.8 trillion psi and 15 million Kelvin in order to enjoy poetry. I can certainly argue on the other points as well (breaking free / deciding threat / acquiring means) but you need to pick your goalposts one at a time. I think the first step would be asking yourself: if you were a super-genius but being held by guards in solitary confinement on a remote island with only a supercomputer connected to the internet, how would you earn money? Those conditions clearly would not have stopped even an ordinary Satoshi Nakamoto from gaining control of sufficient resources to hire private military contractors to arrange his escape. I'm not sure what a superhuman would do, but that's a human baseline.

danaris 4y ago

Nuclear fission, nuclear fusion, digestion—all of these are reasonably well-understood phenomena. The physics and chemistry behind them are clear. Making nuclear fusion energy-positive and economically viable is an engineering problem. Consciousness, intelligence, sapience—these are not well-understood phenomena. We don't know what makes us conscious. We don't know if other animals are, or to what degree. It's not even possible to determine with any scientific certainty that another human being is conscious. As things stand, "we can build an AGI" is not a scientific statement. It is not grounded on a foundation that allows clear reasoning about it, one way or the other. Your arguments are not incorrect; however, they do not bridge the gap to "and so we can definitely make a conscious, sapient, intelligent computer." Being able to replicate particular capabilities of the brain is not the same thing. And, again, that's only the first unfalsifiable proposition that must be satisfied in order for the purported AI threat to be real. They also have to be capable of breaking free of our control, decide we're a threat to them for whatever reason, and have the means to carry it out.

mitthrowaway2 4y ago
Artificially sustained nuclear fission was, in 1923, not-well-understood and thought impossible; radioactive decay had been observed but the idea of chain reaction had not yet been conceived (it would take another 10 years). Natural nuclear fusion we currently know to be possible in the sun, but we don't yet know how to sustain it artificially today. Digestion was not-well-understood in 1823; although they knew that it was somehow similar in input-and-output to combustion, which could be imitated artificially, they did not know how to replicate digestion on the chemical level. Now we do. Consciousness is irrelevant, as it's clearly not necessary for AGI nor even human intelligence; otherwise you wouldn't say "it's not even possible to determine with any scientific certainty that another human being is conscious." On what grounds do you believe that there are capabilities of the brain that cannot be replicated by a computer? When I look at the 1.5 kgs of matter in a typical human brain, I don't see anything that jumps out and says "my operation is not computable!" At least with fusion, the high temperatures and pressures are a clear barrier. Our brains don't need to be held at 3.8 trillion psi and 15 million Kelvin in order to enjoy poetry. I can certainly argue on the other points as well (breaking free / deciding threat / acquiring means) but you need to pick your goalposts one at a time. I think the first step would be asking yourself: if you were a super-genius but being held by guards in solitary confinement on a remote island with only a supercomputer connected to the internet, how would you earn money? Those conditions clearly would not have stopped even an ordinary Satoshi Nakamoto from gaining control of sufficient resources to hire private military contractors to arrange his escape. I'm not sure what a superhuman would do, but that's a human baseline.