5 ms·

A lot of experts also deny dangers coming from global warming and other invasions in complex systems we don't fully understand. Scientists in the 1940s also dow

by rndn 12y ago

A lot of experts also deny dangers coming from global warming and other invasions in complex systems we don't fully understand. Scientists in the 1940s also downplayed the danger of the atomic bomb. It's easy to brush away a hypothetical danger with arguments that resources might be wasted or that people in charge might embarrass themselves if their assumptions turn out to be wrong, or that research might suffer due to overblown expectations. When, do you think, is the time that we should start worrying about SMI? When machines outsmart a frog, or a chimpanzee? What if intelligence is just a matter of scale at this point, i.e. that adding more short and long term memory makes it easily reach and even surpass human intelligence?

lazaroclapp 12y ago
By the time the atomic bomb was truly imminent (in the sense of it being within the reach of massive dedicated investment by the most powerful nations on Earth), scientists knew very well of the dangers [1]. As for global warming. No expert denies global warming, and you'll be hard pressed to find many that deny anthropic global warming [2]. This has been the case for many decades. I am not an AI expert, but from what I can see thus far, our knowledge of cognition seems to be as far from human or post-human intelligence as Nitroglycerin was from the Atomic Bomb (e.g. fundamentally different principles at work, which is not subject to exponential improvement without one or more breakthroughs or qualitatively different approaches). Also, our AI-Nitroglycerin seems scary enough: autonomous weapons (think self-driving self-firing drones, not terminator), massive changes in our means of production that reduce human labor/employment and might cause social earthquakes and the ability to draw meaningful conclusions from surveillance data of 7 billion people... [1] https://en.wikipedia.org/wiki/Einstein%E2%80%93Szil%C3%A1rd_letter https://en.wikipedia.org/wiki/Einstein%E2%80%93Szil%C3%A1rd_... (and this seems to have been understood as an approaching risk in the physics community way before anyone thought of telling FDR). [2] https://en.wikipedia.org/wiki/Global_warming_controversy#Scientific_consensus https://en.wikipedia.org/wiki/Global_warming_controversy#Sci...
- rndn 12y ago
  Fair enough, but I wouldn't exclude the possibility that AI experts are biased for at least three reasons, namely (1) that they believe in some sort of sanctity of the human mind (like many people do), (2) that they don't want to be regulated and (3) that they don't want to raise overblown expectations. There are definitely historical examples in which the majority of experts was wrong. I also find it incredibly difficult to tell what is missing about an algorithm. Perhaps multiple breakthroughs are necessary, perhaps just one (there is no good reason to believe that an algorithm that yields AGI can't be considerably simpler than what happens in the human brain). Initially, nobody thought that search could be done in O(log n) and similarly not many researchers expected neural networks to perform useful tasks until back-propagation was invented and demonstrated.
- MelatoninRonin 12y ago
  Brilliant. If, however, it isn't cognition that needs to be fully understood, but learning and meta-learning, then we could theoretically have an 'and then there was light' scenario that just speeds right out of the gate. This case is dissimilar from your examples in that AI could alter itself, it could in some ways be considered alive, if not obviously alive. I think we need to soberly look at the fine details with our collective perspectives, observe the sociocultural response, and gauge our best course of action. Perhaps progress should sometimes be stopped. The effects of the atomic bomb and its research are the very foundations of this world, AI has contemporary magnitude, and could alter the game even more significantly. It is my guess that paradigm shifts in human consciousness have quite unpredictable effects.

by rndn 12y ago

lazaroclapp 12y ago

By the time the atomic bomb was truly imminent (in the sense of it being within the reach of massive dedicated investment by the most powerful nations on Earth), scientists knew very well of the dangers [1]. As for global warming. No expert denies global warming, and you'll be hard pressed to find many that deny anthropic global warming [2]. This has been the case for many decades. I am not an AI expert, but from what I can see thus far, our knowledge of cognition seems to be as far from human or post-human intelligence as Nitroglycerin was from the Atomic Bomb (e.g. fundamentally different principles at work, which is not subject to exponential improvement without one or more breakthroughs or qualitatively different approaches). Also, our AI-Nitroglycerin seems scary enough: autonomous weapons (think self-driving self-firing drones, not terminator), massive changes in our means of production that reduce human labor/employment and might cause social earthquakes and the ability to draw meaningful conclusions from surveillance data of 7 billion people... [1] https://en.wikipedia.org/wiki/Einstein%E2%80%93Szil%C3%A1rd_letter https://en.wikipedia.org/wiki/Einstein%E2%80%93Szil%C3%A1rd_... (and this seems to have been understood as an approaching risk in the physics community way before anyone thought of telling FDR). [2] https://en.wikipedia.org/wiki/Global_warming_controversy#Scientific_consensus https://en.wikipedia.org/wiki/Global_warming_controversy#Sci...

rndn 12y ago
Fair enough, but I wouldn't exclude the possibility that AI experts are biased for at least three reasons, namely (1) that they believe in some sort of sanctity of the human mind (like many people do), (2) that they don't want to be regulated and (3) that they don't want to raise overblown expectations. There are definitely historical examples in which the majority of experts was wrong. I also find it incredibly difficult to tell what is missing about an algorithm. Perhaps multiple breakthroughs are necessary, perhaps just one (there is no good reason to believe that an algorithm that yields AGI can't be considerably simpler than what happens in the human brain). Initially, nobody thought that search could be done in O(log n) and similarly not many researchers expected neural networks to perform useful tasks until back-propagation was invented and demonstrated.
MelatoninRonin 12y ago
Brilliant. If, however, it isn't cognition that needs to be fully understood, but learning and meta-learning, then we could theoretically have an 'and then there was light' scenario that just speeds right out of the gate. This case is dissimilar from your examples in that AI could alter itself, it could in some ways be considered alive, if not obviously alive. I think we need to soberly look at the fine details with our collective perspectives, observe the sociocultural response, and gauge our best course of action. Perhaps progress should sometimes be stopped. The effects of the atomic bomb and its research are the very foundations of this world, AI has contemporary magnitude, and could alter the game even more significantly. It is my guess that paradigm shifts in human consciousness have quite unpredictable effects.