4 ms·

Something can go wrong. The outcome of the thing going wrong is literally the worst thing that can happen to the entire humanity. Therefore we should do every

Something can go wrong. The outcome of the thing going wrong is literally the worst thing that can happen to the entire humanity. Therefore we should do everything in our power to avoid doing that thing untill we can eliminate the risk.

23B1 4y ago
By that rather sloppy logic, we should be urgently working on capping the Yellowstone supervolcano and constructing impenetrable meteorite defense shields. I acknowledge that it's a possibility; that's not the argument at hand.
- moonchrome 4y ago
  Except there's concrete evidence that the execution of safety is subpar due to market pressures to release and monetize (Google panic release). There's also evidence that progress is exceeding people's expectations and we might not even realize what we have is dangerous untill too late. If we had a probable collision trajectory of a meteor or signs of Yellowstone activating then yeah those would be good ideas.
  - 23B1 4y ago
    The chain of 'logic' that leads to the outcome you propose is bonkers, though. Here's Elizer's logic from his recent polemic in Time https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... "A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing." This is Ridley Scott, not Aristotle.
- flangola7 4y ago
  If we saw Yellowstone spawning a sequence of ever massively larger quakes and ground disturbance, or if we observed a massive distant asteroid continually closing in on Earth with higher and higher collision probability, then yes we should be pulling out all the stops and reconfiguring society to do whatever is necessary to save the species.
  - 23B1 4y ago
    Agreed! But there is no evidence of the eventuality described by these AI doomers. The same hyperbole was employed by Ehrlich in the late 60s when we were talking about the population bomb, or Nevil Shute in 1945 on the nuclear apocalypse - both scenarios seem to be increasingly unlikely. I think we can both agree on the fallibility of human beings – but the leaps of logic require such a tenuous series of gymnastic leaps that the polemic is not only hard to take seriously, but indeed can cloud our ability to perceive actual threats! From Eliezer's recent article in TIME: "A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing." (https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...) C'mon.
    - flangola7 4y ago
      The fact that we got lucky in the past is an opportunity to make sure we don't have to rely on luck in the future, not to continue on as normal and keep playing Russian Roulette. If you see fresh leopard tracks maybe it hasn't smelled you yet, but you shouldn't wait to start readying your rifle ready until it making a pounce at you. I also would definitely not call the risk of nuclear war decreasing, with the Ukraine conflict ongoing and multiple nuclear powers experiencing growing instability. The number of friends I have who are knowledgeable about surviving a nuclear strike and/or own a CBRN mask went from 25% to 90% last year.

23B1 4y ago

By that rather sloppy logic, we should be urgently working on capping the Yellowstone supervolcano and constructing impenetrable meteorite defense shields. I acknowledge that it's a possibility; that's not the argument at hand.

moonchrome 4y ago
Except there's concrete evidence that the execution of safety is subpar due to market pressures to release and monetize (Google panic release). There's also evidence that progress is exceeding people's expectations and we might not even realize what we have is dangerous untill too late. If we had a probable collision trajectory of a meteor or signs of Yellowstone activating then yeah those would be good ideas.
- 23B1 4y ago
  The chain of 'logic' that leads to the outcome you propose is bonkers, though. Here's Elizer's logic from his recent polemic in Time https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... "A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing." This is Ridley Scott, not Aristotle.
flangola7 4y ago
If we saw Yellowstone spawning a sequence of ever massively larger quakes and ground disturbance, or if we observed a massive distant asteroid continually closing in on Earth with higher and higher collision probability, then yes we should be pulling out all the stops and reconfiguring society to do whatever is necessary to save the species.
- 23B1 4y ago
  Agreed! But there is no evidence of the eventuality described by these AI doomers. The same hyperbole was employed by Ehrlich in the late 60s when we were talking about the population bomb, or Nevil Shute in 1945 on the nuclear apocalypse - both scenarios seem to be increasingly unlikely. I think we can both agree on the fallibility of human beings – but the leaps of logic require such a tenuous series of gymnastic leaps that the polemic is not only hard to take seriously, but indeed can cloud our ability to perceive actual threats! From Eliezer's recent article in TIME: "A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing." (https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...) C'mon.
  - flangola7 4y ago
    The fact that we got lucky in the past is an opportunity to make sure we don't have to rely on luck in the future, not to continue on as normal and keep playing Russian Roulette. If you see fresh leopard tracks maybe it hasn't smelled you yet, but you shouldn't wait to start readying your rifle ready until it making a pounce at you. I also would definitely not call the risk of nuclear war decreasing, with the Ukraine conflict ongoing and multiple nuclear powers experiencing growing instability. The number of friends I have who are knowledgeable about surviving a nuclear strike and/or own a CBRN mask went from 25% to 90% last year.