4 ms·

> Though analogies between artificial intelligence and nuclear weapons can be overstretched, the way this concern went from idle speculation to serious concern

> Though analogies between artificial intelligence and nuclear weapons can be overstretched, the way this concern went from idle speculation to serious concern can teach us lessons regarding fears swirling around today's emerging technologies. Sounds like an overstretched analogy. There is no such principle which says that just because a theoretical concern in one area turned out to not be a problem in practice, all theoretical concerns will not be a problem in practice.

scotty79 3y ago
I think the analogy is apt. Same way the atmosphere is not a bag of nuclear gunpowder, our crappy, slow data centers (and even sand on the beaches that could be turned into silicon) are nothing even close to something being capable of powering humanity devastating AI singularity. It's not algorithms that stand in the way, it's crap computational capacity of our local physical reality.
- flangola7 3y ago
  We don't know that. Humanity exterminating AI may only need a kilowatt of compute with the right algo
  - random_mutex 3y ago
    And an some specimens from our species could do it with less
  - scotty79 3y ago
    And people thought one nuke might start chain reaction of Earth atmosphere. It turned out it's not possible. Single human runs on 100W with algo and hardware optimized for more than million years. I think at some point we can do better with sand and electricity, but it's still far away. And AI research is terrible with algos. Modern miracle GPT is plain old multilayer neural network with some forward shortcuts and some parts hard coded according to ad-hoc "attention" idea. And learning algorithm is more than half a century old. The reason GPT performs is just the RAM and compute. We have every reason to be sceptical about AI apocalypse.
    - flangola7 3y ago
      When it comes to safety you play on the safe side, not the dangerous one. I don't share the philosophy of amateur submariners and I've had my fill of their hollow and reckless arguments. A handful of humans (a few hundred watts of compute) succeeded at devastating the world from 1939-1945. AI already knows more domains of knowledge, is faster, and does not require 20 years to spin up each new instance. Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea.
      - scotty79 3y ago
        > I don't share the philosophy of amateur submariners That makes perfect sense in 2023. Not so much in seventeen hundreds, or whenever people first started to attempting it. > A handful of humans (a few hundred watts of compute) succeeded at devastating the world from 1939-1945. I wouldn't really count it towards how powerful a person is. Literally billions of people amount to almost nothing their whole life. Wars were results of historical processes. People are nothing than a match under the barrel of gunpowder filled with economical and societal forces. If there are such forces that could end us what's the difference if the match will be biological or silicon? > Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea. Imagined cave troll.
- circuit10 3y ago
  We don’t know for sure how much computational capacity it takes or how quickly we will create more. It seems like a good idea to be cautious
  - scotty79 3y ago
    Like we didn't know for sure if the atmosphere gona burn in self sustaining nuclear fire. I know it's not the same because AI research is closer to social sciences than to physics but still, there are parallels.
- manderley 3y ago
  That's quite the logical leap.
  - scotty79 3y ago
    Sure, bo no more fantastic than thinking we'll get extinct by suprise AI singularity.

scotty79 3y ago

I think the analogy is apt. Same way the atmosphere is not a bag of nuclear gunpowder, our crappy, slow data centers (and even sand on the beaches that could be turned into silicon) are nothing even close to something being capable of powering humanity devastating AI singularity. It's not algorithms that stand in the way, it's crap computational capacity of our local physical reality.

flangola7 3y ago
We don't know that. Humanity exterminating AI may only need a kilowatt of compute with the right algo
- random_mutex 3y ago
  And an some specimens from our species could do it with less
- scotty79 3y ago
  And people thought one nuke might start chain reaction of Earth atmosphere. It turned out it's not possible. Single human runs on 100W with algo and hardware optimized for more than million years. I think at some point we can do better with sand and electricity, but it's still far away. And AI research is terrible with algos. Modern miracle GPT is plain old multilayer neural network with some forward shortcuts and some parts hard coded according to ad-hoc "attention" idea. And learning algorithm is more than half a century old. The reason GPT performs is just the RAM and compute. We have every reason to be sceptical about AI apocalypse.
  - flangola7 3y ago
    When it comes to safety you play on the safe side, not the dangerous one. I don't share the philosophy of amateur submariners and I've had my fill of their hollow and reckless arguments. A handful of humans (a few hundred watts of compute) succeeded at devastating the world from 1939-1945. AI already knows more domains of knowledge, is faster, and does not require 20 years to spin up each new instance. Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea.
    - scotty79 3y ago
      > I don't share the philosophy of amateur submariners That makes perfect sense in 2023. Not so much in seventeen hundreds, or whenever people first started to attempting it. > A handful of humans (a few hundred watts of compute) succeeded at devastating the world from 1939-1945. I wouldn't really count it towards how powerful a person is. Literally billions of people amount to almost nothing their whole life. Wars were results of historical processes. People are nothing than a match under the barrel of gunpowder filled with economical and societal forces. If there are such forces that could end us what's the difference if the match will be biological or silicon? > Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea. Imagined cave troll.
circuit10 3y ago
We don’t know for sure how much computational capacity it takes or how quickly we will create more. It seems like a good idea to be cautious
- scotty79 3y ago
  Like we didn't know for sure if the atmosphere gona burn in self sustaining nuclear fire. I know it's not the same because AI research is closer to social sciences than to physics but still, there are parallels.
manderley 3y ago
That's quite the logical leap.
- scotty79 3y ago
  Sure, bo no more fantastic than thinking we'll get extinct by suprise AI singularity.