3 ms·

We don’t know where either of these technologies will be in 20 years time. You seem convinced that unknown biotech advancements won’t be that dangerous but unkn

by bnralt 4y ago

We don’t know where either of these technologies will be in 20 years time. You seem convinced that unknown biotech advancements won’t be that dangerous but unknown AI advancements could be, but there’s no reason to believe that someone who thinks unknown biotech advancements are more dangerous is wrong. In fact, we actually have examples of a new species being able to completely devastate other species, but self-replicating technology that requires human infrastructure becoming self-sufficient is still a dream. Neither do we have any example of hyper intelligence being able to completely dominate lower forms of intelligence. A lone human without societal support might be able to leverage some of its intelligence against a great white shark, but it’s going to have limited success. An immobilized person is going to have no success. It certainly wouldn’t lead to the extinction of great white sharks as a whole. AI doomsday/alignment/etc. folk seem to start with the conclusion that AI tech is inherently more dangerous than other tech, and then work their way backwards from there. But there’s little evidence that this is true. So far, the most dangerous tech has come from nuclear physics.

ethanbond 4y ago
Are you just choosing to ignore the actual contents of my responses? Both nuclear and biotech are highly, highly regulated spaces. They are such because their power for good seems to rise in direct proportion to their power for bad. You are the one making the claim that AI doesn’t land on the same trend line as most other technology. Sure, AI doesn’t seem able to self-replicate yet. Want to know how we’re testing that? By giving it money and directives to self-replicate on shared cloud networks. This is like testing a new helmet by loading modern human life[0] into one and catapulting it into a brick wall at Mach 3. If that seems okay, now do it again at Mach 4. If that seems okay, do it again at Mach 5. I have seen no remotely believable explanation as to why this is an inaccurate description of what we’re doing. Sure, we might get a really great helmet out of it. Maybe the best ever. Maybe one that saves lives. But what signal do we have that it has reached its appropriate potential other than it shattering? [0] This likely isn’t (at this stage) equivalent to loading all of humanity itself into your untested helmet, but pretty close to everything we care about, which isn’t much of a mitigation as far as I’m concerned.
- bnralt 4y ago
  > Are you just choosing to ignore the actual contents of my responses? Both nuclear and biotech are highly, highly regulated spaces. Apologies, I have limited time and tried to focus on what I felt were your stronger arguments. But if you want me to address current regulations, I can. We have regulations now on technology that 1. exists and 2. we know can be dangerous. I hope most people will understand why a technology that 1. doesn’t exist and 2. that we don’t know will be dangerous if it ever does exist doesn’t have any comparable regulation. Yes, we have regulation on nuclear power now. As far as I know, we didn’t have any regulation restricting Niels Bohr’s research in the 1920’s. Correct me if I’m wrong. If we want AI to be treated like other tech, we’d wait until an actual danger presented itself, and then apply appropriate regulation to address that danger.
  - ethanbond 4y ago
    We have no idea if we are Niels Bohr in the 1920s or Oppenheimer on July 15th 1945. We have no idea what the distance is between those two points, but again, the trend line of technology (and especially of AI tech) is that we should expect it to be a lot shorter than 20 years. If you have good reason to believe we're in the 1920s and not 1945, I'm open to hearing it. Additionally, it's not exactly self-evident that we shouldn't have stopped nuclear research at a more nascent level, and even if we accept that, it's not evident that'd justify introducing another looming technology catastrophe. By the time of the first nuclear detonation, yes, there was immense control already being exerted on all of the relevant ingredients. Do you disagree with the claim that AI technology, on its current trajectory, (i.e. not necessarily this generation of tech) has at least a small chance of yielding at least an extremely bad outcome?
    - bnralt 4y ago
      The first laws regulating nuclear energy were signed about a year after the atomic bombings, no? As far as I know, the first nuclear fission experiments were completely unregulated.
      - 4y ago
mrob 4y ago
Human capabilities are greatly limited by other humans, and by weaknesses imposed by biology. The first AGI will have no peer competitors, and no biological weaknesses. A single intelligent human, with unrestricted access to all the worlds resources, with no tiredness or other weaknesses of the body, with perfect motivation and focus, and with the ability to perfectly clone themself, would undoubtedly be able to drive great white sharks to extinction. And that's just with human-level intelligence. Nuclear bombs are highly unlikely to drive humans to extinction because nuclear bombs could never make improved nuclear bombs.

by bnralt 4y ago

ethanbond 4y ago

Are you just choosing to ignore the actual contents of my responses? Both nuclear and biotech are highly, highly regulated spaces. They are such because their power for good seems to rise in direct proportion to their power for bad. You are the one making the claim that AI doesn’t land on the same trend line as most other technology. Sure, AI doesn’t seem able to self-replicate yet. Want to know how we’re testing that? By giving it money and directives to self-replicate on shared cloud networks. This is like testing a new helmet by loading modern human life[0] into one and catapulting it into a brick wall at Mach 3. If that seems okay, now do it again at Mach 4. If that seems okay, do it again at Mach 5. I have seen no remotely believable explanation as to why this is an inaccurate description of what we’re doing. Sure, we might get a really great helmet out of it. Maybe the best ever. Maybe one that saves lives. But what signal do we have that it has reached its appropriate potential other than it shattering? [0] This likely isn’t (at this stage) equivalent to loading all of humanity itself into your untested helmet, but pretty close to everything we care about, which isn’t much of a mitigation as far as I’m concerned.

bnralt 4y ago
> Are you just choosing to ignore the actual contents of my responses? Both nuclear and biotech are highly, highly regulated spaces. Apologies, I have limited time and tried to focus on what I felt were your stronger arguments. But if you want me to address current regulations, I can. We have regulations now on technology that 1. exists and 2. we know can be dangerous. I hope most people will understand why a technology that 1. doesn’t exist and 2. that we don’t know will be dangerous if it ever does exist doesn’t have any comparable regulation. Yes, we have regulation on nuclear power now. As far as I know, we didn’t have any regulation restricting Niels Bohr’s research in the 1920’s. Correct me if I’m wrong. If we want AI to be treated like other tech, we’d wait until an actual danger presented itself, and then apply appropriate regulation to address that danger.
- ethanbond 4y ago
  We have no idea if we are Niels Bohr in the 1920s or Oppenheimer on July 15th 1945. We have no idea what the distance is between those two points, but again, the trend line of technology (and especially of AI tech) is that we should expect it to be a lot shorter than 20 years. If you have good reason to believe we're in the 1920s and not 1945, I'm open to hearing it. Additionally, it's not exactly self-evident that we shouldn't have stopped nuclear research at a more nascent level, and even if we accept that, it's not evident that'd justify introducing another looming technology catastrophe. By the time of the first nuclear detonation, yes, there was immense control already being exerted on all of the relevant ingredients. Do you disagree with the claim that AI technology, on its current trajectory, (i.e. not necessarily this generation of tech) has at least a small chance of yielding at least an extremely bad outcome?
  - bnralt 4y ago
    The first laws regulating nuclear energy were signed about a year after the atomic bombings, no? As far as I know, the first nuclear fission experiments were completely unregulated.
    - 4y ago

mrob 4y ago

Human capabilities are greatly limited by other humans, and by weaknesses imposed by biology. The first AGI will have no peer competitors, and no biological weaknesses. A single intelligent human, with unrestricted access to all the worlds resources, with no tiredness or other weaknesses of the body, with perfect motivation and focus, and with the ability to perfectly clone themself, would undoubtedly be able to drive great white sharks to extinction. And that's just with human-level intelligence. Nuclear bombs are highly unlikely to drive humans to extinction because nuclear bombs could never make improved nuclear bombs.