5 ms·

> Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. Did they? I've never heard about

> Electricity, the Industrial Revolution, the internet, gene editing/bionengineering also came with unknown existential risks. Did they? I've never heard about electricity creating fears about human extinction. The industrial revolution does pose an existential risk: climate change. Did bioengineering garner worries about existential risk? It certainly had and has a lot of people worried about various risks. And then lots of countries banned human cloning: https://en.wikipedia.org/wiki/Human_cloning#Current_law https://en.wikipedia.org/wiki/Human_cloning#Current_law > Evidence? This has been stated in sci-fi books from before I was born Yes, many sci-fi authors of the past were very forward-thinking and predicted future technologies... that's sort of the point. > and I don’t see any proof that we’re building something remotely close to this Again, this is a fundamental misunderstanding of the risk. 1. We are building AI. AI is possible. 2. We are advancing AI. AI advancement is possible. 3. We have not built superhuman AGI. But superhuman AGI is probably possible. The fact that AGI doesn't exist today is frequently argued as a reason we won't have it in the near future, but it's a non-sequitur. We will definitely have more advanced AI in 5 years than we do today, and we can't say what that AI will be capable of. Therefore, it's possible it will be AGI.

dragonwriter 3y ago
> The industrial revolution does pose an existential risk: climate change. And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. It’s true that once we understood the concrete mechanisms and could evaluate alternatives, not mitigating the risk has been irresponsible, but that was…significantly later than the Industrial Revolution itself.
- digging 3y ago
  > And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change or any other existential threat. It might have made us more vulnerable to natural pandemics... but it also would have slowed the emergence of natural pandemics and virtually prohibited artificial pandemics.
  - dragonwriter 3y ago
    > Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change Neither do I, but then, 100 years ago doesn't clearly meet the description, anyhow. The discovery of the greenhouse effect and CO2’s role in it was after not very long after the dates usually used for the end of the (first) industrial revolution (which is usually what is meant without modifiers, not the second—fourth industrial revolutions) and well over 100 years ago.
    - digging 3y ago
      What are you even arguing now? Replace "100 years" with whatever timeframe is perfect for you and I'll make the same argument. Please respond to that instead of nitpicking.
      - dragonwriter 3y ago
        > Replace "100 years" with whatever timeframe is perfect for you and I'll make the same argument At any time before we had the information to understand thr mechanism of problems and evaluate mitigations, its very unlikely that we would choose tbe right mitigations. Sure, if we choose thr mitigations that we would choose based on information today before we had that information, that would be great, but its at least as likely that any attempt to mitigate risk of unknown mechanism vefore we had that infornation would have done exactly the wrong thing. Basically, “Well, I have information today by which I can design a policy which would have been beneficial if implemented before that information was available” is not an argument that people seeking to mitigate a vaguely imagined risk without any understanding of its mechanism would have the means to design a productive intervention.
haldujai 3y ago
> 3. We have not built superhuman AGI. But superhuman AGI is probably possible. "Dumb" AGI is yet to be proven theoretically possible let alone superhuman.
- digging 3y ago
  It hasn't been proven possible, but it's reasonable that it's possible. There's no theory I know of saying that it's unlikely to be possible. Compared to FTL travel, for example, which we have theory enough to state is not likely to be possible.

dragonwriter 3y ago

> The industrial revolution does pose an existential risk: climate change. And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. It’s true that once we understood the concrete mechanisms and could evaluate alternatives, not mitigating the risk has been irresponsible, but that was…significantly later than the Industrial Revolution itself.

digging 3y ago
> And yet, until much later when we understood the mechanisms and could evaluate mitigations, any restriction on industrialization based on purely speculative “existential risk” that we could neither adequately explain nor provide a factually-grounded framework for evaluating relative risks of alternatives would mostly likely not have made things better, and could have made them (and even the real existential risk, as well as experienced conditions at the time) much worse. Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change or any other existential threat. It might have made us more vulnerable to natural pandemics... but it also would have slowed the emergence of natural pandemics and virtually prohibited artificial pandemics.
- dragonwriter 3y ago
  > Really? I don't agree that strictly limiting plastic production and carbon emissions 100 years ago would have had zero or negative effect on the timeline of apocalyptic climate change Neither do I, but then, 100 years ago doesn't clearly meet the description, anyhow. The discovery of the greenhouse effect and CO2’s role in it was after not very long after the dates usually used for the end of the (first) industrial revolution (which is usually what is meant without modifiers, not the second—fourth industrial revolutions) and well over 100 years ago.
  - digging 3y ago
    What are you even arguing now? Replace "100 years" with whatever timeframe is perfect for you and I'll make the same argument. Please respond to that instead of nitpicking.
    - dragonwriter 3y ago
      > Replace "100 years" with whatever timeframe is perfect for you and I'll make the same argument At any time before we had the information to understand thr mechanism of problems and evaluate mitigations, its very unlikely that we would choose tbe right mitigations. Sure, if we choose thr mitigations that we would choose based on information today before we had that information, that would be great, but its at least as likely that any attempt to mitigate risk of unknown mechanism vefore we had that infornation would have done exactly the wrong thing. Basically, “Well, I have information today by which I can design a policy which would have been beneficial if implemented before that information was available” is not an argument that people seeking to mitigate a vaguely imagined risk without any understanding of its mechanism would have the means to design a productive intervention.

haldujai 3y ago

> 3. We have not built superhuman AGI. But superhuman AGI is probably possible. "Dumb" AGI is yet to be proven theoretically possible let alone superhuman.

digging 3y ago
It hasn't been proven possible, but it's reasonable that it's possible. There's no theory I know of saying that it's unlikely to be possible. Compared to FTL travel, for example, which we have theory enough to state is not likely to be possible.