4 ms·

They have no real models to base their predictions on. It's speculation at the level of "what happens if an immovable meets an unstoppable force". No rigor in i

by Satam 3y ago

They have no real models to base their predictions on. It's speculation at the level of "what happens if an immovable meets an unstoppable force". No rigor in it.

mitthrowaway2 3y ago
What qualifies in your view as a "real model"? Do we need to empirically observe the human race going extinct first?
- Satam 3y ago
  No but we should still be level headed. It's like arguing that because 1) fire kills and 2) spreads, that the whole earth will thus be engulfed in fire and everyone will die. If there's no real model behind this, I can argue just as well that a sufficiently intelligent AGI will be able to protect me from any harm because it's so smart and powerful. And newsflash, if nothing changes, we're all going to die anyway. As it is, our existence is quite limited and it is only through constant creation of unnatural contraptions that we have managed to improve our quality of life.
  - mitthrowaway2 3y ago
    OK. Well, it turns out that the rationale behind AI risk is a bit more sophisticated than that.
    - Satam 3y ago
      Not ok. Show me evidence that the AI panic is driven by such accepted and sophisticated models.
      - mitthrowaway2 3y ago
        Take your pick? https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://intelligence.org/files/AlignmentHardStart.pdf https://intelligence.org/files/AlignmentHardStart.pdf https://www.youtube.com/watch?v=pYXy-A4siMw https://www.youtube.com/watch?v=pYXy-A4siMw https://europepmc.org/article/med/26185241 https://europepmc.org/article/med/26185241 https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang... I could keep going but the reading on this spans for tens of thousands of pages of detailed reasoning and analysis, including mathematical proofs and lab-scale experiments.

by Satam 3y ago

They have no real models to base their predictions on. It's speculation at the level of "what happens if an immovable meets an unstoppable force". No rigor in it.

mitthrowaway2 3y ago

What qualifies in your view as a "real model"? Do we need to empirically observe the human race going extinct first?

Satam 3y ago
No but we should still be level headed. It's like arguing that because 1) fire kills and 2) spreads, that the whole earth will thus be engulfed in fire and everyone will die. If there's no real model behind this, I can argue just as well that a sufficiently intelligent AGI will be able to protect me from any harm because it's so smart and powerful. And newsflash, if nothing changes, we're all going to die anyway. As it is, our existence is quite limited and it is only through constant creation of unnatural contraptions that we have managed to improve our quality of life.
- mitthrowaway2 3y ago
  OK. Well, it turns out that the rationale behind AI risk is a bit more sophisticated than that.
  - Satam 3y ago
    Not ok. Show me evidence that the AI panic is driven by such accepted and sophisticated models.
    - mitthrowaway2 3y ago
      Take your pick? https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://intelligence.org/files/AlignmentHardStart.pdf https://intelligence.org/files/AlignmentHardStart.pdf https://www.youtube.com/watch?v=pYXy-A4siMw https://www.youtube.com/watch?v=pYXy-A4siMw https://europepmc.org/article/med/26185241 https://europepmc.org/article/med/26185241 https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang... I could keep going but the reading on this spans for tens of thousands of pages of detailed reasoning and analysis, including mathematical proofs and lab-scale experiments.