6 ms·

I also think people get too caught up on the expected time-frame. The large majority of active AI researchers think that AGI will happen at some point in the (

I also think people get too caught up on the expected time-frame. The large majority of active AI researchers think that AGI will happen at some point in the (sub-1000 year) future. When exactly isn't a very interesting question, relatively speaking. We're going to have to deal with AGI eventually, and whether it's going to do what we want is not something that can be theoretically predicted from the armchair.

fossuser 6y ago
Yeah - and people are famously bad about predicting these events: https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no-fire-alarm-for-artificial-general-intelligence https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no... If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help, but since it's hard to know that seems reasonable for people to be working on it now? It's also possible to figure things out before the necessary tech is possible (I think a lot of CS papers in the 60s became more interesting later when the hardware caught up to be more useful, arguably the recent NN stuff falls into this category too).
- walleeee 6y ago
  > If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help Circular justification of technological development is the reason unfriendly AGI is a threat in the first place (and also the reason we are unlikely to see it realized, imo; the internal combustion engine for instance poses existential risks not only to people but to the possibility of machine intelligence). Technology is not a monolith, forms can and do preclude other forms
- ketzo 6y ago
  Just wanna say I absolutely loved reading that blog post, thanks for the link.
  - fighterpilot 6y ago
    Check out the blog post in the comment that's parallel to /u/fossuser above. It's also great.
deleted 6y ago
[deleted]
rakhodorkovsky 6y ago
It's the critical question. The time frame is critical for reasoning about alignment and what to do now. If you expect AGI to happen tomorrow, you throw up your hands in despair or exhilaration, if you expect it in a hundred years, you might also throw up your hands, but it's a different picture; a hundred years of development will see the development of new technologies transformative in their own right. This excellent blog post hones in on this question: https://fantasticanachronism.com/2021/03/23/two-paths-to-the-future/ https://fantasticanachronism.com/2021/03/23/two-paths-to-the...
- deleted 6y ago
  [deleted]

fossuser 6y ago

Yeah - and people are famously bad about predicting these events: https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no-fire-alarm-for-artificial-general-intelligence https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no... If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help, but since it's hard to know that seems reasonable for people to be working on it now? It's also possible to figure things out before the necessary tech is possible (I think a lot of CS papers in the 60s became more interesting later when the hardware caught up to be more useful, arguably the recent NN stuff falls into this category too).

walleeee 6y ago
> If it is something that's a hundred plus years out then we'll probably need whatever tech develops in the mean time to help Circular justification of technological development is the reason unfriendly AGI is a threat in the first place (and also the reason we are unlikely to see it realized, imo; the internal combustion engine for instance poses existential risks not only to people but to the possibility of machine intelligence). Technology is not a monolith, forms can and do preclude other forms
ketzo 6y ago
Just wanna say I absolutely loved reading that blog post, thanks for the link.
- fighterpilot 6y ago
  Check out the blog post in the comment that's parallel to /u/fossuser above. It's also great.

deleted 6y ago

[deleted]

rakhodorkovsky 6y ago

It's the critical question. The time frame is critical for reasoning about alignment and what to do now. If you expect AGI to happen tomorrow, you throw up your hands in despair or exhilaration, if you expect it in a hundred years, you might also throw up your hands, but it's a different picture; a hundred years of development will see the development of new technologies transformative in their own right. This excellent blog post hones in on this question: https://fantasticanachronism.com/2021/03/23/two-paths-to-the-future/ https://fantasticanachronism.com/2021/03/23/two-paths-to-the...

deleted 6y ago
[deleted]