3 ms·

There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly

There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly massively increasing that risk while writing articles that say that they feel kind of bad about the risk. o_O I'm not very worried about AI safety. But if I was, it'd be hard to think of groups like OpenAI as working on the same side.

nerdponx 9y ago
"Corporate demagoguery"
p-funk 9y ago
Personally, I'd rather have the people working on AI be the ones who are aware of the potential theoretical pitfalls, rather than someone who is not aware of any pitfalls and rushes full steam ahead.
AndrewKemendo 9y ago
It's honestly an impossible goal to make AGI safe for humans. If an AGI cannot be independent in it's goals, meaning it could potentially harm humans, then it's not really AGI.
arikr 9y ago
> But if I was, it'd be hard to think of groups like OpenAI as working on the same side. How so?
- cjbprime 9y ago
  Because I would think that they're currently making unsafe AI more likely rather than less likely, contradicting their stated goal.
  - tlb 9y ago
    How so? What should they do differently?
    - astrofinch 9y ago
      http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/ http://benjaminrosshoffman.com/openai-makes-humanity-less-sa... Personally, I think making AI tech broadly available could be a bad idea if AI tech changes warfare so that offense works better than defense: https://news.ycombinator.com/item?id=10721621 https://news.ycombinator.com/item?id=10721621 (This is already true, but I think it's likely that further technological development will make it even more true.)
erasmuse 9y ago
Yes. We are irrational in rearing and educating human children for example. Sandboxing. Is it like this? http://www.ptaugustacsc.sa.edu.au/images/sandpit.jpg http://www.ptaugustacsc.sa.edu.au/images/sandpit.jpg Or like this? https://s3.amazonaws.com/audreywatters/factory_counting.jpg https://s3.amazonaws.com/audreywatters/factory_counting.jpg Yet such things are done by apparently good and nice people trying to work in the best interests of humanity. So how can we expect to somehow get it right in the case of an AGI without turning him into a resentful demon?

nerdponx 9y ago

"Corporate demagoguery"

p-funk 9y ago

Personally, I'd rather have the people working on AI be the ones who are aware of the potential theoretical pitfalls, rather than someone who is not aware of any pitfalls and rushes full steam ahead.

AndrewKemendo 9y ago

It's honestly an impossible goal to make AGI safe for humans. If an AGI cannot be independent in it's goals, meaning it could potentially harm humans, then it's not really AGI.

arikr 9y ago

> But if I was, it'd be hard to think of groups like OpenAI as working on the same side. How so?

cjbprime 9y ago
Because I would think that they're currently making unsafe AI more likely rather than less likely, contradicting their stated goal.
- tlb 9y ago
  How so? What should they do differently?
  - astrofinch 9y ago
    http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/ http://benjaminrosshoffman.com/openai-makes-humanity-less-sa... Personally, I think making AI tech broadly available could be a bad idea if AI tech changes warfare so that offense works better than defense: https://news.ycombinator.com/item?id=10721621 https://news.ycombinator.com/item?id=10721621 (This is already true, but I think it's likely that further technological development will make it even more true.)

erasmuse 9y ago

Yes. We are irrational in rearing and educating human children for example. Sandboxing. Is it like this? http://www.ptaugustacsc.sa.edu.au/images/sandpit.jpg http://www.ptaugustacsc.sa.edu.au/images/sandpit.jpg Or like this? https://s3.amazonaws.com/audreywatters/factory_counting.jpg https://s3.amazonaws.com/audreywatters/factory_counting.jpg Yet such things are done by apparently good and nice people trying to work in the best interests of humanity. So how can we expect to somehow get it right in the case of an AGI without turning him into a resentful demon?