4 ms·
Might just be BSing, but the stock market is technically already a superintelligent agent with the implicit goal of investing funds in profitable ventures. The
by colorincorrect 7y ago
Might just be BSing, but the stock market is technically already a superintelligent agent with the implicit goal of investing funds in profitable ventures. The question is: should we fear the stock market?
- thelazydogsback 7y ago>should we fear the stock market Yes
- AnimalMuppet 7y agoPerhaps, but why? Because it's too smart? Or because it's too stupid?
- colorincorrect 7y agoprobably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on the question of this "alignment problem". Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: https://www.youtube.com/watch?v=EUjc1WuyPT8 https://www.youtube.com/watch?v=EUjc1WuyPT8
- sliken 7y agoBecause endlessly optimizing profit will result in the destruction of the environment and the loss of most jobs.
- Tenoke 7y agoBecause it's too smart but due to misaligned incentives it sometimes does things that harm us.
- lern_too_spel 7y agoI came to post a very similar point. There already exist superintelligences in the form of organizations of humans, and the misspecification of their objectives has already caused tremendous disasters. Think East India Company and its subjugation of entire countries in pursuit of tea profits or CCP and the horrors of the Great Leap Forward and the Cultural Revolution. Until we have built a framework to stop these things from happening again, what hope do we have against intelligences that are to human organizations what human organizations are to individuals?