3 ms·
How do you feel about the possibility of a HAL-9000 like scenario where oversights in the design of the program lead to it exhibiting unpredictable and dangerou
by adrianm 11y ago
How do you feel about the possibility of a HAL-9000 like scenario where oversights in the design of the program lead to it exhibiting unpredictable and dangerous decision making behavior under certain conditions?
The novel of '2001' goes into deeper detail than the film regarding the context in which HAL was programmed. To summarize, his mission was to deliberately withhold certain information about the mission from the crew due to the likelihood of xenophobic sentiment developing amongst them if they knew they were traveling to the site of extra terrestrial intelligence. In a sense, he was fulfilling the parameters of his mission in a way which was contrary to the purpose of it. I always interpreted this story as showing us that the more complex a system, the less accurate our predictions of its failure modes are going to be.
I do not think absolute destruction of humanity is likely either, but I also don't think our first steps with AI on this level will be without the loss of life either (of both human and machine). Keep in mind, the same was and is true of space exploration and is essentially an inherent danger of working with the unknown and on the frontier of our knowledge.
- jjaredsimpson 11y agoI would say any intelligent entity must be unpredictable to actually be considered intelligent. Any person on the street could kill me for no reason whatsoever. But I don't live my life like that is a possibility. I also don't think about future superintelligences in that way either. I don't worry that they will be holocaust machines teetering on the brink of genocide.