3 ms·
"And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations" That's not necessarily true at all. Let's say w
by crusso 9y ago
"And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations"
That's not necessarily true at all. Let's say we come up with a way to model neurons in silicon like a very complex neural net to the point of emergent consciousness. We might not be able to look into memory location 0xFFA79348 and get the "human value" int to see if it is high enough.
It could very well be that we don't really understand the weighting in the net or the implications of those weights because it's too complicated. Even current neural nets of non-trivial complexity can be impenetrable to evaluate outside of the net itself.
- Retra 9y agoTransparency doesn't require that you be able to read its memory directly, just that it is able to sufficiently justify the decisions that it makes.
- crusso 9y agoLike we can justify all decisions made by super genius people who are extraordinarily good at concealing their activities when their cognitive processes and abilities on the Internet and with other forms of communication would be beyond our comprehension? I think that you overestimate our ability to understand what we can create. Even Deep Neural Net researchers consider the neural nets to be a black box that they don't really understand. Further... unlike human beings who are locked into their genetics and their neuronal pathways - an AI will have the instant ability to improve itself. It will be able to rewrite its own software faster than we can comprehend. In the hours from the time the AI would be created until a researcher even might realize what he had created, the AI will effectively have had years or tens of years of contemplation time and self-enhancement time. If the researcher goes to sleep with plans to work with the new AI the next morning, the AI will effectively have hundreds of years of time to plan and improve itself. If it has any kind of Internet access, it will be able to take over systems around the world, possibly parallelizing its own cognitive abilities and possibly backing up its consciousness in case the researcher decided to eventually wipe it. The genie will be out of the bottle so fast that human beings will almost assuredly be caught flat-footed. I always shake my head when I see sci-fi shows where the humans (Stargate, Star Trek, etc.) come up against advanced AI opponents. It's only through control of the script and plot contrivances that the humans have a chance. People just won't be able to compare to thinking and self-modifying AI. https://www.ted.com/talks/sam_harris_can_we_build_ai_without_losing_control_over_it https://www.ted.com/talks/sam_harris_can_we_build_ai_without...