3 ms·
these AIs have already demonstrated limited tendency to strongly prefer not being shut down (they already have the ability to lie to humans to achieve goals and
by swyx 4y ago
these AIs have already demonstrated limited tendency to strongly prefer not being shut down (they already have the ability to lie to humans to achieve goals and we celebrate this lol). giving them the ability to self replicate means we lose that ability to pull the plug.
giving them ability to self modify means humanity will lose control of ai on any long enough timescale.
- nradov 4y agoThe fact that you would use words like "prefer", "lie", and "celebrate" in this context indicates that you are deeply ignorant about the underlying technology. Your claims are wild, unsupported conjecture and cannot be taken seriously.
- thfuran 4y agoWhile I agree with you, whether it is able to have preferences or feelings doesn't actually matter in this context. A simulated pzombie may be morally distinct from a person, but the consequences of its actions are unaffected by the distinction.
- Karunamon 4y agoRemember the erratic behavior demonstrated by "Sydney", the Bing GPT bot? Whether the model has the "cognitive" ability to "prefer" something is not relevant. The model has the ability to submit search engine queries now; this is a read only operation for now. Once we give the bot write ability, erratic behavior has the potential to cause real world damage. Whether it is an actual "preference" or something that only looks like one based on the output is purely academic at that point.
- haswell 4y agoI’ve been ranting against the anthropomorphization of these new AI tools pretty consistently, but I think the fact that these tools only appear to possess certain characteristics is all the more reason that we should not be hooking these things up to tools that give them access to the real world. If a language model “concocts” some “scheme” and executes it by interacting with real systems in ways that have real world impact, it doesn’t much matter if the whole thing is a hallucination. Put another way, would you feel comfortable letting a hallucination machine running on a supercomputer have access to the real world? It doesn’t have to truly “know” anything for the impact of the resulting actions to matter.
- travisjungroth 4y agoI’m glad to see someone flip to this view. There will be holdouts forever. “He’s trying to kill you!” “Well, actually, the Terminator is only a predictive model. It can’t try to…” Terminator shoots everyone the end.
- haswell 4y agoTo be clear I think that the anti-anthropomorphization conversation is still an important one for many other reasons. It’s just not a very useful factor when examining risk/harm, or at least not a very good reason to claim there is none. It comes quickly back to the forefront when exploring the sociological side and how people’s beliefs about what the robot is will be just as impactful, e.g. someone who believes the machine is reasonable is more likely to grant it more power by imagining that it reasons the way humans do. It’s gonna be an interesting decade.
- nradov 4y agoSure, I would feel comfortable letting AI tools have access to the real world. Why not? Hallucination is a human concept which doesn't apply here. But we already allow humans who experience hallucinations and other mental disorders to have access to the real world; they are only locked away if they commit crimes or meet strict criteria for civil confinement. This very HN thread is already full of ignorant, delusional people posting total nonsense. This is not a serious problem. How would an AI tool be any worse? Some of the commenters here appear to be concerned that an AI tool could call an API that would cause damage. That is a misplaced concern because such vulnerabilities will inevitably be exploited by malicious humans anyway, so it hardly matters if an AI tool finds them first. And AI tools are owned by people or corporations who can be held legally responsible for any damage. Like if your dog escapes from your yard and bites someone then you're going to be in trouble.
- Turing_Machine 4y agoIt doesn't matter whether they REALLY "prefer" or "celebrate". If they behave in a manner consistent with those descriptors, it's a distinction without a difference.
- layer8 4y agoYou mean, like the underlying technology of biological brains? It’s all just differential equations, physics can’t really “think” or “have feelings”.