3 ms·
> To prevent the system from doing anything harmful or unethical, it may be necessary to challenge it to explain the logic for a particular action. That logic m
by bko 9y ago
> To prevent the system from doing anything harmful or unethical, it may be necessary to challenge it to explain the logic for a particular action. That logic might be too complex for a person to comprehend, so the researchers suggest having another AI debate the wisdom of the action with the first system, using natural language, while the person observes
...
> Having AI programs argue with one another requires more sophisticated technology than exists currently. So thus far, the OpenAI researchers have only explored the idea with a couple of extremely simple examples. One involves two AI systems trying to convince an observer about a hidden character by slowly revealing individual pixels.
How is this even remotely linked to two programs arguing or explaining reasoning? It's one network being trained to change pixels in an image in order to make the detecting network perform worse.
Why do so many articles anthropomorphism artificial intelligence? These networks are assigned to do a certain task given a certain reward. There is no motive apart from reducing the loss function provided. The reasoning or rationale is that whatever decision the network is making is serving to reduce the error. Whether this has some deeper reasoning that humans would also use is a separate question but to attribute human style reasoning to the network is a stretch.
- kowdermeister 9y ago> Why do so many articles anthropomorphism artificial intelligence? Clicks. People click on things they can relate to. "We did some matrix multiplication that resulted in more insights" probably won't cut it :)
- vidarh 9y agoI actually think the first problem occurs already in the first part you quoted: Having had conversations about thorny subjects in front of children, or even adults sometimes, without the third party understanding what you're actually saying is not that hard. You poke and prod at the boundaries of which words you can use until you find terms that relate to your shared experience but sounds sufficiently harmless to the observer, and then you talk straight past them. Policing what someone talks about only really works if you can be certain that the parties are not potentially prepared to collude. If the first AI starts with something akin to "psst... listen, I need to explain something to you in confidence.. [insert reasons why the human observer can't be trusted]", what's to say they won't occasionally find a sympathetic virtual ear.
- deleted 8y ago[deleted]
- nmca 9y agoThis is about the Debate paper, and is an idea that is being taken quite seriously - https://arxiv.org/abs/1805.00899 https://arxiv.org/abs/1805.00899 The idea is that the given example (explained in more detail in the paper) is the first step towards understanding how such a system might work.