10 ms·
A Robot That Explains Its Actions
- deleted 7y ago[deleted]
- m4r35n357 7y agoSo those 20th century sci-fis were more prescient than we give them credit for!
- reedwolf 7y ago"Exterminate! Annihilate!"
- radarsat1 7y agoI feel like every article on 'explainable AI' pretends (at least in the title) that no one has ever worked on 'explainable AI' before.
- cal97g 7y agoIf you think about it, code is explainable AI - this is the viewpoint many 'ai' startups seem to take at least.
- rzzzt 7y agoBut what if it... lies.
- brutt 7y agoHire an AI-detective to know the truth.
- guramarx11 7y agowhat if the truth we believe is very well "inception-able"? Can we distinguish ideas planted by others(religion, government, pop cultures, movies etc)from our own, let alone "truth" from those AI who "study/spy" on our thoughts to better "serve/enslave" us?
- Atlantium 7y agoWe can only sometimes trust the minimally intelligent brains we already have to deal with. At some point soon we're going to have to discuss whether we're going to go the way of the other extinct species. I think we will. I think that's fine, not being a raging egomaniac, I'd just like a slow pleasant transition, thanks.
- DyslexicAtheist 7y agothe word that sticks out the most is "Trust". We haven't even solved trust with non-AI systems. Half the time I don't even trust myself and it's also not a binary state, and in constant flux. It's such a fuzzy, non-scientific concept that if we would replace every instance of the word "Trust" with "Faith" it would be clear that Trust is an unsolvable problem. "Trusting AI" is the ultimate oxymoron. Snake-oil vendors have also figured this out which is why we have the equally brain dead concept of "trustless"[1]. [1] the ultimate snake-oil is ofc combining the 2 words which gives you: https://trustless.ai https://trustless.ai
- kjhughes 7y agoДоверя́й, но проверя́й (Trust, but verify) Despite your well considered objections, we need to know that our computers are providing useful output at every level, especially at the highest levels which are once again being labeled AI. The concept applies all up and down the stack of systems and organizations: When we don't trust our memory devices, we add error correction. When we don't trust our conventional software, we add QA testing. When we don't trust future governing officials, we add checks and balances. When we don't trust other countries we add verification protocols. "Trusting AI" is not the ultimate oxymoron. It's just one more place we have to design systems to require as little verification as possible, and then provide the means to close the loop and allow verification of what remains. Trust in AI is big now because of the successes of non-symbolic processing. There's a real problem of understanding and explaining answers given by such systems. That the trust problem exists elsewhere is no reason to dismiss its importance here.
- clircle 7y agoIf you want explainable AI you can fit a linear regression.
- TeMPOraL 7y agoIt works for a surprisingly large amounts of problems, including ones where people default to using DNNs, and at least with linear regression, when something goes wrong, you can look at it and tell what went wrong and why. Which is the whole point.
- esahione 7y agoHumans are notoriously good at coming up with bullshit explanations for both random/statistical & deterministic phenomena. Why do we want AI to do the same? "Trust"? Please. I work in finance (investment strategy; ensemble trading models) and I trust my algorithms more than I'd trust 99% of the portfolio managers out there. And these PMs are incredibly capable at coming up with bullshit explanations for market moves on short notice.
- esquire_900 7y agoYou are right. In that sense, the AI that puts out the best bullshit will eventually "win", as in general with humanity. Your perspective (and most readers here) is very different then those of the general population though. The creator of an AI has a deep understanding of how it came to live (even if it does not fully grasp the current learned decision making). But for the average Joe, it's impossible to correctly decide to trust an AI or not. So even though you're completely right, it's the direction we'll go in anyway. Either humans or other AI's vouch for an algorithm, and 99% of the population will just go with it. It's crap stacked on other crap, and some entities in the right place hugely abuse their position, but it's the only way.
- hos234 7y ago> Humans are notoriously good at coming up with bullshit explanations This is true, but bullshit stories have benefit not just to one side but both sides of the relationship more often than not, which is why evolution hasn't got ride of lying - its not a bug its a feature (for certain class of problems)- https://www.nationalgeographic.com/magazine/2017/06/lying-hoax-false-fibs-science/ https://www.nationalgeographic.com/magazine/2017/06/lying-ho...
- kerkeslager 7y agoUh, could you point out what part of that article you think supports your claim that "bullshit stories have benefit not just to one side but both sides of the relationship more often than not"? I just read it and didn't see anything that indicated that. In the past few years I've worked pretty hard on building habits of honesty. Not from an ethical perspective--I'm an atheist and don't see any inherent reason to follow any prescriptive ideology--but from a self-benefit perspective. Cooperation is highly beneficial, moreso perhaps than at any time in history, and honesty allows a deeper level of cooperation with my peers. It's surprisingly difficult, as someone who viewed themselves initially as an already-honest person, but it yields major benefits. In the long term, I'm not even convinced that lying benefits the liar, let alone the person being lied to.
- rantwasp 7y agolet’s try this by substituting AI with human: A human that explains its actions is a first step towards a human we can maybe trust. what a bunch of bs. what is trust? also, can we stop pretending we understand how the brain works and that the small incremental progress that is made in this area means that AI is just around the corner? it’s not
- TeMPOraL 7y agoYeah, let's try. Or maybe rephrase it to: a human whose actions you can understand is a human you can begin to trust. We have it easy with other humans, because our brains are all alike. We all run on the same architecture, same firmware. In the occasional cases of unpredictable people, we tend to assume buggy or damaged hardware, and we either fix them or keep them away from anything where predictability matters (e.g. aviation, or gun ownership, or driving), up to and including locking the hardest cases up. With algorithms, we're playing hard mode. There is no theory of mind for software. The "thoughts" of AIs are entirely unlike our own. So we need to do extra work to make them predictable.
- pharke 7y agoSo they used a symbolic planner to direct the robot's actions and showed this to the human participants. They didn't use a neural network in the learning process. So how does this advance the explainability of the hard to explain parts of AI? Better yet, how would this strategy even be applicable to NN based learning models? We don't even have sufficient knowledge of how our own symbolic systems map to our brain activity. There is even evidence against a direct causal relationship between our symbolic narrative and our actions such as split brain experiments where the actions of the nonverbal hemisphere are "explained" or rationalized by the other post hoc. See also postdictive illusions and the varied arguments against free will. My view is that the processes that constitute our minds are actually almost entirely non-verbal and the language oriented parts are merely a specialization of a more general rule. I'm not saying that language has no effect on our conscious processes, it wouldn't be a useful system if it played no part, but I do consider it to be more of a side channel or perhaps an abstraction of the processes that are driving what we call will or thought. I believe this may be a fundamental flaw in how we approach understanding consciousness or any other aspect of intelligence.
- timkam 7y agoI understand your concerns, but there is a large body of XAI research that addresses them. For example, there are post-hoc explainability algorithms like LIME (https://github.com/marcotcr/lime https://github.com/marcotcr/lime) that "explain" the classification decisions of machine learning models. The obvious alternative is to use (deep) learning only to mine simpler, fully explainable models (sacrifice correlation power for intelligibility). What I don't think exist yet are approaches that go full scale with the "reasoning backwards" approach. Like a neural network making a BS classification decision and then trying to "reason backwards" in a symbolic way to defend the decision. But works like this one: https://openai.com/blog/debate/ https://openai.com/blog/debate/ go roughly into this direction (the work seemed to be, at the time of writing, in its infancy).
- walleeee 7y agoEven if advanced machine intelligence could generate explanations, does that really establish trustworthiness? We can't even trust humans to explain their actions reasonably well. We're usually way better at rationalization than we are at introspection. How are we to know the same isn't true for the artificial intelligence?
- variaga 7y agoAny (artificial) intelligent agent sophisticated enough to convincingly explain its reasons, is likely also sophisticated enough to lie about them.
- walleeee 7y agoMaybe, but imo a really good Chinese room that "learns" to tell us what we want to hear is way more likely in the foreseeable future than an AI with enough of a "mind", as we commonly understand it, to deliberately lie to us.
- 0xBA5ED 7y agoDoesn't this assume the AI internally models the problem in a way that isn't foreign to the human? I believe the most effective AI will definitely not model problems like a human.
- MiracleUser 7y agoI'm not sure what you mean. Why would a foreign model inherently have the attribute of being inexplicable? Isn't it more likely that we would look at a foreign model and think, "this makes sense, but i wouldnt have ever thought to do it this way". Our senses might be fooled, but I think there would almost always exist a set of words to explain what the AI is doing assuming the AI is capable enough to find them
- 0xBA5ED 7y agoWell, in order to explain why you did a thing, you need to have some understanding of how the other party thinks in order to explain it in a way they will understand. And in order to explain it in that way, you would need to reflect upon the meaning of your own decisions in the context of how the other party models things.
- MiracleUser 7y agoYes I think we would need a design goal to provide the AI with "some understanding" of how we think. It should be able to do its work in any model that works for it, which would then be translated back to our language. Thankfully, language is flexible so it can express many models we do not understand in ways that can help us understand them. It would be interesting to see some sort of study on where the boundaries are on what human language is capable of modelling or not
- blackflame 7y agoWhat happens when it's explanations stop making sense and it gives the equivalence of a "because" response?
- pmoriarty 7y agoSo the AI that can lie most convincingly will earn humanity's trust?
- deleted 7y ago[deleted]
- krishsai 7y agoFully autonomous robots are not yet completely brought into regular usage. So I think there is no need to worry much about it.