8 ms·
While I mostly agree, I’ll also point out that humans are also black boxes, who have often hard time explaining their decisions.
by jhrmnn 7y ago
While I mostly agree, I’ll also point out that humans are also black boxes, who have often hard time explaining their decisions.
- ptah 7y agohumans can be held accountable for their decisions though
- taneq 7y agoOnly because we all agree that a human is a ‘person’ ie. a thing that can take blame.
- mehrdadn 7y agoFascinating. I'd never thought of defining a 'person' this way.
- rehasu 7y agoYou might have never been in a VP+ position in a company then. A lot of tasks in these positions are about avoiding consequences or shifting blame for actions that are either necessary but look bad or that had to be made with way too little input information. And not necessarily just one's own actions.
- taneq 7y agoI was definitely thinking about corporate blame-shifting when I wrote that, and actually nearly said "legal 'person'" instead.
- empath75 7y agoSo make these ai’s corporations — instant personhood.
- ivanhoe 7y agoIn this case the analogy would be that software companies should be hold accountable for the decisions their AI makes?
- salawat 7y agoThe article makes the case that if an interoperable model (one we can explain) isn't used, then the user of the black box model should have the burden of proof to prove that no interoperable model exists that does the job, and some level of responsibility for trying to develop one.
- picometer 7y agoThis point is really about incentives: since we know that our fellow humans are held accountable, we know they have the incentive to make high-quality decisions that will not impact the world negatively. Therefore, it's easier to trust them, even when they go with their gut. We can't project the same incentive structure onto software companies. They are a different scale than humans, and might be able to tolerate the possible hit to their reputation better than a human can. Their incentives are usually money-based rather than social-based. And for the models themselves, they have no incentives, unless you count "reduce their error rate" or "be interesting enough for researchers to continue to research them". We barely know why they work. Our basis for trust is rather tenuous.
- TeMPOraL 7y agoWe have hundreds of thousands of years (if not more) of trying to understand other humans behind us. It's encoded in our biology (theory of mind, mirror neurons), and more recently, in countless of disciplines we engage in - from arts, through law, to psychology. Truly unexpected human behavior is a very rare thing. We ought to be able to do better than building our ML models as black boxes and saying "humans are black boxes too". We're engineering this technology, so we have an opportunity to make it inspectable. We should take it.
- carlmr 7y agoOne of the things neuroscience and psychology showed us is that humans always have a post hoc reasoning they can explain, but it's not always the actual reason they did something. So humans explaining themselves is often misleading data, which may be worse than no data.
- Ensorceled 7y ago"humans explaining themselves" is entirely different thing than "humans understanding human behaviour". You may have rationalized your choices, but I understand both your choices AND that you rationalized them.
- TeMPOraL 7y agoPersonally, I currently believe that this research is quite probably bunk. There was a recent HN thread in which it was pointed out that some of the results of experiments involved might have been data processing errors. And then there's the most obvious confounding factor - peoples' reported reasoning may not be their true, or complete, reasoning, because people aren't always comfortable with revealing all the details. But regardless of that, while post-hoc explanations may not be that good for building predictive models of behavior, observed behavior is. We generally know how people behave, and we know the expected variance due to individual and situational circumstances. Truly unexpected behavior is rare in society, and we tend to filter it out. Truly unpredictable people get locked up and/or are given medical help, but even less unpredictable people are tested and kept out of professions and activities where that unpredictability could cause problems.