5 ms·
Humans do not use explainable models. We confabulate an explanation post hoc.. There is some evidence that we may do it down to very simple cases (“why did you
by gumby 7y ago
Humans do not use explainable models. We confabulate an explanation post hoc.. There is some evidence that we may do it down to very simple cases (“why did you reach for that cereal? A: it’s good for me”).
We don’t do this for complex plans where we track multiple intermediate goals, even not on a piece of paper, but the intermediate gaps between what is written down are truly inscrutable.
That being said I have some sympathy for the technical portions of her argument (and for the rest too).
- deleted 7y ago[deleted]
- Retric 7y agoHumans don’t have completely explainable models, but we do have partiality explainable models which are still meaningful. A doctor who for example knows that antibiotics don’t work on viral infections can explain sufficient reasons to justify their actions. Shareable heuristics are a powerful tool and tossing them away binds you to the biases in your training sets.
- gumby 7y agoPerhaps I was not clear; here's an example of what I meant: We make complex reasoned plans like "how to make coffee": I'm going to need a filter in the pot, I'm going to need ground coffee, water, etc; let's make sure the filter's there before I add the coffee and both before the water, etc." Then we use some sort of hierarchical planning heuristics to run it. If someone asks what I did I can explain it at varying levels of resolution ("I made coffee", "I got these pieces together (xxx) and then made coffee...") again depending on various "explanation heuristics" which we learned as part of being eusocial organisms. These plans are complex (even "get out of bed and pee" is a pretty complex plan). However below that are a ton of decisions we aren't even aware we're making. I'm convinced (and my software reflects) that the actual "plans" we make organically are very short, and that the interesting plans -- the ones we can talk about and that we typically care about -- are very abstract ones. Making coffee is super abstract, after all. Perhaps for an analogy: the extremely abstract reasoning for not using an antibiotic is at the level of "chemistry" while what I'm talking about is at the level of "physics".
- deleted 7y ago[deleted]
- jeffdavis 7y agoWe have lived and evolved with human decision making since we've existed. Taking a new system of machine decisions and adding it to the mix sounds problematic.
- skybrian 7y agoI'm reminded of how proofs work. We don't understand how mathematicians really think, and each mathematician might think differently, using different mental images. But a mathematician can write a proof, and other mathematicians can verify it. Similarly, the output of an opaque and perhaps unreliable process could be not just an answer but also a justification. If the justification can be verified more easily than starting from scratch, we've made progress. Even in the humanities there is a difference between argument from authority (blind trust) and argument based on reason and justification. And sometimes we have both. Court decisions include not just the decisions but also an explanation of why the court ruled that way. It certainly doesn't eliminate bad decisions, but it's useful. So the discipline of being able to explain yourself seems pretty important. I don't think AI will be able to fully participate in high-stakes decision-making until it can do that.
- gumby 7y agoYes! An excellent example of what I was attempting to describe.
- gambler 7y agoThis kind of sentiment is just a higher-level version of gaslighting. It conflates rationality and explainability. Humans have a very good intuitive understanding of how other humans think, even when that thinking is irrational. This understanding has evolved over millions of years and it even predates our species. We developer all kids of mechanisms for controlling irrational behavior and tested them for thousands of years.
- autokad 7y agoI recently watched a video on why you should hire a managing company if you rent to tenants. One example they gave: say you rent your place to someone because you met them and they seemed trustworthy, you just broke fair housing laws in many states and can be sued because you wont be able to provide objective criteria on why you didn't rent out to other tenants. I think this relates to the current discussion. You don't need an explainable model for what cereal you choose, but you need a 100% clear one on what tenants you choose, who you lend to, who you hire, etc.