4 ms·

I'm a firm believer that it's a mistake to ask machines to do X in the hope that they'll do Y. We can't train an ANN to classify images then ask it to explain i

I'm a firm believer that it's a mistake to ask machines to do X in the hope that they'll do Y. We can't train an ANN to classify images then ask it to explain its reasoning; that's not the task we trained it for! How might we train an ANN to explain its reasoning? One approach would be to learn programs: have the ANN write programs which classify images. Then we have a classifier (run the program) and an explanation of how it works (read the program). We don't have an explanation of how the ANN chose the program, but again, we didn't train it to tell us. In principle, we could keep adding meta-layers; in practice, the search space and evaluation time would explode :)

seiji 11y ago
that's not the task we trained it for! Yes, exactly. We're not training consciousness networks. That's not even a goal of all this research. We're training dumb high-speed classifiers. How might we train an ANN to explain its reasoning? There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner"), which is closer, but still just pairwise training. The "thought" process isn't recurrent or turing complete so it can't make progress on its own.
- poizan42 11y ago
  > There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner"), which is closer, but still just pairwise training. The "thought" process isn't recurrent or turing complete so it can't make progress on its own. Aren't those generated by RNNs, which are Turing complete?
- chriswarbo 11y ago
  > There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner") I was thinking more along the lines of: "This picture contains pizza. This is because there is an ellipse which appears to be covered with cheese and pepperoni. I say that because the dominant colour is yellow and there are elliptical patches of a more reddish colour. ..." and so on, down to the pixel level. We can kind of get this by running the networks backwards, but it's not really deductive reasoning; for example, we can't correct the network by saying "that yellow ellipse covered with red ellipses is actually a pile of vomit", and watch that knowledge propagate through the weights. Instead, we have to re-train with more examples of pizzas and vomit.
pokkanome 11y ago
The biggest problem I see to training a network to program is that it would require a lot of example programs to serve as training data. Which means we need a lot of really good, bug free programs that already accomplish similar tasks. And in my opinion, we do not have enough reliable seed programs to make anything useful, at least currently. It is very easy to take a picture of a car and tell a computer what it is, not so easy to write a program and explain to a computer what it does.
teddyh 11y ago
Even humans can’t explain their own reasoning. We can only rationalize our intuitive decisions after the fact, but it’s pure hokum – all decisions are made intuitively using no rational thought whatsoever, only afterward can we make up rationalizations for it. It follows that it should be perfectly possible for an A.I. to do this, too. (Edit: I now realize that this rationalization process was theorized by Douglas Adams in Dirk Gently’s Holistic Detective Agency, as described by Wikipedia: “The story also mocks the corporate world with [the software] package called Reason, which inverts the idea of a decision-making program. Instead of proceeding from ideas and logic to a decision, it takes a decision that has already been made and creates a reasoned justification for it.” — https://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detective_Agency#Major_themes https://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detec... )
- ectoplasm 11y ago
  Let's say you want as much money as you can get. This is probably not rational. Now let's say you have a choice of being given $5 or $10, which do you choose? Why was that not a rational choice? Because it's predicated on an irrational desire?
  - teddyh 11y ago
    No, that is not what I meant – the choice itself is not made rationally, even though a theoretical rational actor might have made the same choice. The most we humans can do it to make an intuitive choice, allow the post-hoc rationalization to occur and then check its rationality by analyzing its logic. If the logic is bad, we go back and try another intuitive choice.
    - ectoplasm 11y ago
      By your own logic, you just made an intuitive decision about whether I have a point, and now you're just rationalizing it. Not much point in trying to have a rational conversation.
      - 11y ago

seiji 11y ago

that's not the task we trained it for! Yes, exactly. We're not training consciousness networks. That's not even a goal of all this research. We're training dumb high-speed classifiers. How might we train an ANN to explain its reasoning? There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner"), which is closer, but still just pairwise training. The "thought" process isn't recurrent or turing complete so it can't make progress on its own.

poizan42 11y ago
> There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner"), which is closer, but still just pairwise training. The "thought" process isn't recurrent or turing complete so it can't make progress on its own. Aren't those generated by RNNs, which are Turing complete?
chriswarbo 11y ago
> There are networks that exist today that'll happily explain an entire scene to you (the whole "this picture contains a pizza sitting on an oven in a kitchen and there is a dog in the corner") I was thinking more along the lines of: "This picture contains pizza. This is because there is an ellipse which appears to be covered with cheese and pepperoni. I say that because the dominant colour is yellow and there are elliptical patches of a more reddish colour. ..." and so on, down to the pixel level. We can kind of get this by running the networks backwards, but it's not really deductive reasoning; for example, we can't correct the network by saying "that yellow ellipse covered with red ellipses is actually a pile of vomit", and watch that knowledge propagate through the weights. Instead, we have to re-train with more examples of pizzas and vomit.

pokkanome 11y ago

The biggest problem I see to training a network to program is that it would require a lot of example programs to serve as training data. Which means we need a lot of really good, bug free programs that already accomplish similar tasks. And in my opinion, we do not have enough reliable seed programs to make anything useful, at least currently. It is very easy to take a picture of a car and tell a computer what it is, not so easy to write a program and explain to a computer what it does.

teddyh 11y ago

Even humans can’t explain their own reasoning. We can only rationalize our intuitive decisions after the fact, but it’s pure hokum – all decisions are made intuitively using no rational thought whatsoever, only afterward can we make up rationalizations for it. It follows that it should be perfectly possible for an A.I. to do this, too. (Edit: I now realize that this rationalization process was theorized by Douglas Adams in Dirk Gently’s Holistic Detective Agency, as described by Wikipedia: “The story also mocks the corporate world with [the software] package called Reason, which inverts the idea of a decision-making program. Instead of proceeding from ideas and logic to a decision, it takes a decision that has already been made and creates a reasoned justification for it.” — https://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detective_Agency#Major_themes https://en.wikipedia.org/wiki/Dirk_Gently%27s_Holistic_Detec... )

ectoplasm 11y ago
Let's say you want as much money as you can get. This is probably not rational. Now let's say you have a choice of being given $5 or $10, which do you choose? Why was that not a rational choice? Because it's predicated on an irrational desire?
- teddyh 11y ago
  No, that is not what I meant – the choice itself is not made rationally, even though a theoretical rational actor might have made the same choice. The most we humans can do it to make an intuitive choice, allow the post-hoc rationalization to occur and then check its rationality by analyzing its logic. If the logic is bad, we go back and try another intuitive choice.
  - ectoplasm 11y ago
    By your own logic, you just made an intuitive decision about whether I have a point, and now you're just rationalizing it. Not much point in trying to have a rational conversation.
    - 11y ago