5 ms·

this is a fool's errand. today's statistical inference based "AI" is a probability distribution that squishes itself through cracks in data to "learn" a path of

this is a fool's errand. today's statistical inference based "AI" is a probability distribution that squishes itself through cracks in data to "learn" a path of least resistance. subject-object reasoning systems, such as English, are cut from a different cloth. semantic language is a discontinuous function. Asking asking an entity that can only discern its environment through continuous functions to explain itself is like asking an amoeba to play piano. Machine Learning is a statistics renaissance, it is changing the world, but "Explainable AI" is a notion that is totally divorced from where the field is at atm.

FullMtlAlcoholc 9y ago
Or, it's like expecting a mathematical function to solve itself
dsjoerg 9y ago
It's easy to be dismissive, but the project becomes more tractable when you think of the task of "explaining" as yet another classification task. The goal is not necessarily to explain _exactly_ what's happening inside the AI, no more than a human can explain their decisions by describing _exactly_ what's happening inside their heads. The explanation is an additional _decision_ that accompanies the original classification decision. The explanation space has its own utility function and the goal is to find the explanation with the highest utility.
- torbjorn 9y ago
  I don't mean to come off as dismissive, and yes my comment is a sassy one. Machine Vision systems are proof that neural networks and other machine learning techniques are revolutionary. I really believe that. But the notion of "Explainable AI" just reeks of upper middle management folks performing what they see as visionary risk management. When in reality these people don't know what they are talking about. Explaining exactly what's happening inside "the AI" is the domain of linear algebra. We can't be as exact in plain english but the generalization "it's fitting the training data" speaks volumes more than any _just_around_the_corner_ "explainable AI" secret sauce algorithm ever will.
  - cossatot 9y ago
    I can't speak for the middle management, but as I scientist I am very uninterested in AI based on neural nets because of the lack of explication. Success at prediction without a commensurate advancement in theory is pretty useless for me, because I haven't learned anything new about the world. And while Google et al are good at stacking ML models, those models are nowhere near as elegant and composable as scientific advancements.
    - torbjorn 9y ago
      You are right there is a large gulf between the deep learning theory hype and the measurable improvements in physical quality of life that the hard sciences have proved themselves capable of. However I submit a notable exception! The vision system of self driving cars is a neural network that is currently changing the world and there is no lack of explication, we know exactly how it works. These "convolutional neural network" infer object boundaries from differences in intensity among matrices of pixels by scanning different patches of the pixel matrix and cross comparing these patches. But you are right there is a lot of hype. I think part of the issue is the collective idolization of "algorithms". The limiting factor is data. And we don't have the data required to development models that are agent like in structure. Self interested agent systems, that employ subject-object language, have only been developed once. And they needed a training dataset measured in millions of years of evolution.
      - visarga 9y ago
        I think you're glossing over many accomplishments in ML. It's not just vision that is successful. There's NLP, optimization, ranking, voice, speech, recommendations and many other tasks that work well. You say the limiting factor is data. But there has been a trend in the last 2 years to run ML on data generated from simulations (games, auto, chemical bonds, robots in VR, AlphaGo, etc). Simulations are dynamic datasets with unlimited flexibility. The better we learn to simulate, the closer we will get to reasoning. Both simulations and reasoning are based on object-relation graphs for describing the scene. My vision is that we will build better, more precise simulators that would allow an AI to input a problem and run experiments and search ("what happens if"). Just like AlphaGo, but for the real world. A combination of neural nets for "intuition" and simulators for precision would solve the problem. Simulators could be of many kinds: chemical molecule simulator, cell simulator, physical simulator, city traffic simulator, car simulator, flight simulator, and so on. Basically what's been the object of activity of supercomputing, but this time with neural nets selecting the experiments.
drvdevd 9y ago
In theory, one could make an amoeba play piano somehow, if we think of a piano broadly speaking as just an array of notes with various associated variables (pitch, force, etc). I'm thinking... using chemical signals of some sort to force amoeboid cells to move in certain directions or take certain shapes, we can control gradiants of amoeba in sequence, and thus "teach" them encoded piano music. But I'm not sure how this applies to AI.
- torbjorn 9y ago
  this is a pretty cool idea actually... if we just spent some time manipulating their chemical reward systems such that their amoeboid population densities opened and closed circuits on a piano key we could make them produce music. and i think the transference of this pattern would be enough to imbue them with the fire of thought! they would seek to control their collective amoeboid destiny! their will to be would drive them to express themselves, they would form an amoeboid super structure, spelling in english letters: "s e n d n u d e s". And AI is born.
tim333 9y ago
And yet a fair bit of human existence is trying to explain what our neural networks did. (See "don't know why I didn't come" and many similar lyrics)
blazespin 9y ago
It is a statistics renaissance, you answers your own question. The AI needs to explain in easily understandable ways how the stats arrived at a conclusion. Eg, "We discovered there is a correlation that on sunny days, people drink more icemochas when you play the music louder. That's why we're telling you right now to increase the volume in your stores."
chewyshine 9y ago
This is exactly on point. Current "AI" is pattern matching without the notion of cause or object. There is no understanding or representation.
nl 9y ago
Sorry, but this comment is completely uniformed. Explaining ML inference is an active area of research, and there are plenty of decent approaches which are getting good results. For example, LIME[1] can explain why a neural network doing visual classification came out with the result that it did. Distilling the Knowledge in a Neural Network[2] is another approach which leads towards simpler representations. LIME isn't perfect, but that's the nature of research (and why DARPA funds this kind of thing). this is a fool's errand "Distilling the Knowledge in a Neural Network", authors: Geoffrey Hinton, Oriol Vinyals, Jeff Dean Fools, all of them. DARPA should ask HN next time it wants to know about the current state of AI. [1] https://github.com/marcotcr/lime https://github.com/marcotcr/lime [2] https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531
- bayonetz 9y ago
  I've done some experiments with LIME -- it's one of the most promising approaches for extracting prediction reasons from arbitrary and otherwise opaque models. Another interesting approach specific to random forests is decision paths: http://blog.datadive.net/interpreting-random-forests/ http://blog.datadive.net/interpreting-random-forests/ You get some weird outputs from these sometimes though which makes it hard to automatically show them to users. For example, a reason might be "because you liked salad restaurant X you should check out check BBQ place Y" and it's because there happens to be an overlap in the users who like both captured in your model. Yet it can cause cognitive dissonance for the users who are either strictly healthy eaters (salads only, no BBQ) or delude themselves into thinking they are (forgetting how much they actually order BBQ in addition to salads). That's the main challenge I see -- figuring out how to filter out reasons from these approaches that don't jive with common intuitions or, even harder, get people to learn to trust the reasons as counter-intuitive as they may seem.
  - nl 9y ago
    Yes, there is plenty of work to do. Tree-based classifiers always have the reputation of being "explainable". As you note this isn't always as simple as it should be. In the unsupervised space I really like plotting dendrograms on hierarchical clustering. There's an excellent example in the recent DeepMoji paper[1] where they show how similar emojis cluster together AND how you can truncate the hierarchy at different depths to get capture different ranges of emotion. It's laughable when people insist that DARPA are foolish for funding work in this area. [1] https://arxiv.org/pdf/1708.00524.pdf https://arxiv.org/pdf/1708.00524.pdf
KKKKkkkk1 9y ago
Why does the same criticism not apply to brain cells?
hyperion2010 9y ago
I think there is quite a bit of fertile ground to study what kinds of answers really are 'explainable.' The story(ies) I usually bring up in this case are 'Who Sunk the Boat?' and 'The Straw That Broke the Camel's Back.' Any decision process that includes summation to a bound (literally any integrate and fire neuron), is going to require much more complex study to understand the real circumstances under which it fires. If we could build these things we would already know, far, far more about the universe than we do. Maybe I will add a new story to my list: 'Why did the chicken cross the road?'

FullMtlAlcoholc 9y ago

Or, it's like expecting a mathematical function to solve itself

dsjoerg 9y ago

It's easy to be dismissive, but the project becomes more tractable when you think of the task of "explaining" as yet another classification task. The goal is not necessarily to explain _exactly_ what's happening inside the AI, no more than a human can explain their decisions by describing _exactly_ what's happening inside their heads. The explanation is an additional _decision_ that accompanies the original classification decision. The explanation space has its own utility function and the goal is to find the explanation with the highest utility.

torbjorn 9y ago
I don't mean to come off as dismissive, and yes my comment is a sassy one. Machine Vision systems are proof that neural networks and other machine learning techniques are revolutionary. I really believe that. But the notion of "Explainable AI" just reeks of upper middle management folks performing what they see as visionary risk management. When in reality these people don't know what they are talking about. Explaining exactly what's happening inside "the AI" is the domain of linear algebra. We can't be as exact in plain english but the generalization "it's fitting the training data" speaks volumes more than any _just_around_the_corner_ "explainable AI" secret sauce algorithm ever will.
- cossatot 9y ago
  I can't speak for the middle management, but as I scientist I am very uninterested in AI based on neural nets because of the lack of explication. Success at prediction without a commensurate advancement in theory is pretty useless for me, because I haven't learned anything new about the world. And while Google et al are good at stacking ML models, those models are nowhere near as elegant and composable as scientific advancements.
  - torbjorn 9y ago
    You are right there is a large gulf between the deep learning theory hype and the measurable improvements in physical quality of life that the hard sciences have proved themselves capable of. However I submit a notable exception! The vision system of self driving cars is a neural network that is currently changing the world and there is no lack of explication, we know exactly how it works. These "convolutional neural network" infer object boundaries from differences in intensity among matrices of pixels by scanning different patches of the pixel matrix and cross comparing these patches. But you are right there is a lot of hype. I think part of the issue is the collective idolization of "algorithms". The limiting factor is data. And we don't have the data required to development models that are agent like in structure. Self interested agent systems, that employ subject-object language, have only been developed once. And they needed a training dataset measured in millions of years of evolution.
    - visarga 9y ago
      I think you're glossing over many accomplishments in ML. It's not just vision that is successful. There's NLP, optimization, ranking, voice, speech, recommendations and many other tasks that work well. You say the limiting factor is data. But there has been a trend in the last 2 years to run ML on data generated from simulations (games, auto, chemical bonds, robots in VR, AlphaGo, etc). Simulations are dynamic datasets with unlimited flexibility. The better we learn to simulate, the closer we will get to reasoning. Both simulations and reasoning are based on object-relation graphs for describing the scene. My vision is that we will build better, more precise simulators that would allow an AI to input a problem and run experiments and search ("what happens if"). Just like AlphaGo, but for the real world. A combination of neural nets for "intuition" and simulators for precision would solve the problem. Simulators could be of many kinds: chemical molecule simulator, cell simulator, physical simulator, city traffic simulator, car simulator, flight simulator, and so on. Basically what's been the object of activity of supercomputing, but this time with neural nets selecting the experiments.

drvdevd 9y ago

In theory, one could make an amoeba play piano somehow, if we think of a piano broadly speaking as just an array of notes with various associated variables (pitch, force, etc). I'm thinking... using chemical signals of some sort to force amoeboid cells to move in certain directions or take certain shapes, we can control gradiants of amoeba in sequence, and thus "teach" them encoded piano music. But I'm not sure how this applies to AI.

torbjorn 9y ago
this is a pretty cool idea actually... if we just spent some time manipulating their chemical reward systems such that their amoeboid population densities opened and closed circuits on a piano key we could make them produce music. and i think the transference of this pattern would be enough to imbue them with the fire of thought! they would seek to control their collective amoeboid destiny! their will to be would drive them to express themselves, they would form an amoeboid super structure, spelling in english letters: "s e n d n u d e s". And AI is born.

tim333 9y ago

And yet a fair bit of human existence is trying to explain what our neural networks did. (See "don't know why I didn't come" and many similar lyrics)

blazespin 9y ago

It is a statistics renaissance, you answers your own question. The AI needs to explain in easily understandable ways how the stats arrived at a conclusion. Eg, "We discovered there is a correlation that on sunny days, people drink more icemochas when you play the music louder. That's why we're telling you right now to increase the volume in your stores."

chewyshine 9y ago

This is exactly on point. Current "AI" is pattern matching without the notion of cause or object. There is no understanding or representation.

nl 9y ago

Sorry, but this comment is completely uniformed. Explaining ML inference is an active area of research, and there are plenty of decent approaches which are getting good results. For example, LIME[1] can explain why a neural network doing visual classification came out with the result that it did. Distilling the Knowledge in a Neural Network[2] is another approach which leads towards simpler representations. LIME isn't perfect, but that's the nature of research (and why DARPA funds this kind of thing). this is a fool's errand "Distilling the Knowledge in a Neural Network", authors: Geoffrey Hinton, Oriol Vinyals, Jeff Dean Fools, all of them. DARPA should ask HN next time it wants to know about the current state of AI. [1] https://github.com/marcotcr/lime https://github.com/marcotcr/lime [2] https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531

bayonetz 9y ago
I've done some experiments with LIME -- it's one of the most promising approaches for extracting prediction reasons from arbitrary and otherwise opaque models. Another interesting approach specific to random forests is decision paths: http://blog.datadive.net/interpreting-random-forests/ http://blog.datadive.net/interpreting-random-forests/ You get some weird outputs from these sometimes though which makes it hard to automatically show them to users. For example, a reason might be "because you liked salad restaurant X you should check out check BBQ place Y" and it's because there happens to be an overlap in the users who like both captured in your model. Yet it can cause cognitive dissonance for the users who are either strictly healthy eaters (salads only, no BBQ) or delude themselves into thinking they are (forgetting how much they actually order BBQ in addition to salads). That's the main challenge I see -- figuring out how to filter out reasons from these approaches that don't jive with common intuitions or, even harder, get people to learn to trust the reasons as counter-intuitive as they may seem.
- nl 9y ago
  Yes, there is plenty of work to do. Tree-based classifiers always have the reputation of being "explainable". As you note this isn't always as simple as it should be. In the unsupervised space I really like plotting dendrograms on hierarchical clustering. There's an excellent example in the recent DeepMoji paper[1] where they show how similar emojis cluster together AND how you can truncate the hierarchy at different depths to get capture different ranges of emotion. It's laughable when people insist that DARPA are foolish for funding work in this area. [1] https://arxiv.org/pdf/1708.00524.pdf https://arxiv.org/pdf/1708.00524.pdf

KKKKkkkk1 9y ago

Why does the same criticism not apply to brain cells?

hyperion2010 9y ago

I think there is quite a bit of fertile ground to study what kinds of answers really are 'explainable.' The story(ies) I usually bring up in this case are 'Who Sunk the Boat?' and 'The Straw That Broke the Camel's Back.' Any decision process that includes summation to a bound (literally any integrate and fire neuron), is going to require much more complex study to understand the real circumstances under which it fires. If we could build these things we would already know, far, far more about the universe than we do. Maybe I will add a new story to my list: 'Why did the chicken cross the road?'