3 ms·

I get the author's feelings about why we shouldn't tell this story, but I still disagree. It's a pithy, funny example of GIGO in machine learning. People could

I get the author's feelings about why we shouldn't tell this story, but I still disagree. It's a pithy, funny example of GIGO in machine learning. People could read conclusions about the abilities of neural networks from the story, but they're wrong to do so -- it's a PEBKAC error, not a technology one. "Truthy" cautionary tales are a near-universal feature of human cultures -- why shouldn't machine learning have some?

lgas 9y ago
> I suggest that dataset bias is real but exaggerated by the tank story, giving a misleading indication of risks from deep learning and that it would be better to not repeat it but focus on established risks like AI systems optimizing for wrong utility functions. He's not arguing against having cautionary tales, he's arguing that we should base them on actual problems instead of imaginary ones.
- Rotten194 9y ago
  But ensuring correct datasets is a problem that people new to machine learning have to be made aware of. It's easy for people to see that NNs are good with 'noisy' data and incorrectly assume they can throw any data at it and get good results. And I think the quote is a false dilemma -- its not like we can't have multiple different stories for different problems. Make up / find a "truthy" story to spread for misoptimized utility functions if one doesn't exist, but there's no need to kill this useful story in the process.
  - goialoq 9y ago
    If a failure mode has never been reported in the wild, why is it so important to tell a juicy story about it, at the expense of attention to empirically observed failure modes?
    - digi_owl 9y ago
      Google's image tech confusing black people and gorillas?
    - aqsalose 9y ago
      Sure, it wouldn't do any harm to move to relevant and well-sourced examples, or construct such demonstrations specifically for the class. This HN discussion alone has gathered several instances of classifiers overfitting or fitting to features that do not generalize outside the training set as well as it first may appear, e.g. the curious case of leopard sofa [1] linked by andreasvc [2] demonstrates more or less same problem as the tank parable (leopard detector performs quite well but in reality is not detecting the exact thing we'd expect). However, also consider that the tank parable has circulated in textbooks and undergraduate introductory lectures for several decades. The main lesson I learned from the story that after training a model, one should validate it to see if it does generalize to detecting tanks both during night and daytime. Is it really a surprise that it might be difficult find egregiously naive mistakes? [1] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht... [2] https://news.ycombinator.com/item?id=15486441 https://news.ycombinator.com/item?id=15486441
robotresearcher 9y ago
It's a plausible story. Fine to present as a parable, but we should stop presenting it as true unless we can find a reliable source for it. Until today, I believed it was true. It was told to me as an undergrad, by a professor who believed it himself.
- rrmm 9y ago
  I first heard it in an AI class as well. When I relate the story I usually give it as an anecdote of apocryphal origin. But I do love the story, because it brings into focus several issues: what do you actually want your classifier to learn? what does your training set actually teach? how do you know you've learned the right thing? Many times it seems like people go into these things hoping that the machine learning part will figure out things for them and relieve themselves of the problem of thinking hard about the problem. It doesn't. It only moves your problem over a bit and increases the difficulty. In fact this problem pops up even in pedagogy where the lessons people are taught actually train them to do the wrong thing (for example pilots responding to aircraft attitude upsets). The parable's lesson is a simplistic one, basically: "stop and think about what you're doing". But like other simple lessons about crying wolf or stitching in-time, it bears repeating.
  - robotresearcher 9y ago
    > It only moves your problem over a bit and increases the difficulty. Well, that's not quite true. In robot sensing several things have recently moved from the nigh-on-impossible column to the holy-shit-that-actually-works-pretty-well column, thanks to ML. But I agree with the rest of it.

lgas 9y ago

> I suggest that dataset bias is real but exaggerated by the tank story, giving a misleading indication of risks from deep learning and that it would be better to not repeat it but focus on established risks like AI systems optimizing for wrong utility functions. He's not arguing against having cautionary tales, he's arguing that we should base them on actual problems instead of imaginary ones.

Rotten194 9y ago
But ensuring correct datasets is a problem that people new to machine learning have to be made aware of. It's easy for people to see that NNs are good with 'noisy' data and incorrectly assume they can throw any data at it and get good results. And I think the quote is a false dilemma -- its not like we can't have multiple different stories for different problems. Make up / find a "truthy" story to spread for misoptimized utility functions if one doesn't exist, but there's no need to kill this useful story in the process.
- goialoq 9y ago
  If a failure mode has never been reported in the wild, why is it so important to tell a juicy story about it, at the expense of attention to empirically observed failure modes?
  - digi_owl 9y ago
    Google's image tech confusing black people and gorillas?
  - aqsalose 9y ago
    Sure, it wouldn't do any harm to move to relevant and well-sourced examples, or construct such demonstrations specifically for the class. This HN discussion alone has gathered several instances of classifiers overfitting or fitting to features that do not generalize outside the training set as well as it first may appear, e.g. the curious case of leopard sofa [1] linked by andreasvc [2] demonstrates more or less same problem as the tank parable (leopard detector performs quite well but in reality is not detecting the exact thing we'd expect). However, also consider that the tank parable has circulated in textbooks and undergraduate introductory lectures for several decades. The main lesson I learned from the story that after training a model, one should validate it to see if it does generalize to detecting tanks both during night and daytime. Is it really a surprise that it might be difficult find egregiously naive mistakes? [1] http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht... [2] https://news.ycombinator.com/item?id=15486441 https://news.ycombinator.com/item?id=15486441

robotresearcher 9y ago

It's a plausible story. Fine to present as a parable, but we should stop presenting it as true unless we can find a reliable source for it. Until today, I believed it was true. It was told to me as an undergrad, by a professor who believed it himself.

rrmm 9y ago
I first heard it in an AI class as well. When I relate the story I usually give it as an anecdote of apocryphal origin. But I do love the story, because it brings into focus several issues: what do you actually want your classifier to learn? what does your training set actually teach? how do you know you've learned the right thing? Many times it seems like people go into these things hoping that the machine learning part will figure out things for them and relieve themselves of the problem of thinking hard about the problem. It doesn't. It only moves your problem over a bit and increases the difficulty. In fact this problem pops up even in pedagogy where the lessons people are taught actually train them to do the wrong thing (for example pilots responding to aircraft attitude upsets). The parable's lesson is a simplistic one, basically: "stop and think about what you're doing". But like other simple lessons about crying wolf or stitching in-time, it bears repeating.
- robotresearcher 9y ago
  > It only moves your problem over a bit and increases the difficulty. Well, that's not quite true. In robot sensing several things have recently moved from the nigh-on-impossible column to the holy-shit-that-actually-works-pretty-well column, thanks to ML. But I agree with the rest of it.