6 ms·

Unknown Unknowns in Machine Learning

60 points by atg_abhishek 6y ago

trhway 6y ago
>To score points in this competition, participants should submit a set of image-label pairs that will be confirmed by human-in-the-loop raters, whose votes should be in disagreement with the average machine score for the label over a number of machine learning models. haven't checked since then, in 2016 GoogLeNet won the Large Scale Visual Recognition Challenge 2014 (ILSVRC2014), and among misclassified by the model images was this image labeled "hay", and the model's predictions were: sorrel, hartebeest, Arabian camel, dromedary, Camelus dromedarius, warthog, gazelle. http://cs.stanford.edu/people/karpathy/ilsvrc/val/ILSVRC2012_val_00042652.JPEG http://cs.stanford.edu/people/karpathy/ilsvrc/val/ILSVRC2012...
mjburgess 6y ago
"Unknown unknowns" is an interesting term of art, but it might hide what the issue really is. A UU Example for an ML system is simply proof that it has failed to acquire the relevant classification (etc.) criteria -- and on most problems, ML systems cannot in principle do so (eg., consider that 3D data is often needed and only 2D given). I'd prefer to call these "breaking examples", or, "beyond-scope examples". What's happening is that the domain of inference (X) fails to contain data about the target (Y), though Y belongs to the "Problem Domain" of which X stands as an incomplete sample. eg., X = images of roads; Y = pedestrian in street; PD = X + 3D data + behavioural data + pedestraian_theoryofmind + weather_data + .. Actually deciding Y requires PD; we have X. So there are many "breaking examples" which fall within PD but outside of X. Note, in general, we cannot measure the full PD using contemporary machine systems. Humans are able to capture much more of the PD in complex ways which would take a while to spell out. Much of modern ML, as advertised, is a bit of delusion and a bit of charlatanism: we are offered a means of obtaining Y via X, but never consider PD. We just keep "evaluating" the systems on examples consistent with X. This is the point of this article. However, it is a much more radical one than it is making out. This practice is really a means of disguising that ML systems do not work.
- deleted 6y ago
  [deleted]
- plutonorm 6y ago
  What would you make of Tesla driving AI? It is exposed to the full problem domain. Does this make a different kind of thing?
  - nerdponx 6y ago
    It is not. It is exposed to whatever training data it was trained on, and maybe whatever new data it encounters if it is being continuously trained during use.
    - plutonorm 6y ago
      That doesn't make any sense. It has eyes and ears and the training data is the environment.
- 0-_-0 6y ago
  I've heard the difference being referred to as aleatoric uncertainty (noise inherent in the observations) vs. epistemic uncertainty (model uncertainty caused by lack of data)
- hallqv 6y ago
  What is the real issue in your view? That ml-systems don’t work? In that case I strong disagree, I use multitudes of them every day (and you most likely do as well) and they work very well. As someone working with building ML-systems that solve real-world problems I can say that distinction between known unkowns and unknowns unkowns is a very useful one.
  - fock 6y ago
    what kind of ML-systems are you using daily? (non-embedded ones) Probably stuff around search qualifies today, but this does actually not work for me more often than it does. Sometimes I'd just like the index...
teekert 6y ago
"Unknown unknowns, on the other hand, are examples where a model is confident about its answer, but is actually wrong." So, these models are actually very much like humans :) "It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" -unsure [0] At least humans know how to fix this (from Pocahontas): "But if you walk the footsteps of a stranger You'll learn things you never knew you never knew" [0] https://quoteinvestigator.com/2018/11/18/know-trouble/ https://quoteinvestigator.com/2018/11/18/know-trouble/
- nerdponx 6y ago
  The problem is not that data scientists don't know this. The problem is always one of finding the right "stranger" whose footsteps can be walked in. Or in the case of really huge models, it's a case of cost-benefit calculation so that you don't walk in more footsteps than you can afford.

60 points by atg_abhishek 6y ago

>To score points in this competition, participants should submit a set of image-label pairs that will be confirmed by human-in-the-loop raters, whose votes should be in disagreement with the average machine score for the label over a number of machine learning models. haven't checked since then, in 2016 GoogLeNet won the Large Scale Visual Recognition Challenge 2014 (ILSVRC2014), and among misclassified by the model images was this image labeled "hay", and the model's predictions were: sorrel, hartebeest, Arabian camel, dromedary, Camelus dromedarius, warthog, gazelle. http://cs.stanford.edu/people/karpathy/ilsvrc/val/ILSVRC2012_val_00042652.JPEG http://cs.stanford.edu/people/karpathy/ilsvrc/val/ILSVRC2012...

mjburgess 6y ago

"Unknown unknowns" is an interesting term of art, but it might hide what the issue really is. A UU Example for an ML system is simply proof that it has failed to acquire the relevant classification (etc.) criteria -- and on most problems, ML systems cannot in principle do so (eg., consider that 3D data is often needed and only 2D given). I'd prefer to call these "breaking examples", or, "beyond-scope examples". What's happening is that the domain of inference (X) fails to contain data about the target (Y), though Y belongs to the "Problem Domain" of which X stands as an incomplete sample. eg., X = images of roads; Y = pedestrian in street; PD = X + 3D data + behavioural data + pedestraian_theoryofmind + weather_data + .. Actually deciding Y requires PD; we have X. So there are many "breaking examples" which fall within PD but outside of X. Note, in general, we cannot measure the full PD using contemporary machine systems. Humans are able to capture much more of the PD in complex ways which would take a while to spell out. Much of modern ML, as advertised, is a bit of delusion and a bit of charlatanism: we are offered a means of obtaining Y via X, but never consider PD. We just keep "evaluating" the systems on examples consistent with X. This is the point of this article. However, it is a much more radical one than it is making out. This practice is really a means of disguising that ML systems do not work.

deleted 6y ago
[deleted]
plutonorm 6y ago
What would you make of Tesla driving AI? It is exposed to the full problem domain. Does this make a different kind of thing?
- nerdponx 6y ago
  It is not. It is exposed to whatever training data it was trained on, and maybe whatever new data it encounters if it is being continuously trained during use.
  - plutonorm 6y ago
    That doesn't make any sense. It has eyes and ears and the training data is the environment.
0-_-0 6y ago
I've heard the difference being referred to as aleatoric uncertainty (noise inherent in the observations) vs. epistemic uncertainty (model uncertainty caused by lack of data)
hallqv 6y ago
What is the real issue in your view? That ml-systems don’t work? In that case I strong disagree, I use multitudes of them every day (and you most likely do as well) and they work very well. As someone working with building ML-systems that solve real-world problems I can say that distinction between known unkowns and unknowns unkowns is a very useful one.
- fock 6y ago
  what kind of ML-systems are you using daily? (non-embedded ones) Probably stuff around search qualifies today, but this does actually not work for me more often than it does. Sometimes I'd just like the index...

teekert 6y ago

"Unknown unknowns, on the other hand, are examples where a model is confident about its answer, but is actually wrong." So, these models are actually very much like humans :) "It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" -unsure [0] At least humans know how to fix this (from Pocahontas): "But if you walk the footsteps of a stranger You'll learn things you never knew you never knew" [0] https://quoteinvestigator.com/2018/11/18/know-trouble/ https://quoteinvestigator.com/2018/11/18/know-trouble/

nerdponx 6y ago
The problem is not that data scientists don't know this. The problem is always one of finding the right "stranger" whose footsteps can be walked in. Or in the case of really huge models, it's a case of cost-benefit calculation so that you don't walk in more footsteps than you can afford.