3 ms·

I mean, you could always do an audit and see what percent of a thing your automated system catches. That definitely doesn't seem inherently unknowable.

salawat 7y ago
You're not quite getting it. You measure accuracy of detection via a training set of definitely known examples, and comparing it to the detections reported by your system. You can say with certainty that their system catches 99% of your training data set, but that doesn't mean it Will generalize to "all permutations of child abuse ever". That's two totally different things. In the end their use of that statistic illustrates an attempt to lie with statistics more than anything else.
- edmundsauto 7y ago
  By this standard, no system could ever say that it generalizes to all permutations. Especially when the definition of the measurement outcome (child abuse) varies.
  - salawat 7y ago
    Now you get it. You are now the thorn in the side of every AI salesman trying to pitch you his Neural Net based wonder machine, or politician trying to sell you on giving up rights because they need to be able to violate everyone's privacy because statistics. If you understand the actual nature of these measurements, it becomes quite a bit more difficult to let the "zomg neato" factor run away with you.
  - cwkoss 7y ago
    This is true. Until we implement omnipotent god-AIs, generalizable perfection is an unattainable standard.
- lozaning 7y ago
  Alex Pinto gave a phenomenal DefCon talk several years ago that really highlights this issue, "Secure Because Math". It gets into how modern ML/AI/Voodoo IPS systems essentially can't work because the training data or baseline is so impossible to accurately come up with. https://www.youtube.com/watch?v=TYVCVzEJhhQ https://www.youtube.com/watch?v=TYVCVzEJhhQ
- 6gvONxR4sf7o 7y ago
  If I say 15% of cars in San Francisco are Subarus, are you going to say that's impossible to know? Sure it may be true in my dataset, "but that doesn't mean it will generalize to all possible permutations of [traffic] ever." Well, no shit. But that's not the statement being made. If I wanted to know the accuracy of an important system. I'd do continual audits. Take a (stratified) sample of all things, get ground truth labels for whether they should have been detected, and see what percent were in fact detected. Then I could estimate how much my systems did and didn't catch at any point in time.
  - salawat 7y ago
    >If I say 15% of cars in San Francisco are Subarus, are you going to say that's impossible to know? No, I'd ask you what your methodology was when you measured. For instance, going to the DMV and getting a list of all registered vehicles, and using that as a representative dataset to reason from is fine. (Depending on your definition of cars in San Francisco, and whether or not that is meant to include cars transiently moving through or not.) Now if you told me you were using a Neural Network to recognize all Subaru models that drove by a particular camera, I'd start asking you questions, and want to see your training data, output, etc. >Sure it may be true in my dataset, but that doesn't mean it will generalize to all possible permutations of traffic ever. Well, no shit. But that's not the statement being made. That generally is the statement being made when people make claims about the accuracy of neural networks. Particularly the marketing people. You can only measure their accuracy within a constrained dataset, and overfitting is a thing. >If I wanted to know the accuracy of an important system. I'd do continual audits. Take a (stratified) sample of all things, get ground truth labels for whether they should have been detected, and see what percent were in fact detected. Then I could estimate how much my systems did and didn't catch at any point in time. Which, again means you're operating off things you know a priori to be the case. You don't know the full extent of everything to begin with, which phrasing in this article implies. I'm not saying audits can't be useful; it'll get you a number, and sometimes that's all you need. You just need to be clear about what the numbers actually mean. There's lies, damn lies, and statistics after all.

salawat 7y ago

You're not quite getting it. You measure accuracy of detection via a training set of definitely known examples, and comparing it to the detections reported by your system. You can say with certainty that their system catches 99% of your training data set, but that doesn't mean it Will generalize to "all permutations of child abuse ever". That's two totally different things. In the end their use of that statistic illustrates an attempt to lie with statistics more than anything else.

edmundsauto 7y ago
By this standard, no system could ever say that it generalizes to all permutations. Especially when the definition of the measurement outcome (child abuse) varies.
- salawat 7y ago
  Now you get it. You are now the thorn in the side of every AI salesman trying to pitch you his Neural Net based wonder machine, or politician trying to sell you on giving up rights because they need to be able to violate everyone's privacy because statistics. If you understand the actual nature of these measurements, it becomes quite a bit more difficult to let the "zomg neato" factor run away with you.
- cwkoss 7y ago
  This is true. Until we implement omnipotent god-AIs, generalizable perfection is an unattainable standard.
lozaning 7y ago
Alex Pinto gave a phenomenal DefCon talk several years ago that really highlights this issue, "Secure Because Math". It gets into how modern ML/AI/Voodoo IPS systems essentially can't work because the training data or baseline is so impossible to accurately come up with. https://www.youtube.com/watch?v=TYVCVzEJhhQ https://www.youtube.com/watch?v=TYVCVzEJhhQ
6gvONxR4sf7o 7y ago
If I say 15% of cars in San Francisco are Subarus, are you going to say that's impossible to know? Sure it may be true in my dataset, "but that doesn't mean it will generalize to all possible permutations of [traffic] ever." Well, no shit. But that's not the statement being made. If I wanted to know the accuracy of an important system. I'd do continual audits. Take a (stratified) sample of all things, get ground truth labels for whether they should have been detected, and see what percent were in fact detected. Then I could estimate how much my systems did and didn't catch at any point in time.
- salawat 7y ago
  >If I say 15% of cars in San Francisco are Subarus, are you going to say that's impossible to know? No, I'd ask you what your methodology was when you measured. For instance, going to the DMV and getting a list of all registered vehicles, and using that as a representative dataset to reason from is fine. (Depending on your definition of cars in San Francisco, and whether or not that is meant to include cars transiently moving through or not.) Now if you told me you were using a Neural Network to recognize all Subaru models that drove by a particular camera, I'd start asking you questions, and want to see your training data, output, etc. >Sure it may be true in my dataset, but that doesn't mean it will generalize to all possible permutations of traffic ever. Well, no shit. But that's not the statement being made. That generally is the statement being made when people make claims about the accuracy of neural networks. Particularly the marketing people. You can only measure their accuracy within a constrained dataset, and overfitting is a thing. >If I wanted to know the accuracy of an important system. I'd do continual audits. Take a (stratified) sample of all things, get ground truth labels for whether they should have been detected, and see what percent were in fact detected. Then I could estimate how much my systems did and didn't catch at any point in time. Which, again means you're operating off things you know a priori to be the case. You don't know the full extent of everything to begin with, which phrasing in this article implies. I'm not saying audits can't be useful; it'll get you a number, and sometimes that's all you need. You just need to be clear about what the numbers actually mean. There's lies, damn lies, and statistics after all.