5 ms·
You can become a suspect for all sorts of random things you have no control over, like being near a crime, knowing certain people, etc. I don't see how it can
by BurningFrog 6y ago
You can become a suspect for all sorts of random things you have no control over, like being near a crime, knowing certain people, etc.
I don't see how it can work any other way.
- donkeyd 6y agoWhich means there will be 100 suspects where usually there would be maybe 2. Which means the data is useless and therefore won't be used. Law enforcement doesn't want a system that incorrectly flags hundreds of people, unlike what some people seem to think. They want systems that reliably flag potential suspects, because that reduces work in stead of increasing it.
- inefficient 6y agoThat is only true if you assume they truly care about having the correct suspect rather than having a suspect they can get convicted. However, there is a long history of wrongful convictions and police and prosecutors using bad data to get them. Check out the Central Park Five for a big name one.
- AWildC182 6y agoThis. To put it simply, they like Precise data, not Accurate data. They want a bunch of data-points saying one thing and couldn't care less if it's objectively the right target, not a bunch of data points saying a bunch of things, one of which is actually right. Precision data gives them convictions and convictions give them promotions. As they say, any metric that becomes a target... This is why giving the police invasive surveillance tech is a terrible idea, they will only focus on the data that fits their narrative and discard the rest. The defense doesn't even need to know that any other data exists.
- rbanffy 6y ago> They want a bunch of data-points saying one thing and couldn't care less if it's objectively the right target If your system produces data points with 8 significant digits of wrong data, we have two problems: the system and whoever approved the purchase. Any convictions will have to be corroborated by other evidence.
- AWildC182 6y agoUntil that other-evidence is just the product of some other hellish surveillance system. What I'm really saying is broadly we need fewer of these systems as they will only be abused, especially in concert. If you have one system that says the suspect had their phone off at the time of the bombing, and another saying they searched for "how do bombs work" on google a few years ago the jury will eat that shit up even though in actuality it provides almost zero evidence of anything.
- frogpelt 6y agoUnless they can use it target they people they already "know" committed the crime, maybe?
- throwaway0a5e 6y agoWhen the crime in question is a headline crime they're happy to go through hundreds of "leads" in order to find someone they can pin it on.
- da_big_ghey 6y agoThe problem is that one is explainable and the other is not. "He was near the scene of the crime and has no convincing alibi," is very different from, "The computer said he's a suspect and we don't know why, but we still want a warrant." People being targeted for being anomalous is bad, but centralizing and scaling it up is worse.
- BurningFrog 6y agoTo me "being a suspect" is a long way from a warrant being issued. If cops can get a warrant on mere hunches, then I agree we have real problems. But a separate problem.
- chronic3802 6y ago> The computer said he's a suspect and we don't know why, but we still want a warrant. Mission critical ML systems (G/FB ads, crime forensics, medical decision support, financial algorithms) do not work like this. The designers know exactly which features are causing responses/predictions in the model, their respective perturbation sensitivities, and have clear bounds regarding adversarial inputs/outliers. “Not knowing how the model works” is definitely true for deep learning, though.
- thylacine222 6y agoThis seems... incorrect. First of all, deep learning absolutely is used in mission critical systems: https://www.techrepublic.com/article/intel-and-ge-healthcares-x-ray-machine-uses-embedded-ai-to-prioritize-scans/ https://www.techrepublic.com/article/intel-and-ge-healthcare... Second, simply because designers use systems that are formally interpretable, that doesn't mean that designers know exactly what causes a given response in the model. Formal interpretability means you know what function the model is approximating -- there's a long way from that to a human-readable interpretation. Finally, even if the designers can tell you what features cause a certain response in the model, they can only tell you that for features that are encoded in the data! There are plenty of features that aren't encoded in your data that can nevertheless affect the outcome of a model!
- andrei_says_ 6y agoAll metadata being collected for the lifetime of all our phone numbers also includes all relationships, with varying degrees of separation.
- lotsofpulp 6y agoOr even data from genetic testing, literally showing relationships.
- throwaway_kufu 6y agoBeing near a crime when it occurred is entirely different than your phone being at your house when a crime occurred somewhere else.
- yters 6y agoEventually everyone is a potential criminal and guilty until presumed innocent.