5 ms·

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. But what the court finds uncons

by t8sr 4y ago

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. But what the court finds unconstitutional about it is that they're using computers. "Automated data analysis or interpretation requires justification under constitutional law." I'm sure something is lost in translation. But even assuming a lot of context, I find it hilarious. Germany's highest court seems to have stumbled into avoiding a dystopia not because they don't want some biased ML model bullying minorities, but because they plain and simple don't like all this computer nonsense. (Grumble grumble.)

martijnvds 4y ago
Requiring humans to sift through the data makes it easier to limit its use. You either need a lot more budget (= oversight) to hire all the people to do the work, or do less of it. Sure computers also cost money but you can keep using them as long as you pay the (already budgeted) power bill and (also already budgeted) maintenance/upkeep.
- jjk166 4y ago
  On the contrary, computers follow their programming. They don't cut corners to make their workload easier, they don't make more mistakes when they are overworked and tired. When you don't have enough resources to make the computer follow it's programming, you also don't have the resources to use the computer. Keeping things manual means you're adding in all the errors that are unavoidable with humans, and limiting use by constraining resources introduces even more errors. Appropriate oversight is an additional cost on top of the labor you already can't afford, running without adequate oversight is not only possible but strongly selected for. Furthermore, whereas with computers you can theoretically go through their programming and say "ah this is why a crime was predicted" and the model which generated that insight could be potentially challenged and changed, with a human you'll never really know if there is an issue, and your options for dealing with issues are incredibly limited. If you can't trust a computer to do math, you certainly can't trust a human to do it.
  - BlueTemplar 4y ago
    More efficiency isn't necessarily better. There's a balance of power, and for instance, allowing different databases to be linked increases the power of those with access. Exponentially. That's why it's heavily limited. And the new programs using neural networks, if allowed to be used, would bring with them even more issues coming from their black box behavior, which would encourage a lack of accountability (doubtful that programmers would volunteer to be directly responsible for all their mistakes !)
    - jjk166 4y ago
      The argument has nothing to do with efficiency, except insofar as human inefficiency makes abuse of power more likely. The problem is there isn't fundamentally a balance of power, having unlinked databases doesn't allow the "good" databases to shield you from the "bad" databases, it just increases the potential for errors to arise. I'll amend my previous closing argument, if you wouldn't trust someone with a knife, why would you trust them with a whole cutlery set? We're not trying to make the system more efficient, we are trying to eliminate potential failure points, which every human involved is. If it is more efficient, that's just gravy on top. If you are really concerned with who is in a position of power, consider the difference between a small team of maybe a dozen people working behind closed doors versus software which could be made as public as the laws it is meant to uphold and evaluated by anyone on earth. Any issue, be it from malice or incompetence, would be plain as day. And if you don't like black box neural networks, I have some terrible news about the human brain. There is no technical reason why the creators of an AI should be any less accountable for a flaw in their product than the producers of any other technology. This wouldn't generally be the programmers themselves, but it would be the companies employing those programmers. If a car company for example becomes aware of a serious flaw in their product and they don't take appropriate steps to fix it, they're liable. Likewise for most things. The creators of these systems don't need to volunteer to be responsible for how they perform, they are compelled. There have been countless tools added to law enforcement's utility belt over the centuries like radar guns and dna matching, and every legal system in the world has dealt with evaluating their efficacy and potential for bias. There is nothing about an AI that makes it incompatible with these existing safeguards.
Wolle33 4y ago
That's not correct. In the same paragraph it states that in general a digital analysis of information is necessary for increasing police effectiveness and that this is suitable under constitutional law. The issue that the constitutional court is pointing out, is that there is no appropriate threshold for using such analysis software. In other words they that that you can't use is for anything, just when it is really necessary and proportional.
thenaturalist 4y ago
As a German native I think you're a bit off both on translation and based on understanding of the foundations of the German constitution. No constitutional law expert myself, trying to contextualize in the hope it helps: Core pillars of the German constitutional system are informational self-determination (i.e. personal data has a very high degree of protection under the constitution, see Germany's history) and proportionality (in using its powers, the state must choose methods appropriate to specific situations, obv. to hinder misuse of power - see Germany's history again). So the ruling in essence, as I understand it, says among other things: 1. Automated data analysis needs to respect informational self-determination of anyone (i.e. no data mining per se on people who are not suspects for a given type of crime being investigated) 2. Automated data analysis needs to be proportional, meaning that the purpose of running any analysis "for the benefit of the public" must be reasonably well established and the analysis must have a strict goal. I.e. if John stole something, you have to analyse the data such that you can credibly prove that identification for theft was the purpose. Starting with looking for thieves but ending up identifying X movie pirates, Y weed consumers etc. is not proportional and crosses the right to informational self-determination. So in the above example, the movie piraters and weed smokers might have not had an official investigation into their offenses on record, so by law the presumption of innocence must hold - you cannot do blanket investigations without reasonable cause under German law. In the final paragraphs, the text actually states under which conditions automated data analysis would fullfil constitutional requirements: 1. The legislative needs to codify which data points exactly can be used for a given offense where offenders are to be identified with data analysis. Laws also need to explicitly codify limits of usable data. 2. If automated data analysis is performed by authorities, they must document the reason for initiating an analysis, the methods chosen, the criteria for labeling the outputs as "guilty/innocent", document the entire procedure and be able to release all of it to the public domain so that public data protection officers - or anyone else - can audit the results. All of this sounds super reasonable to me. To be clear, automatic data analysis for the purpose of extreme threats to national or public security are exempt if I understand correctly. These would be cases where it is proportional to perform such analyses for the greater good. But you cannot do mass number crunching on ordinary citizens. Yay in my book.
- patates 4y ago
reaperducer 4y ago
This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. Police in the United States are close to this already. Chicago's police department used to brag about its data-crunching ability to predict where crime would happen. At the time, it was just by area, but it was stated that with enough data, it could be narrowed down to the block. It's not that big a leap to extrapolate that to the individual. My guess, though, is that it didn't work, and the city was just parroting the promises made by the salespeople of whatever system they bought for this task. If it worked, then police would preemptively flood certain areas where they know crime is about to happen. But, as is seen on television most nights, and in the complaints of the aldermen, the police mostly seem to react to crime, rather than prevent it. The use of social media by crime mobs has only made it worse.
- predictivep 4y ago
  Pre-crime https://en.wikipedia.org/wiki/Pre-crime https://en.wikipedia.org/wiki/Pre-crime Predictive policing https://en.wikipedia.org/wiki/Predictive_policing https://en.wikipedia.org/wiki/Predictive_policing Predictive policing in the United States https://en.wikipedia.org/wiki/Predictive_policing_in_the_United_States https://en.wikipedia.org/wiki/Predictive_policing_in_the_Uni... Gunfire locator https://en.wikipedia.org/wiki/Gunfire_locator https://en.wikipedia.org/wiki/Gunfire_locator - Shot spotter https://en.wikipedia.org/wiki/ShotSpotter https://en.wikipedia.org/wiki/ShotSpotter Radiolocation https://en.wikipedia.org/wiki/Radiolocation https://en.wikipedia.org/wiki/Radiolocation Trilateration https://en.wikipedia.org/wiki/Trilateration https://en.wikipedia.org/wiki/Trilateration https://youtu.be/zb5RJyrk4gc?t=1m05s https://youtu.be/zb5RJyrk4gc?t=1m05s Fusion center: https://en.wikipedia.org/wiki/Fusion_center https://en.wikipedia.org/wiki/Fusion_center ResponsePod, Camp-in-a-Container https://shiftpod.com/responsepod/our-2020-catalog https://shiftpod.com/responsepod/our-2020-catalog Zello Work https://support.zello.com/hc/en-us/articles/230748827-How-does-Zello-Work-differ-from-Zello-Free-for-consumers- https://support.zello.com/hc/en-us/articles/230748827-How-do... Sangoma Meet https://www.sangoma.com/products/communications-services/team-collaboration/sangoma-meet/ https://www.sangoma.com/products/communications-services/tea... : "Use the Sangoma Meet app to join a video call, or join audio-only by calling the dial-in number in the meeting invite." https://www.networkworld.com/article/3134324/beyond-911-other-n-1-1-codes-you-should-know.html https://www.networkworld.com/article/3134324/beyond-911-othe... https://988lifeline.org/ https://988lifeline.org/ Clean language interviewing, coaching https://en.wikipedia.org/wiki/Clean_language https://en.wikipedia.org/wiki/Clean_language
maCDzP 4y ago
I no lawyer but a friend of who is said that there are differences in how countries practices law. The nordic countries in Europe are concerned with the process. The assumption being that if the process is right the judgment will be just. In southern Europe one is more concerned with the outcome, as long as the judgment is just, then there is wiggle room in the process. This seems like a judgment that focuses on process and not outcome. I have probably butchered what my friend said.

by t8sr 4y ago

martijnvds 4y ago

Requiring humans to sift through the data makes it easier to limit its use. You either need a lot more budget (= oversight) to hire all the people to do the work, or do less of it. Sure computers also cost money but you can keep using them as long as you pay the (already budgeted) power bill and (also already budgeted) maintenance/upkeep.

jjk166 4y ago
On the contrary, computers follow their programming. They don't cut corners to make their workload easier, they don't make more mistakes when they are overworked and tired. When you don't have enough resources to make the computer follow it's programming, you also don't have the resources to use the computer. Keeping things manual means you're adding in all the errors that are unavoidable with humans, and limiting use by constraining resources introduces even more errors. Appropriate oversight is an additional cost on top of the labor you already can't afford, running without adequate oversight is not only possible but strongly selected for. Furthermore, whereas with computers you can theoretically go through their programming and say "ah this is why a crime was predicted" and the model which generated that insight could be potentially challenged and changed, with a human you'll never really know if there is an issue, and your options for dealing with issues are incredibly limited. If you can't trust a computer to do math, you certainly can't trust a human to do it.
- BlueTemplar 4y ago
  More efficiency isn't necessarily better. There's a balance of power, and for instance, allowing different databases to be linked increases the power of those with access. Exponentially. That's why it's heavily limited. And the new programs using neural networks, if allowed to be used, would bring with them even more issues coming from their black box behavior, which would encourage a lack of accountability (doubtful that programmers would volunteer to be directly responsible for all their mistakes !)
  - jjk166 4y ago
    The argument has nothing to do with efficiency, except insofar as human inefficiency makes abuse of power more likely. The problem is there isn't fundamentally a balance of power, having unlinked databases doesn't allow the "good" databases to shield you from the "bad" databases, it just increases the potential for errors to arise. I'll amend my previous closing argument, if you wouldn't trust someone with a knife, why would you trust them with a whole cutlery set? We're not trying to make the system more efficient, we are trying to eliminate potential failure points, which every human involved is. If it is more efficient, that's just gravy on top. If you are really concerned with who is in a position of power, consider the difference between a small team of maybe a dozen people working behind closed doors versus software which could be made as public as the laws it is meant to uphold and evaluated by anyone on earth. Any issue, be it from malice or incompetence, would be plain as day. And if you don't like black box neural networks, I have some terrible news about the human brain. There is no technical reason why the creators of an AI should be any less accountable for a flaw in their product than the producers of any other technology. This wouldn't generally be the programmers themselves, but it would be the companies employing those programmers. If a car company for example becomes aware of a serious flaw in their product and they don't take appropriate steps to fix it, they're liable. Likewise for most things. The creators of these systems don't need to volunteer to be responsible for how they perform, they are compelled. There have been countless tools added to law enforcement's utility belt over the centuries like radar guns and dna matching, and every legal system in the world has dealt with evaluating their efficacy and potential for bias. There is nothing about an AI that makes it incompatible with these existing safeguards.

Wolle33 4y ago

That's not correct. In the same paragraph it states that in general a digital analysis of information is necessary for increasing police effectiveness and that this is suitable under constitutional law. The issue that the constitutional court is pointing out, is that there is no appropriate threshold for using such analysis software. In other words they that that you can't use is for anything, just when it is really necessary and proportional.

thenaturalist 4y ago

As a German native I think you're a bit off both on translation and based on understanding of the foundations of the German constitution. No constitutional law expert myself, trying to contextualize in the hope it helps: Core pillars of the German constitutional system are informational self-determination (i.e. personal data has a very high degree of protection under the constitution, see Germany's history) and proportionality (in using its powers, the state must choose methods appropriate to specific situations, obv. to hinder misuse of power - see Germany's history again). So the ruling in essence, as I understand it, says among other things: 1. Automated data analysis needs to respect informational self-determination of anyone (i.e. no data mining per se on people who are not suspects for a given type of crime being investigated) 2. Automated data analysis needs to be proportional, meaning that the purpose of running any analysis "for the benefit of the public" must be reasonably well established and the analysis must have a strict goal. I.e. if John stole something, you have to analyse the data such that you can credibly prove that identification for theft was the purpose. Starting with looking for thieves but ending up identifying X movie pirates, Y weed consumers etc. is not proportional and crosses the right to informational self-determination. So in the above example, the movie piraters and weed smokers might have not had an official investigation into their offenses on record, so by law the presumption of innocence must hold - you cannot do blanket investigations without reasonable cause under German law. In the final paragraphs, the text actually states under which conditions automated data analysis would fullfil constitutional requirements: 1. The legislative needs to codify which data points exactly can be used for a given offense where offenders are to be identified with data analysis. Laws also need to explicitly codify limits of usable data. 2. If automated data analysis is performed by authorities, they must document the reason for initiating an analysis, the methods chosen, the criteria for labeling the outputs as "guilty/innocent", document the entire procedure and be able to release all of it to the public domain so that public data protection officers - or anyone else - can audit the results. All of this sounds super reasonable to me. To be clear, automatic data analysis for the purpose of extreme threats to national or public security are exempt if I understand correctly. These would be cases where it is proportional to perform such analyses for the greater good. But you cannot do mass number crunching on ordinary citizens. Yay in my book.

patates 4y ago

reaperducer 4y ago

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. Police in the United States are close to this already. Chicago's police department used to brag about its data-crunching ability to predict where crime would happen. At the time, it was just by area, but it was stated that with enough data, it could be narrowed down to the block. It's not that big a leap to extrapolate that to the individual. My guess, though, is that it didn't work, and the city was just parroting the promises made by the salespeople of whatever system they bought for this task. If it worked, then police would preemptively flood certain areas where they know crime is about to happen. But, as is seen on television most nights, and in the complaints of the aldermen, the police mostly seem to react to crime, rather than prevent it. The use of social media by crime mobs has only made it worse.

predictivep 4y ago
Pre-crime https://en.wikipedia.org/wiki/Pre-crime https://en.wikipedia.org/wiki/Pre-crime Predictive policing https://en.wikipedia.org/wiki/Predictive_policing https://en.wikipedia.org/wiki/Predictive_policing Predictive policing in the United States https://en.wikipedia.org/wiki/Predictive_policing_in_the_United_States https://en.wikipedia.org/wiki/Predictive_policing_in_the_Uni... Gunfire locator https://en.wikipedia.org/wiki/Gunfire_locator https://en.wikipedia.org/wiki/Gunfire_locator - Shot spotter https://en.wikipedia.org/wiki/ShotSpotter https://en.wikipedia.org/wiki/ShotSpotter Radiolocation https://en.wikipedia.org/wiki/Radiolocation https://en.wikipedia.org/wiki/Radiolocation Trilateration https://en.wikipedia.org/wiki/Trilateration https://en.wikipedia.org/wiki/Trilateration https://youtu.be/zb5RJyrk4gc?t=1m05s https://youtu.be/zb5RJyrk4gc?t=1m05s Fusion center: https://en.wikipedia.org/wiki/Fusion_center https://en.wikipedia.org/wiki/Fusion_center ResponsePod, Camp-in-a-Container https://shiftpod.com/responsepod/our-2020-catalog https://shiftpod.com/responsepod/our-2020-catalog Zello Work https://support.zello.com/hc/en-us/articles/230748827-How-does-Zello-Work-differ-from-Zello-Free-for-consumers- https://support.zello.com/hc/en-us/articles/230748827-How-do... Sangoma Meet https://www.sangoma.com/products/communications-services/team-collaboration/sangoma-meet/ https://www.sangoma.com/products/communications-services/tea... : "Use the Sangoma Meet app to join a video call, or join audio-only by calling the dial-in number in the meeting invite." https://www.networkworld.com/article/3134324/beyond-911-other-n-1-1-codes-you-should-know.html https://www.networkworld.com/article/3134324/beyond-911-othe... https://988lifeline.org/ https://988lifeline.org/ Clean language interviewing, coaching https://en.wikipedia.org/wiki/Clean_language https://en.wikipedia.org/wiki/Clean_language

maCDzP 4y ago

I no lawyer but a friend of who is said that there are differences in how countries practices law. The nordic countries in Europe are concerned with the process. The assumption being that if the process is right the judgment will be just. In southern Europe one is more concerned with the outcome, as long as the judgment is just, then there is wiggle room in the process. This seems like a judgment that focuses on process and not outcome. I have probably butchered what my friend said.