9 ms·

Automated data analysis for the prevention of criminal acts is unconstitutional

246 points by _tk_ 4y ago

CPLX 4y ago
(In Germany)
- wongarsu 4y ago
  Also it's not unconstitutional in principle, but the range where it would be allowable is fairly narrow. If I understood it correctly, it would be permissible in similar circumstances as those that allow wiretapping
the_generalist 4y ago
This is straight up some Minority Report stuff
- IYasha 4y ago
  OMFD, just watched it yesterday! 8( I think it's more like borderline presumption of guilt.
- rootusrootus 4y ago
  The only problem with Minority Report (aside from torturing the precogs, I mean) was treating the prevented murder as a crime and punishing it. If they had just used it as an intervention tool ... well, there'd be no movie.
  - zirgs 4y ago
    Attempted murder is a crime pretty much everywhere.
luckylion 4y ago
Editorialized title unfortunately. The legislation of two German states to allow automated data analysis ... is unconstitutional. They'll need to narrow the scope and specify thresholds when to apply the data (and will probably do well to limit application and storage to not run the risk to have it struck down again). It's not a broad decision that automated data analysis for crime-prevention is unconstitutional.
t8sr 4y ago
This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. But what the court finds unconstitutional about it is that they're using computers. "Automated data analysis or interpretation requires justification under constitutional law." I'm sure something is lost in translation. But even assuming a lot of context, I find it hilarious. Germany's highest court seems to have stumbled into avoiding a dystopia not because they don't want some biased ML model bullying minorities, but because they plain and simple don't like all this computer nonsense. (Grumble grumble.)
- martijnvds 4y ago
  Requiring humans to sift through the data makes it easier to limit its use. You either need a lot more budget (= oversight) to hire all the people to do the work, or do less of it. Sure computers also cost money but you can keep using them as long as you pay the (already budgeted) power bill and (also already budgeted) maintenance/upkeep.
  - jjk166 4y ago
    On the contrary, computers follow their programming. They don't cut corners to make their workload easier, they don't make more mistakes when they are overworked and tired. When you don't have enough resources to make the computer follow it's programming, you also don't have the resources to use the computer. Keeping things manual means you're adding in all the errors that are unavoidable with humans, and limiting use by constraining resources introduces even more errors. Appropriate oversight is an additional cost on top of the labor you already can't afford, running without adequate oversight is not only possible but strongly selected for. Furthermore, whereas with computers you can theoretically go through their programming and say "ah this is why a crime was predicted" and the model which generated that insight could be potentially challenged and changed, with a human you'll never really know if there is an issue, and your options for dealing with issues are incredibly limited. If you can't trust a computer to do math, you certainly can't trust a human to do it.
    - 4y ago
V__ 4y ago
It is unconstitutional because the law was written too broadly. The court said two things. It's fine to use this for highly legally protected goods, e.g., protection of body, life, or freedom. However, the same high standards need to apply like for hidden surveillance measures currently used.
green_dee 4y ago
Well, every model has its errors, and intrusive criminal "prevention" is self-fulfilling. Just imagine, everyday 6AM an officer come at your door to check your apartment "to prevent" your crimes, suddenly every random guy in the street looks suspicious and spying on you, then some officers just randomly walks by your workplace, police stop you everytime you travel... This would eventually lead to an insult to the mafia guy and then boom you have already commited contumacy and the model was right! You are a criminal and you should be punished!
- lostlogin 4y ago
  It’s a little like a machine controlled ‘stop and frisk’.
  - weard_beard 4y ago
    But would I have done it if the police weren’t following me? What if the computer is identifying at risk paranoid schizophrenics likely to be pushed over the edge by noticeable increases in their personal surveillance? Frankly that seems more plausible than telling the future
    - lostlogin 4y ago
      I don’t make any claim at all for its capability. But if you load a dataset into some sort of ML algorithm right now, with the data coming from current policing statistics, you are going to be perpetuating some pretty dark stuff. Police target various groups and minorities.
      - weard_beard 4y ago
        "The data is racist" will be a common refrain in this AI revolution.
- naijaboiler 4y ago
  you just described the minority experience in many parts of US
_s_a_m_ 4y ago
As if they give a shit. This was the case since decades they doing similar practices.
LatteLazy 4y ago
For small crimes. For bigger crimes (limit TBC) they're fine. And I think that is ok...
omgomgomgomg 4y ago
I speak German and I have just read the text, it is vastly complicated court and lawyer speak. But it is clear that this legislature is pushing for crime prevention based on aggregated data from computer systems(where from, exactly?) The arguments are that a manual extraction and collection would be too slow and that this should be limited to only a handful of severe potential crimes. As a prevention rather than mitigation technique. This is quite bad. Now the objection is not purely because the data is coming from computers, but because it will reveal data on people tangentially connected to the targets. This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic. It is very clear that these things are pushed by rather smart minds, matter experts and this makes it clear they do not care about the innocent peoples privacy, as they could not possibly be so ignorant to overlook this matter. Justice even the conventianal way spawns injustices, now the lazy government wants to automate and scale their procedures. W.t.h.
- c4mpute 4y ago
  > innocent peoples privacy It isn't just about their privacy but also their very freedom. Being in some police database might very well land you in the crosshairs of some investigation and related niceness like searching your home, taking you in for questioning and keeping you there because you didn't say something...
  - omgomgomgomg 4y ago
    True, I equate privacy and freedom, give or take. Anyone who ever end up an their radar would probably never be removed from that,and never even know. It is highly offensive that the German government pretends it does not know where the crime is happening. How about they deal with a few of these clans numbering 100s of people? This can be done , has to be done the convential way, as these clans will not be as stupid as leave digital traces, as computer illeterate as they might be. The government has not even defined who they are targeting, they just use the commonly thrown around terms.
    - 4y ago
thenaturalist 4y ago
Super interesting ruling in my book. Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me. Basically the court mandates that any future laws passed for legalizing automated data analysis must explicitly 1. For any given crime to be automatically analysed, define the data model and define appropriate analysis techniques or methods. Those laws should be interesting reads. 2. For any one analysis happening, document and later publish to the public domain the reason for initiating an analysis, the methods chosen, the criteria for labeling someone as "guilty/innocent" - basically the entire process so that the analysis is auditable end to end. Again, this is not a suggestion in the ruling, but a requirement. This is really good news for personal liberties in my book. Stark contrast to US/ China data mining on citizens.
- jll29 4y ago
  An earlier poster wrote: > This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic. Whereas you say: >Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me. I think this reads like the court curtailed reasonably broad data grabbing behaviors such as those Snowdon has revealed happened illegally and are still happening in the US. It still permits effective, focused/targeted data mining. So at least the court learned from the Nazi/Stasi times but also from the times of left-wing terrorism ("Rasterfahndung" during the Bader-Meinhoff terrorist gang) - i.e., that these techniques are effective yet must not be applied to broadly. If this reading is correct (I'm not a legal expert), then the main danger is that lawmakers are incapable (or unwilling) to implement requirements (1.) and (2).
  - omgomgomgomg 4y ago
    I agree fully, I am struggling to imagine how they can go and push such a legislature infront of this governing body with a straight face. This is more dangerous than most people would think, once something like this is codified into law, many eu countries will follow suit, just for good measure. What are the motives and who benefits from this? This idea was born somewhere and writing all the drafts costs a lot of money, who was footing this bill and why?
shasts 4y ago
So how would fraud prevention be interpreted? For eg: denying service from a German retailer depending on the ML prediction result of the transaction being classified as fraud probable(Germany has pay by invoice), would also be constitutional or not? Or credit score/risk rating using ML, trained on a feature extracted from parameters like Zip code, ethnicity could also classified as automated data analysis for prevention of criminal acts, right? Or this is only applicable to police?
- rom-antics 4y ago
  In general a retailer can refuse any customer for any reason (unless you're discriminating against a protected class) The reason they do this isn't to fight crime, but to save the business from losing money. So I don't see how a ruling about preventing crime would apply.
- thenaturalist 4y ago
  Only applicable to police use.
possiblelion 4y ago
I think it's crazy that people are considering "linking previously unconnected automated databases and data sources in analysis platforms and permitting systematic access of data across sources through searches." as something like Minority Report. The truth is that Law Enforcement investigations are currently taking information from dozens of silos/databases, manually copy&pasting them into Excel and Word and then finally making a pdf or powerpoint to share the results. This is obviously a really inefficient way to do things and wastes a lot of taxpayer money. A technical solution that would allow LE to crosslink data they already have is a no-brainer. P.S: Very biased view, as I'm building/selling exactly this tool :)
- beecafe 4y ago
  How do you sleep at night?
  - suction 4y ago
    [dead]
  - mistrial9 4y ago
    in an expensive bed, with high-tech security all around of course!
- thenaturalist 4y ago
  > A technical solution that would allow LE to crosslink data they already have is a no-brainer. A technical solution that is Open Source, that falls under federally mandated criteria for how it can be legally used, that automatically documents any and all access and makes the entire audit log part of any output result so that all happenings are easily auditable by 3rd parties... might be a no brainer. Anything else I'm highly skeptical. If going after syndicated and or international crime, fine. First feature I'd like to see is linking Finanzamt with LE and go after money laundering - Germany being an absolute paradise to the detriment of everyone else. If it takes manual copy paste to protect ordinary citizens from overreaching police, I welcome that technical limitation. Very biased, as I am an ordinary citizen.
DubiousPusher 4y ago
Every human needs to understand that having a computer look at data representing you is a search.
kqr 4y ago
The one argument I rarely see against automated analysis is that any analysis has a false positive rate. If you hire 1,000 humans to do that analysis there's simply a limit to how many errors they can make in total. If you introduce automation doing the same thing at a scale corresponding to 10,000× the human capacity, you will have 10,000× as many errors. This will violate Blackstone's ratio and erode trust in the system.
- ravivooda 4y ago
  TIL: Blackstone's ratio
arvidkahl 4y ago
"Dieser Eingriffsanlass bleibt angesichts der besonders daten- und methodenoffen formulierten Befugnisse weit hinter der wegen des konkreten Eingriffsgewichts verfassungsrechtlich gebotenen Schwelle einer konkretisierten Gefahr zurück." This roughly translates into "there has to be a sufficiently concrete danger to warrant such a ambiguously formulated infringement." I truly appreciate this sentiment. Any collection or analysis of data "just because there might be something in there" has to be weighed against the right of everyone that is being data-mined. This freaked me out about Minority Report when I first watched it: those precogs are essentially hacking into the future memories of everyone involved in _and by happenstance surrounding_ the lives of future criminals and their crimes. Mass-scale datamining is effectively the real-world equivalent of telepathy. And I want neither precogs nor the government in my mind.
binarymax 4y ago
In the US - this should also be the case. It should fall under the 4th amendment as unreasonable search and seizure. But the NSA, and probably other agencies, has been doing it for years anyway.
gotoeleven 4y ago
If it prevents even one misgendering it will all be worth it.
- Rallen89 4y ago
  There has to be a poe's law equivalent for gender issues at this point, if the article is anyway political someone's gonna mention it because uhh they gotta be mad at someone
  - justincredible 4y ago
    [dead]
nivenkos 4y ago
Why? IMO we should use facial recognition, digital ID, a cashless society and everything at our disposal to stamp out crime. It's the working class who pay the price at the moment. We get burgled, mugged and killed, meanwhile the wealthy lawyers and politicians can pontificate over constitutionality from their gated communities and private vehicles. It's insane that the EU is trying to ban automatic facial recognition with the terrible AI act - https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC020 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A... - nevermind the sheer amount of bureaucracy it introduces for startups (a government register of "possibly harmful" AI - wtf?).
- flangola7 4y ago
  /s?
  - nivenkos 4y ago
    No, bureaucracy and crime are awful. Bear in mind this is the same country that bans Google Street View (and on the road to ban nuclear power and GM crops). People here support this ban because they think it's "harmful", but how long until they think some technology you rely on is also harmful? GPS? Phone cameras or Google Lens? Encryption - https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-mass-surveillance-of-the-eu/ https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-ma... ? Technology is the future, and we should embrace it wherever possible - crime prevention, education, healthcare, nuclear fusion, genetic engineering, etc.
    - flangola7 4y ago
      > No, bureaucracy and crime are awful. My best friend is a sex worker. I use cannabis to control my anxiety. Neither of those are legal. Looking historically, virtually every human rights movement required illegal direct action. Where do we fit into your cashless panopticon society? The worst crimes I'm worried about are the ones that will never be illegal, the ones that benefit the powerful and rich. At-scale invasion of privacy is only one such tremendous crime, one against against human rights and dignity, but barely against the law. > Bear in mind this is the same country that bans Google Street View (and on the road to ban nuclear power and GM crops). > People here support this ban because they think it's "harmful", but how long until they think some technology you rely on is also harmful? GPS? Phone cameras or Google Lens? Encryption - https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-mass-surveillance-of-the-eu/ https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-ma... ? Some of those are genuinely harmful (street view in particular for its privacy invasion - note it is not banned but simply very unpopular) so I'm not sure what your point is. Also, you should send your ideas to the Mullvad blog authors and get their input. I think you'll find that the overlap between people who support universal end to end encryption and people who support an all-seeing police state is very small, as they are fundamentally incompatible ideas. > Technology is the future, and we should embrace it wherever possible - crime prevention, education, healthcare, nuclear fusion, genetic engineering, etc. Technology is not always good all of the time. The past century and even past decade are full of examples of net harms caused by technology that could have been avoided if we developed them in a more intentional controlled manner. A eccentric example to make the point: If there is a technology that can trigger a supernova of the sun simply by mixing water, sugar, gasoline, and bleach together in a certain ratio, then it would be extremely important we not conduct research into that technology until we have other technology that allows us to survive a supernova or can prevent that mixture from being created anywhere on Earth. Even if there are advantages of the technology (maybe a different ratio causes all humans to have 5x longer lifespans and eliminates all disease and suffering, and we gain intelligence allowing us to begin colonizing other planets within 3 years) the risk is just too high that it will be misused accidentally or maliciously until we have a way to handle it. https://en.wikipedia.org/wiki/Differential_technological_development https://en.wikipedia.org/wiki/Differential_technological_dev...
shadowgovt 4y ago
In the medium-term, the cultural shear between the US and Germany will continue to be highlighted by decisions like this. Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority. America leans lightly towards more and more centralization of data because they have the experience in recent memory of what individuals who "slip through the cracks" can do. So in Germany we see decisions like this and in the US we see police using privately-curated DNA databases to defrost the cold case of a serial killer, and they're lauded as heroes. (It's worth noting, of course, that this is all relative. Relative to most of Europe, the US is starting from a place of massive decentralization and minimal individual-tracking. Some people still find social security numbers controversial).
- sneak 4y ago
  > Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority. I theorize that in a hundred years or so, North Americans will have learned this lesson the hard way and will have atrocities in recent history, too. I'm sad I won't see the day when Bluffdale is repurposed as a memorial to state surveillance, much like the Stasi headquarters are now in Berlin.
- bakuninsbart 4y ago
  Can't find the study now, but there was a sociological survey of software engineers in the US and Europe, one really interesting thing was that the top social concern when implementing solutions for Americans was fairness, for Germans it was privacy. The other concern didn't even make the top 5 for either.
- qikInNdOutReply 4y ago
  If china ever gets democratic, a new mighty privacy player will appear. Chinas citizens have the same nasty experience as east germans had with the stasi and the poles, chechs with the cheka. They have been spied upon, reported upon by there fellow citizen and have been victims of "zersetzung". Aka social engineering to destroy a citizen. https://www.stasi-unterlagen-archiv.de/mfs-lexikon/detail/zersetzung/ https://www.stasi-unterlagen-archiv.de/mfs-lexikon/detail/ze... Link is german: Shows how denunciation aka controlled leaking of private life like affairs was used, to destroy the lifes of resisting citizens. PS: FB and Google could in theory use similar instruments against citizens that put pressure on them. Nobody could ever catch them at it. If the ccp ever gets swept away and replaced by a new better china, the resulting parties will outdo one another to promise freedom and privacy.
kilgnad 4y ago
This concept was explored in a stephen spielberg movie. Minority report. Good movie. Watch it. Stars Tom Cruise and has mission impossible vibes. In the end the government cancelled the program because of free will. A person always has the choice to change his actions because his actions aren't predetermined.
super256 4y ago
I think it’s important to mention Palantir in this context.
forevergreenyon 4y ago
time to make amendments! just like once upon a time it was constitutional to own slaves and now it isn't just think about the monetary cost-savings! (nevermind the human misery)
Eduard 4y ago
What do they mean with "first alternative"?

246 points by _tk_ 4y ago

CPLX 4y ago

(In Germany)

wongarsu 4y ago
Also it's not unconstitutional in principle, but the range where it would be allowable is fairly narrow. If I understood it correctly, it would be permissible in similar circumstances as those that allow wiretapping

the_generalist 4y ago

This is straight up some Minority Report stuff

IYasha 4y ago
OMFD, just watched it yesterday! 8( I think it's more like borderline presumption of guilt.
rootusrootus 4y ago
The only problem with Minority Report (aside from torturing the precogs, I mean) was treating the prevented murder as a crime and punishing it. If they had just used it as an intervention tool ... well, there'd be no movie.
- zirgs 4y ago
  Attempted murder is a crime pretty much everywhere.

luckylion 4y ago

Editorialized title unfortunately. The legislation of two German states to allow automated data analysis ... is unconstitutional. They'll need to narrow the scope and specify thresholds when to apply the data (and will probably do well to limit application and storage to not run the risk to have it struck down again). It's not a broad decision that automated data analysis for crime-prevention is unconstitutional.

t8sr 4y ago

This is such a German judgement. So the police are using big data to predict who will commit crimes, which is sort of dystopian. But what the court finds unconstitutional about it is that they're using computers. "Automated data analysis or interpretation requires justification under constitutional law." I'm sure something is lost in translation. But even assuming a lot of context, I find it hilarious. Germany's highest court seems to have stumbled into avoiding a dystopia not because they don't want some biased ML model bullying minorities, but because they plain and simple don't like all this computer nonsense. (Grumble grumble.)

martijnvds 4y ago
Requiring humans to sift through the data makes it easier to limit its use. You either need a lot more budget (= oversight) to hire all the people to do the work, or do less of it. Sure computers also cost money but you can keep using them as long as you pay the (already budgeted) power bill and (also already budgeted) maintenance/upkeep.
- jjk166 4y ago
  On the contrary, computers follow their programming. They don't cut corners to make their workload easier, they don't make more mistakes when they are overworked and tired. When you don't have enough resources to make the computer follow it's programming, you also don't have the resources to use the computer. Keeping things manual means you're adding in all the errors that are unavoidable with humans, and limiting use by constraining resources introduces even more errors. Appropriate oversight is an additional cost on top of the labor you already can't afford, running without adequate oversight is not only possible but strongly selected for. Furthermore, whereas with computers you can theoretically go through their programming and say "ah this is why a crime was predicted" and the model which generated that insight could be potentially challenged and changed, with a human you'll never really know if there is an issue, and your options for dealing with issues are incredibly limited. If you can't trust a computer to do math, you certainly can't trust a human to do it.
  - 4y ago

V__ 4y ago

It is unconstitutional because the law was written too broadly. The court said two things. It's fine to use this for highly legally protected goods, e.g., protection of body, life, or freedom. However, the same high standards need to apply like for hidden surveillance measures currently used.

green_dee 4y ago

Well, every model has its errors, and intrusive criminal "prevention" is self-fulfilling. Just imagine, everyday 6AM an officer come at your door to check your apartment "to prevent" your crimes, suddenly every random guy in the street looks suspicious and spying on you, then some officers just randomly walks by your workplace, police stop you everytime you travel... This would eventually lead to an insult to the mafia guy and then boom you have already commited contumacy and the model was right! You are a criminal and you should be punished!

lostlogin 4y ago
It’s a little like a machine controlled ‘stop and frisk’.
- weard_beard 4y ago
  But would I have done it if the police weren’t following me? What if the computer is identifying at risk paranoid schizophrenics likely to be pushed over the edge by noticeable increases in their personal surveillance? Frankly that seems more plausible than telling the future
  - lostlogin 4y ago
    I don’t make any claim at all for its capability. But if you load a dataset into some sort of ML algorithm right now, with the data coming from current policing statistics, you are going to be perpetuating some pretty dark stuff. Police target various groups and minorities.
    - weard_beard 4y ago
      "The data is racist" will be a common refrain in this AI revolution.
naijaboiler 4y ago
you just described the minority experience in many parts of US

_s_a_m_ 4y ago

As if they give a shit. This was the case since decades they doing similar practices.

LatteLazy 4y ago

For small crimes. For bigger crimes (limit TBC) they're fine. And I think that is ok...

omgomgomgomg 4y ago

I speak German and I have just read the text, it is vastly complicated court and lawyer speak. But it is clear that this legislature is pushing for crime prevention based on aggregated data from computer systems(where from, exactly?) The arguments are that a manual extraction and collection would be too slow and that this should be limited to only a handful of severe potential crimes. As a prevention rather than mitigation technique. This is quite bad. Now the objection is not purely because the data is coming from computers, but because it will reveal data on people tangentially connected to the targets. This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic. It is very clear that these things are pushed by rather smart minds, matter experts and this makes it clear they do not care about the innocent peoples privacy, as they could not possibly be so ignorant to overlook this matter. Justice even the conventianal way spawns injustices, now the lazy government wants to automate and scale their procedures. W.t.h.

c4mpute 4y ago
> innocent peoples privacy It isn't just about their privacy but also their very freedom. Being in some police database might very well land you in the crosshairs of some investigation and related niceness like searching your home, taking you in for questioning and keeping you there because you didn't say something...
- omgomgomgomg 4y ago
  True, I equate privacy and freedom, give or take. Anyone who ever end up an their radar would probably never be removed from that,and never even know. It is highly offensive that the German government pretends it does not know where the crime is happening. How about they deal with a few of these clans numbering 100s of people? This can be done , has to be done the convential way, as these clans will not be as stupid as leave digital traces, as computer illeterate as they might be. The government has not even defined who they are targeting, they just use the commonly thrown around terms.
  - 4y ago

thenaturalist 4y ago

Super interesting ruling in my book. Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me. Basically the court mandates that any future laws passed for legalizing automated data analysis must explicitly 1. For any given crime to be automatically analysed, define the data model and define appropriate analysis techniques or methods. Those laws should be interesting reads. 2. For any one analysis happening, document and later publish to the public domain the reason for initiating an analysis, the methods chosen, the criteria for labeling someone as "guilty/innocent" - basically the entire process so that the analysis is auditable end to end. Again, this is not a suggestion in the ruling, but a requirement. This is really good news for personal liberties in my book. Stark contrast to US/ China data mining on citizens.

jll29 4y ago
An earlier poster wrote: > This is so wrong in so many ways and the people who are pushing from this have learned nothing from the stasi and nazi regimes, or perhaps they are nostalgic. Whereas you say: >Skipping ahead to the fact that automated data analysis is not prohibited per se, just needs to be reasonably targeted, the two most interesting paragraphs are the last ones for me. I think this reads like the court curtailed reasonably broad data grabbing behaviors such as those Snowdon has revealed happened illegally and are still happening in the US. It still permits effective, focused/targeted data mining. So at least the court learned from the Nazi/Stasi times but also from the times of left-wing terrorism ("Rasterfahndung" during the Bader-Meinhoff terrorist gang) - i.e., that these techniques are effective yet must not be applied to broadly. If this reading is correct (I'm not a legal expert), then the main danger is that lawmakers are incapable (or unwilling) to implement requirements (1.) and (2).
- omgomgomgomg 4y ago
  I agree fully, I am struggling to imagine how they can go and push such a legislature infront of this governing body with a straight face. This is more dangerous than most people would think, once something like this is codified into law, many eu countries will follow suit, just for good measure. What are the motives and who benefits from this? This idea was born somewhere and writing all the drafts costs a lot of money, who was footing this bill and why?

shasts 4y ago

So how would fraud prevention be interpreted? For eg: denying service from a German retailer depending on the ML prediction result of the transaction being classified as fraud probable(Germany has pay by invoice), would also be constitutional or not? Or credit score/risk rating using ML, trained on a feature extracted from parameters like Zip code, ethnicity could also classified as automated data analysis for prevention of criminal acts, right? Or this is only applicable to police?

rom-antics 4y ago
In general a retailer can refuse any customer for any reason (unless you're discriminating against a protected class) The reason they do this isn't to fight crime, but to save the business from losing money. So I don't see how a ruling about preventing crime would apply.
thenaturalist 4y ago
Only applicable to police use.

possiblelion 4y ago

I think it's crazy that people are considering "linking previously unconnected automated databases and data sources in analysis platforms and permitting systematic access of data across sources through searches." as something like Minority Report. The truth is that Law Enforcement investigations are currently taking information from dozens of silos/databases, manually copy&pasting them into Excel and Word and then finally making a pdf or powerpoint to share the results. This is obviously a really inefficient way to do things and wastes a lot of taxpayer money. A technical solution that would allow LE to crosslink data they already have is a no-brainer. P.S: Very biased view, as I'm building/selling exactly this tool :)

beecafe 4y ago
How do you sleep at night?
- suction 4y ago
  [dead]
- mistrial9 4y ago
  in an expensive bed, with high-tech security all around of course!
thenaturalist 4y ago
> A technical solution that would allow LE to crosslink data they already have is a no-brainer. A technical solution that is Open Source, that falls under federally mandated criteria for how it can be legally used, that automatically documents any and all access and makes the entire audit log part of any output result so that all happenings are easily auditable by 3rd parties... might be a no brainer. Anything else I'm highly skeptical. If going after syndicated and or international crime, fine. First feature I'd like to see is linking Finanzamt with LE and go after money laundering - Germany being an absolute paradise to the detriment of everyone else. If it takes manual copy paste to protect ordinary citizens from overreaching police, I welcome that technical limitation. Very biased, as I am an ordinary citizen.

DubiousPusher 4y ago

Every human needs to understand that having a computer look at data representing you is a search.

kqr 4y ago

The one argument I rarely see against automated analysis is that any analysis has a false positive rate. If you hire 1,000 humans to do that analysis there's simply a limit to how many errors they can make in total. If you introduce automation doing the same thing at a scale corresponding to 10,000× the human capacity, you will have 10,000× as many errors. This will violate Blackstone's ratio and erode trust in the system.

ravivooda 4y ago
TIL: Blackstone's ratio

arvidkahl 4y ago

"Dieser Eingriffsanlass bleibt angesichts der besonders daten- und methodenoffen formulierten Befugnisse weit hinter der wegen des konkreten Eingriffsgewichts verfassungsrechtlich gebotenen Schwelle einer konkretisierten Gefahr zurück." This roughly translates into "there has to be a sufficiently concrete danger to warrant such a ambiguously formulated infringement." I truly appreciate this sentiment. Any collection or analysis of data "just because there might be something in there" has to be weighed against the right of everyone that is being data-mined. This freaked me out about Minority Report when I first watched it: those precogs are essentially hacking into the future memories of everyone involved in _and by happenstance surrounding_ the lives of future criminals and their crimes. Mass-scale datamining is effectively the real-world equivalent of telepathy. And I want neither precogs nor the government in my mind.

binarymax 4y ago

In the US - this should also be the case. It should fall under the 4th amendment as unreasonable search and seizure. But the NSA, and probably other agencies, has been doing it for years anyway.

gotoeleven 4y ago

If it prevents even one misgendering it will all be worth it.

Rallen89 4y ago
There has to be a poe's law equivalent for gender issues at this point, if the article is anyway political someone's gonna mention it because uhh they gotta be mad at someone
- justincredible 4y ago
  [dead]

nivenkos 4y ago

Why? IMO we should use facial recognition, digital ID, a cashless society and everything at our disposal to stamp out crime. It's the working class who pay the price at the moment. We get burgled, mugged and killed, meanwhile the wealthy lawyers and politicians can pontificate over constitutionality from their gated communities and private vehicles. It's insane that the EU is trying to ban automatic facial recognition with the terrible AI act - https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC020 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A... - nevermind the sheer amount of bureaucracy it introduces for startups (a government register of "possibly harmful" AI - wtf?).

flangola7 4y ago
/s?
- nivenkos 4y ago
  No, bureaucracy and crime are awful. Bear in mind this is the same country that bans Google Street View (and on the road to ban nuclear power and GM crops). People here support this ban because they think it's "harmful", but how long until they think some technology you rely on is also harmful? GPS? Phone cameras or Google Lens? Encryption - https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-mass-surveillance-of-the-eu/ https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-ma... ? Technology is the future, and we should embrace it wherever possible - crime prevention, education, healthcare, nuclear fusion, genetic engineering, etc.
  - flangola7 4y ago
    > No, bureaucracy and crime are awful. My best friend is a sex worker. I use cannabis to control my anxiety. Neither of those are legal. Looking historically, virtually every human rights movement required illegal direct action. Where do we fit into your cashless panopticon society? The worst crimes I'm worried about are the ones that will never be illegal, the ones that benefit the powerful and rich. At-scale invasion of privacy is only one such tremendous crime, one against against human rights and dignity, but barely against the law. > Bear in mind this is the same country that bans Google Street View (and on the road to ban nuclear power and GM crops). > People here support this ban because they think it's "harmful", but how long until they think some technology you rely on is also harmful? GPS? Phone cameras or Google Lens? Encryption - https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-mass-surveillance-of-the-eu/ https://mullvad.net/en/blog/2023/2/2/stop-the-proposal-on-ma... ? Some of those are genuinely harmful (street view in particular for its privacy invasion - note it is not banned but simply very unpopular) so I'm not sure what your point is. Also, you should send your ideas to the Mullvad blog authors and get their input. I think you'll find that the overlap between people who support universal end to end encryption and people who support an all-seeing police state is very small, as they are fundamentally incompatible ideas. > Technology is the future, and we should embrace it wherever possible - crime prevention, education, healthcare, nuclear fusion, genetic engineering, etc. Technology is not always good all of the time. The past century and even past decade are full of examples of net harms caused by technology that could have been avoided if we developed them in a more intentional controlled manner. A eccentric example to make the point: If there is a technology that can trigger a supernova of the sun simply by mixing water, sugar, gasoline, and bleach together in a certain ratio, then it would be extremely important we not conduct research into that technology until we have other technology that allows us to survive a supernova or can prevent that mixture from being created anywhere on Earth. Even if there are advantages of the technology (maybe a different ratio causes all humans to have 5x longer lifespans and eliminates all disease and suffering, and we gain intelligence allowing us to begin colonizing other planets within 3 years) the risk is just too high that it will be misused accidentally or maliciously until we have a way to handle it. https://en.wikipedia.org/wiki/Differential_technological_development https://en.wikipedia.org/wiki/Differential_technological_dev...

shadowgovt 4y ago

In the medium-term, the cultural shear between the US and Germany will continue to be highlighted by decisions like this. Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority. America leans lightly towards more and more centralization of data because they have the experience in recent memory of what individuals who "slip through the cracks" can do. So in Germany we see decisions like this and in the US we see police using privately-curated DNA databases to defrost the cold case of a serial killer, and they're lauded as heroes. (It's worth noting, of course, that this is all relative. Relative to most of Europe, the US is starting from a place of massive decentralization and minimal individual-tracking. Some people still find social security numbers controversial).

sneak 4y ago
> Germany leans heavily to more and more privacy because they have the experience in recent memory of what a government can do with vast information collection and centralized authority. I theorize that in a hundred years or so, North Americans will have learned this lesson the hard way and will have atrocities in recent history, too. I'm sad I won't see the day when Bluffdale is repurposed as a memorial to state surveillance, much like the Stasi headquarters are now in Berlin.
bakuninsbart 4y ago
Can't find the study now, but there was a sociological survey of software engineers in the US and Europe, one really interesting thing was that the top social concern when implementing solutions for Americans was fairness, for Germans it was privacy. The other concern didn't even make the top 5 for either.
qikInNdOutReply 4y ago
If china ever gets democratic, a new mighty privacy player will appear. Chinas citizens have the same nasty experience as east germans had with the stasi and the poles, chechs with the cheka. They have been spied upon, reported upon by there fellow citizen and have been victims of "zersetzung". Aka social engineering to destroy a citizen. https://www.stasi-unterlagen-archiv.de/mfs-lexikon/detail/zersetzung/ https://www.stasi-unterlagen-archiv.de/mfs-lexikon/detail/ze... Link is german: Shows how denunciation aka controlled leaking of private life like affairs was used, to destroy the lifes of resisting citizens. PS: FB and Google could in theory use similar instruments against citizens that put pressure on them. Nobody could ever catch them at it. If the ccp ever gets swept away and replaced by a new better china, the resulting parties will outdo one another to promise freedom and privacy.

kilgnad 4y ago

This concept was explored in a stephen spielberg movie. Minority report. Good movie. Watch it. Stars Tom Cruise and has mission impossible vibes. In the end the government cancelled the program because of free will. A person always has the choice to change his actions because his actions aren't predetermined.

super256 4y ago

I think it’s important to mention Palantir in this context.

forevergreenyon 4y ago

time to make amendments! just like once upon a time it was constitutional to own slaves and now it isn't just think about the monetary cost-savings! (nevermind the human misery)

Eduard 4y ago

What do they mean with "first alternative"?