3 ms·
This article is a prime example of law enforcement passing the buck of responsibility for their own actions to a black box system by claiming that the system's
by heavyset_go 5y ago
This article is a prime example of law enforcement passing the buck of responsibility for their own actions to a black box system by claiming that the system's output is "unbiased".
> “You may wonder why you were enrolled in this program,” the letter continues. “You were selected as a result of an evaluation of your recent criminal behavior using an unbiased, evidence-based risk assessment designed to identify prolific offenders in our community. As a result of this designation, we will go to great efforts to encourage change in your life through enhanced support and increased accountability.”
> Last year, a Tampa Bay Times investigation revealed that the Sheriff’s Office creates lists of people it considers likely to break the law based on criminal histories, social networks and other unspecified intelligence. The agency sends deputies to their homes repeatedly, often without a search warrant or probable cause for an arrest.
Now law enforcement gets to absolve themselves of accusations of bias or targetting because they assume that their computers can't be "biased", even though they only buy systems that confirm their own hunches about which people are criminals or not.
If anyone complains, they can say they were only doing what the supposedly unbiased system told them to do, after all.
- op00to 5y agoIt’s hilarious because the shit they’re basing their model on (arrests/contacts with police) is biased as hell
- anonydsfsfs 5y agoEven if you haven't had any contact with the police, they can still put you on the list if they suspect you committed a crime. The manual they follow[0] uses a point-based system, and "suspicions of an offense" counts for half the points of actually committing the offense: > • Half of the applicable point values are awarded for suspicions of an offense listed in 1-4. [0] https://s3.documentcloud.org/documents/20412738/ilp_manual012918.pdf https://s3.documentcloud.org/documents/20412738/ilp_manual01...
- heavyset_go 5y agoLooking at page 13, they talk about their harassment list for at-risk youth, and mention these as 'identifiable risk factors': - Hanging around in public - Being a victim of personal crime - Coming from broken homes Page 72 show that they use records of children who experience physical abuse, sexual abuse, neglect, or witness domestic violence to put them on lists. Page 73 shows the "At-Risk Youth Spreadsheet" which is both terrifying and strikingly banal.
- jtbayly 5y ago… and well known. For example, we have known that broken homes lead to all sorts of problems a much greater proportion of the time for ages. And one of the things that becomes more likely is crime. Not that that makes a computer algorithm’s predictions true. Just saying that it’s not terrifying to point out that there really are such things that increase risk that we are well aware of.
- easterncalculus 5y agoThis in my opinion is a point that brings a lot of well-needed nuance to this issue. All or at least many of these facts or assumptions are all relatively reasonable or well-studied, that past criminal records and traumatic upbringings can lead to crime, but when put together under the dubious question of is the person a criminal can be greater than the sum of their parts. A lot of people come from broken homes or experience trauma and never commit a(nother) crime. The fact that these assumptions are standard practice has only legitimized taking a black-box algorithm at its word. It's not all just bias, but this kind of system certainly can incentivize it when making seemingly authoritative predictions.
- vkou 5y agoThe nuance doesn't matter, though. If you tell a cop that person A is 1% likely to be guilty of a crime, and person B is 3% likely to be guilty of a crime, they are going to focus their efforts on person B - despite both people being more likely than not, to be innocent. When being harassed by police is itself sufficient to turn a person into a criminal, or ruin their life, you end up in a situation where everyone is doing what is 'reasonable', but leading to outcomes that are both unfair, and even negative overall, to society.
- mcguire 5y agoNot to mention, "The Times found being named a Sheriff’s Office target could have serious consequences. Deputies showed up at homes at all hours of the day and night, writing tickets for violations like overgrown grass and making arrests for any reason they could find. By 2020, some 1,000 people had been ensnared. About 100 were 18 years old or younger." "Since we have determined that you are a criminal, we will ensure that you are a criminal."
- wnevets 5y ago> writing tickets for violations like overgrown grass If I didn't know better I would think this story is from The Onion.
- websites2023 5y agoThis is precisely why I voted against abolishing cash bail in California last year. Because they were going to replace it with some "algorithm." I hate cash bail, I think it's garbage, but this black box approach is worse.
- x86_64Ubuntu 5y agoAlgorithms can be okay, as long as they are transparent. The problem with using things like AI is that the parameters or the algorithm are often hidden from the harmed individuals. And the law enforcement community can kick the can down the road by saying "but the software says...".
- kevin_b_er 5y agoExcept this is a Florida county.
- tgsovlerkhgsel 5y agoThis is exactly why one of the lesser known provisions of GDPR (https://gdpr-info.eu/art-22-gdpr/ https://gdpr-info.eu/art-22-gdpr/) addresses "automated individual decision making" (although there are likely exceptions for government activity, but they may require a law).