5 ms·

I am a little confused by this. Firstly, Gizmodo is reporting on somebody else's investigation: > A new joint investigation by The Markup and Wired... And whe

I am a little confused by this. Firstly, Gizmodo is reporting on somebody else's investigation: > A new joint investigation by The Markup and Wired... And when I go to the page about actual investigation by The Markup [1] > Our investigation stopped short of analyzing precisely how effective Geolitica’s software was at predicting crimes because only 2 out of 38 police departments provided data on when officers patrolled the predicted areas. Geolitica claims that sending officers to a prediction location would dissuade crimes through police presence alone. It would be impossible to accurately determine how effective the program is without knowing which predictions officers responded to and which ones they did not respond to. Also, later in the article > Plainfield officials said they never used the system to direct patrols. Given all this, it's somewhat simplistic to say it's "pretty terrible at predicting crimes", even though that makes for a good clickbait headline. It seems that the software was intended to identify high-crime areas that to target for patrolling, which doesn't seem like a huge problem to me -- but it seems like the software was never actually used as intended in the first place. ---------------------------------------- [1] https://themarkup.org/prediction-bias/2023/10/02/predictive-policing-software-terrible-at-predicting-crimes https://themarkup.org/prediction-bias/2023/10/02/predictive-...

fnordpiglet 3y ago
They used advanced software to predict where clicks will happen with bait.
next_xibalba 3y ago
> Gizmodo is reporting on somebody else's investigation This is Gizmodo's business model. It is a glorified blog. They don't do original reporting. Gizmodo was originally a property of Gawker, and thus the model is essentially commentary and opinion on the reporting generated by other organizations. Because Gizmodo is really just an opinion/editorial blog, it doesn't really attempt to provide unbiased or fact driven reporting. So it is then no surprise that the actual facts align quite poorly with the headline.
- dang 3y ago
  We've since moved the comments in to a different thread. (see https://news.ycombinator.com/item?id=37756211 https://news.ycombinator.com/item?id=37756211)
Swizec 3y ago
> identify high-crime areas that to target for patrolling, which doesn't seem like a huge problem to me The wonderful Weapons of Math Destruction has a chapter on how this leads to a self reinforcing loop. More crimes means more police presence. More police presence means more recorded crimes. More crime data means more police …
- PhilipRoman 3y ago
  If only we had statistical tools to compensate for this...
  - llbeansandrice 3y ago
    What statistical tools compensate for the generally racist and shitty nature of US police officers?
    - pxmpxm 3y ago
      I don't expect to read this level of asinine on HN.
  - KingMob 3y ago
    There are limits to how far stats can compensate for missing data, especially non-random missing data.
- User23 3y ago
  I hate how wet streets cause rain.
- justrealist 3y ago
  Yes if you choose to use your data like an idiot then you will generate idiotic conclusions. This does nothing to negate the value of data used intelligently.
  - Swizec 3y ago
    This is a description of how it is currently being used in practice. Yes you can use data well. It is unfortunately not being used well.
AnimalMuppet 3y ago
But even if they had used it as intended, how could you tell whether it works? How do you tell the difference between "there were going to be crimes there, but you patrolled there, and because you patrolled, there were no crimes" and "there were not going to be crimes there, whether you patrolled or not, so your patrol did exactly nothing."
- jncfhnb 3y ago
  Experimentation and tracking results over time.
- waveBidder 3y ago
  well, if you're a guest on https://www.probablecausation.com/ https://www.probablecausation.com/, then you would randomly assign teams to patrol using this software or not,and compare the changes in rates. sounds like geolitica isn't very careful or interested in results.
  - lazide 3y ago
    If you’re selling tiger-repellent rocks, the last thing you need is a solid controlled experiment to get in the way of separating rubes from their cash.
  - deleted 3y ago
    [deleted]
- cgriswald 3y ago
  There are really two questions here, which should be tested separately: (1) Is the software significantly predictive? Test the software's predictions against actual crime in areas with no patrols. (2) Are patrols effective deterrents to crime? Observe the area in question with and without patrols. (I wouldn't be surprised to learn there are already such studies.) If both (1) and (2) yield positive results, you can then use the software to direct patrols and see if the method itself is effective at reducing crime; which would also serve to further confirm (2). Of course, even if only (1) yields positive results, there would probably be other benefits to using the software to direct patrols, like reducing response times (which could also be tested).
_Algernon_ 3y ago
Maybe the link for this submission should be replaced with one of the originals? The closer to the source the better IMHO
josefresco 3y ago
If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated. The know it doesn't work. And it looks like they had some data: >We examined 23,631 predictions generated by Geolitica between Feb. 25 to Dec. 18, 2018 for the Plainfield Police Department (PD). Each prediction we analyzed from the company’s algorithm indicated that one type of crime was likely to occur in a location not patrolled by Plainfield PD. In the end, the success rate was less than half a percent. Fewer than 100 of the predictions lined up with a crime in the predicted category, that was also later reported to police.
- bumby 3y ago
  >If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated. That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. So it's still what I would expect even if it was reasonably good. I wish the article would have provided more details about why the PDs chose not to use it. Was it because it was bad at predictions? Cumbersome to use? Glitchy? It reminds me of a project I was involved with that used a "real-time" computational fluid dynamics model to optimize datacenter air-conditioning. Management was big on the hype, but if you paid attention, the system was routinely unplugged from the actual control system because the facility engineers just found it too difficult to work with.
  - vkou 3y ago
    > That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. Why on Earth would we want people who have a poor grasp of the law, paired with the means and the license to arrest and kill be transparent about their work..?
    - bumby 3y ago
      We can both agree that transparency is the ideal and still recognize that’s not how incentives are aligned. In general, there are far more downsides and risks to PDs regarding open sharing of information so they generally should be expected to act accordingly. E.g., I’d like to live in a works where lawyers are concerned with justice rather than “winning” but I’m not shocked when they act like winning is the goal.
gsdofthewoods 3y ago
The things you're citing are referring to two different investigations. One is the most recent one that only centered on Plainfield, NJ, which is what Gizmodo is reblogging. The one where they did not investigate Geolitica's effectiveness at predictions was a broader investigation in 2021.
gwern 3y ago
The error is more fundamental. Even if they had some number like 'only 1% of crimes were predicted and that's bad', that's a right answer to a wrong question. Why do they think 1% is not good enough? How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything. (The right number is probably extremely small, because crime is very bad {{citation needed}} and even a small chance of prevention is useful.)
- wolfram74 3y ago
  the "right number" is strongly dependent on what the consequences of false positives are. If you're comfortable pulling numbers from thin air, 0.5% of these guided patrols lead to prevented crimes, but 10% lead to arrests of simply suspicious looking people (suspicious on grounds of being around where crime is predicted) and then while they're being detained are late for a job and get fired. Is 20 people getting fired worth preventing one crime? Say, a catalytic converter being stolen, since "crime" is very bad, not, say, murder particularly.
  - smogcutter 3y ago
    Further, who in the general vicinity of a precrime report is “suspicious looking”? Who gets their day, if not their life, turned upside down? Thankfully, police are notoriously fair-minded and empathetic, and never arbitrary, deceptive, or biased when it comes to probable cause.
- magicalist 3y ago
  > How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything. Uh, seems a lot like it should be the government answering that question before spending tax money on it and subjecting real people to the uncaring US justice system just because crime is bad so we have to do something, even if that something is ML snake oil?
- knallfrosch 3y ago
dang 3y ago
Thanks - we've since merged the comments hither, since this submitter had the original source. Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html (the parent was originally posted to https://news.ycombinator.com/item?id=37753079 https://news.ycombinator.com/item?id=37753079)

fnordpiglet 3y ago

They used advanced software to predict where clicks will happen with bait.

next_xibalba 3y ago

> Gizmodo is reporting on somebody else's investigation This is Gizmodo's business model. It is a glorified blog. They don't do original reporting. Gizmodo was originally a property of Gawker, and thus the model is essentially commentary and opinion on the reporting generated by other organizations. Because Gizmodo is really just an opinion/editorial blog, it doesn't really attempt to provide unbiased or fact driven reporting. So it is then no surprise that the actual facts align quite poorly with the headline.

dang 3y ago
We've since moved the comments in to a different thread. (see https://news.ycombinator.com/item?id=37756211 https://news.ycombinator.com/item?id=37756211)

Swizec 3y ago

> identify high-crime areas that to target for patrolling, which doesn't seem like a huge problem to me The wonderful Weapons of Math Destruction has a chapter on how this leads to a self reinforcing loop. More crimes means more police presence. More police presence means more recorded crimes. More crime data means more police …

PhilipRoman 3y ago
If only we had statistical tools to compensate for this...
- llbeansandrice 3y ago
  What statistical tools compensate for the generally racist and shitty nature of US police officers?
  - pxmpxm 3y ago
    I don't expect to read this level of asinine on HN.
- KingMob 3y ago
  There are limits to how far stats can compensate for missing data, especially non-random missing data.
User23 3y ago
I hate how wet streets cause rain.
justrealist 3y ago
Yes if you choose to use your data like an idiot then you will generate idiotic conclusions. This does nothing to negate the value of data used intelligently.
- Swizec 3y ago
  This is a description of how it is currently being used in practice. Yes you can use data well. It is unfortunately not being used well.

AnimalMuppet 3y ago

But even if they had used it as intended, how could you tell whether it works? How do you tell the difference between "there were going to be crimes there, but you patrolled there, and because you patrolled, there were no crimes" and "there were not going to be crimes there, whether you patrolled or not, so your patrol did exactly nothing."

jncfhnb 3y ago
Experimentation and tracking results over time.
waveBidder 3y ago
well, if you're a guest on https://www.probablecausation.com/ https://www.probablecausation.com/, then you would randomly assign teams to patrol using this software or not,and compare the changes in rates. sounds like geolitica isn't very careful or interested in results.
- lazide 3y ago
  If you’re selling tiger-repellent rocks, the last thing you need is a solid controlled experiment to get in the way of separating rubes from their cash.
- deleted 3y ago
  [deleted]
cgriswald 3y ago
There are really two questions here, which should be tested separately: (1) Is the software significantly predictive? Test the software's predictions against actual crime in areas with no patrols. (2) Are patrols effective deterrents to crime? Observe the area in question with and without patrols. (I wouldn't be surprised to learn there are already such studies.) If both (1) and (2) yield positive results, you can then use the software to direct patrols and see if the method itself is effective at reducing crime; which would also serve to further confirm (2). Of course, even if only (1) yields positive results, there would probably be other benefits to using the software to direct patrols, like reducing response times (which could also be tested).

_Algernon_ 3y ago

Maybe the link for this submission should be replaced with one of the originals? The closer to the source the better IMHO

josefresco 3y ago

If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated. The know it doesn't work. And it looks like they had some data: >We examined 23,631 predictions generated by Geolitica between Feb. 25 to Dec. 18, 2018 for the Plainfield Police Department (PD). Each prediction we analyzed from the company’s algorithm indicated that one type of crime was likely to occur in a location not patrolled by Plainfield PD. In the end, the success rate was less than half a percent. Fewer than 100 of the predictions lined up with a crime in the predicted category, that was also later reported to police.

bumby 3y ago
>If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated. That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. So it's still what I would expect even if it was reasonably good. I wish the article would have provided more details about why the PDs chose not to use it. Was it because it was bad at predictions? Cumbersome to use? Glitchy? It reminds me of a project I was involved with that used a "real-time" computational fluid dynamics model to optimize datacenter air-conditioning. Management was big on the hype, but if you paid attention, the system was routinely unplugged from the actual control system because the facility engineers just found it too difficult to work with.
- vkou 3y ago
  > That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. Why on Earth would we want people who have a poor grasp of the law, paired with the means and the license to arrest and kill be transparent about their work..?
  - bumby 3y ago
    We can both agree that transparency is the ideal and still recognize that’s not how incentives are aligned. In general, there are far more downsides and risks to PDs regarding open sharing of information so they generally should be expected to act accordingly. E.g., I’d like to live in a works where lawyers are concerned with justice rather than “winning” but I’m not shocked when they act like winning is the goal.

gsdofthewoods 3y ago

The things you're citing are referring to two different investigations. One is the most recent one that only centered on Plainfield, NJ, which is what Gizmodo is reblogging. The one where they did not investigate Geolitica's effectiveness at predictions was a broader investigation in 2021.

gwern 3y ago

The error is more fundamental. Even if they had some number like 'only 1% of crimes were predicted and that's bad', that's a right answer to a wrong question. Why do they think 1% is not good enough? How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything. (The right number is probably extremely small, because crime is very bad {{citation needed}} and even a small chance of prevention is useful.)

wolfram74 3y ago
the "right number" is strongly dependent on what the consequences of false positives are. If you're comfortable pulling numbers from thin air, 0.5% of these guided patrols lead to prevented crimes, but 10% lead to arrests of simply suspicious looking people (suspicious on grounds of being around where crime is predicted) and then while they're being detained are late for a job and get fired. Is 20 people getting fired worth preventing one crime? Say, a catalytic converter being stolen, since "crime" is very bad, not, say, murder particularly.
- smogcutter 3y ago
  Further, who in the general vicinity of a precrime report is “suspicious looking”? Who gets their day, if not their life, turned upside down? Thankfully, police are notoriously fair-minded and empathetic, and never arbitrary, deceptive, or biased when it comes to probable cause.
magicalist 3y ago
> How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything. Uh, seems a lot like it should be the government answering that question before spending tax money on it and subjecting real people to the uncaring US justice system just because crime is bad so we have to do something, even if that something is ML snake oil?
knallfrosch 3y ago

dang 3y ago

Thanks - we've since merged the comments hither, since this submitter had the original source. Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html (the parent was originally posted to https://news.ycombinator.com/item?id=37753079 https://news.ycombinator.com/item?id=37753079)