7 ms·

Police Data Accessibility Project

78 points by polm23 6y ago

zepto 6y ago
“Are you affiliated with a political party or have a political agenda? No. Our only motivation is to provide trusted data in an age of disinformation.” If true, this is an awesome project.
- chaps 6y ago
  They're already working with one political candidate: https://pdap.atlassian.net/browse/PDAP-149 https://pdap.atlassian.net/browse/PDAP-149
  - zepto 6y ago
    You have linked to a message saying: “Any chance you guys have data on NJ? I'm running for office there and was looking to get some police misconduct data to double check a couple things” It is not clear how this means that they are working with a political candidate, or what that means to you. Can you explain what conclusions you are drawing from this message?
    - chaps 6y ago
      From their slack, a political candidate posted that they were interested One of two of their volunteer scrapers said that he would drop his work in California to work on New Jersey instead. I guess "working with" is bad language, though "interested in dropping other work in favor of requests by political candidates" is more accurate.
      - zepto 6y ago
        Providing data to people who want it seems like the reason the project exists, so it seems natural for them to respond to users asking for data. It’s not clear to me how you think this relates to their claim: “Are you affiliated with a political party or have a political agenda?“
        chaps 6y ago
        I can see why others would disagree with me here, but I personally see no difference between having party affiliation and having 50% of your relevant volunteers doing work that helps a politician achieve their goals.
        deleted 6y ago
        [deleted]
        zepto 6y ago
        It seems reasonable to assume that 100% of their work will help politicians achieve their goals. Data on NJ was clearly already part of their plan. I don’t see how this has anything to do with party affiliation. Can you explain how it does?
        luxuryballs 6y ago
        I can take a stab at it, there are potentially political motives to why a politician might want data that would bolster the perception of police misconduct.
        zepto 6y ago
        Of course, but that’s an inevitable use of this data, and is about the politician not the project.
        chaps 6y ago
        As far as I can tell, you're being intentionally uncharitable. There's a difference between doing work in isolation that a politician uses for their own political goals and doing work that specifically and intentionally helps a politician on their request. If you were to drop all of your ongoing side projects to work with a political candidate that asked for your help, would you really try to argue that you're not at least somewhat tied to that candidate?
        6y ago
  - jjulius 6y ago
    "Working with" feels like a loaded phrase. Someone created that ticket requesting that data 14 minutes ago, and the ticket has yet to even be assigned to anyone (at the time of my post). I'm not sure how you can conflate this project's lack of a response to a request as "already working with".
  - Hasnep 6y ago
    A political candidate asked for help accessing some data, not the same thing as affiliating with a political party.
ClosedPistachio 6y ago
Some States have comprehensive databases. Here's one for New Jersey: https://force.nj.com/ https://force.nj.com/
- psychlops 6y ago
  "most comprehensive" is different from comprehensive. Curious, I went to take a look. The latest set of data was release November 2018 and covered the years 2012-2016 purchasable for $200.
- electricBllue 6y ago
  Do you happen to have the links for any other states?
- rhcom2 6y ago
  Just to clarify, The Force Report was the result of over a year of work by journalists that had to fight tooth and nail for every FOIA request and they freely acknowledge it is imperfect. To me at least it is something our government should be already doing.
glenda 6y ago
An issue I see is that the public data police departments put out is often only what they choose to, meaning any analysis of that data is potentially biased to an unknown degree. I suppose adding FOIA data into the mix could improve that, but doesn’t sound easy since those are usually delivered as PDFs.
- zepto 6y ago
  They could and should document and account for this somewhere in the data set. It wouldn’t solve the problem but it would allow the bias to be visible.
- deleted 6y ago
  [deleted]
- mistermann 6y ago
  I think this "the dog that didn't bark" phenomenon may be a much more important phenomenon than meets the eye. People's perception of the world is a function of their observations of reality - for starters, the vast majority of people don't even have any exposure to things like police data (other than the typically misleading narratives that they might encounter in the news), but then even the small majority of people who do dig deeper into the details, these people too will leave with a distorted perception of reality if only a subset of the data is revealed, and they do not know that it is only a subset. The end result of all of this complexity is that people think the world is one way, but it is actually another, and the vastness of the gap between the two could be anywhere from small and insignificant, to gigantic and crucially significant, if one applies this idea to not just police data, but everything (the entirety of reality is subject to this flaw, and many others that we do not realize).
  - TeMPOraL 6y ago
    Add the fact that even the businessmen and regulators, influential people, politicians and judges, they all build their image of the world mostly on the same sources as everyone else - and it's becoming worryingly apparent that all the control systems in societies are operating with very low visibility into the things they're controlling. Imagine a PID controller keeping a boiler from overheating. Now imagine the temperature input to that controller is a thermometer mounted to the door of a metal cabinet next to the boiler, and thus it reports anything related to the boiler temperature only when somebody is looking for something in that cabinet (when the opened metal doors touch the boiler and conduct heat). Doesn't sound like a good control system.
cyberlab 6y ago
Is this an American project that sources & aggregates data from American police forces, or is it some general thing that wants to draw from multiple countries? (Because the site doesn't mention anything about locale).
- _joel 6y ago
  I thought the same, but there are apparently 17,985 police forces in USA[1] so it looks to be American centric. I think they should list this in their FAQ or make it more obvious. [1] https://en.wikipedia.org/wiki/Law_enforcement_in_the_United_States https://en.wikipedia.org/wiki/Law_enforcement_in_the_United_...
jimbob45 6y ago
The police force is losing much of its ability to enforce the law. Young casual political observers praise this but we’ve already seen this happen in the 1960s and then culminate in Bernhard Goetz once everyone was fed up with drugged low-lifes bringing down the quality of life of otherwise hard-working individuals. There must surely be a middle road here.
- alexilliamson 6y ago
  There are lots of hard working drug addicts as well as lazy sober people. Quality of life is complicated.
- throwawaysea 6y ago
  The reduction of quality of life is already happening in permissive cities like SF, Portland, and Seattle. Public spaces are taken over by those breaking the law, and yes that very often includes “drugged low-lifes”, and it brings a host of other problems with it. As examples: a popular jogging path near me is now being avoided by women I know because men from the encampments that have taken over the park the path runs through, and sexually harass them sometimes. There are frequent posts on Next Door where some homeless person will be on camera stealing things out of peoples porches or yards, to feed their lifestyle, and police simply don’t have the resources to solve those crimes and deter further criminal acts. Graffiti and littering have increased substantially, despite littering in green spaces carrying a big fine for any law abiding citizens. Criminals that are arrested are mostly simply released in the spirit of restorative justice or similar policies, going on to repeat the same crimes against new victims. This last year has involved a lot of blind hatred of policing and poorly thought out calls to reduce police funding all based on very rare anecdotal incidents which have been spun as data. The reality is that police forces in America have hundreds of millions of interactions with citizens and almost none of them are problematic. It do surprise that surveys show public perceptions of police brutality are incredibly skewed, far from resort (https://nypost.com/2021/02/27/cases-of-police-brutality-against-black-people-are-overestimated/ https://nypost.com/2021/02/27/cases-of-police-brutality-agai...). I can only describe the current trend as a mass hysteria driven by biased news media and dedicated true believers, who organize on social media and real life, and end up influencing the masses into believing the same things.
tyingq 6y ago
On the flip side, this will likely expose a lot more arrest data, and complicate the "innocent until proven guilty" thing that's mostly not true.
- chaps 6y ago
  This group seems to have turned into a group that's only interested in scraping "criminal records" and "court records" over police accountability records, and believes that cop names should be removed from scraped records. They've also openly discussed inviting cops and FBI employees into their slack to discuss scraping. Consider that "Police Data Accessibility Project" is another way of saying "We only make available data that the police want us to make available.". The project itself has tons of managerial types (who happily put PDAP on their linkedin pages..) trying to manage the 2,500 people who have joined their slack, however only a small number of contributors are actually working on scrapers (~2), who like I said -- are focusing on "criminal" datasets and not datasets around accountability. In doing so, they're effectively ignoring the core of the data that exists to aid in police accountability, eg complaints and misconduct records themselves. This shouldn't be a surprise, since police agencies tend NOT to provide information online that could be used against them in court, and they WILL fight to the teeth to prevent the release of damning records. Souce: I spend a lot of time with FOIA trying to get misconduct records.. The group's foundation is also shaky. The group was started by owners of a marketing company, frac.tl that specializes in exploiting emotions to turn something viral, which on its own should raise flags. The blog post that "started" their supposed movement has three listed writers over the course of several months [1][2][3]. The data from this blog post (afaict the only dataset actually released that's tied to this project) was found to contain private information and had to be cleaned up - 6mo after it was pushed onto github and forked many times. If you are looking for a group to support, please consider either volunteering with or donating to local police accountability groups. Your time/donations would be much better served with them. [1] https://web.archive.org/web/20191118214540/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20191118214540/https://lawsuit.o... [2] https://web.archive.org/web/20200518181855/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20200518181855/https://lawsuit.o... [3] https://web.archive.org/web/20200527213804/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20200527213804/https://lawsuit.o...
bigth 6y ago
So where's the data? All I see is a FAQ page and a donation button. Edit: Found the answer, they dont have any publicily available. Lol. > Our data isn't hosted anywhere yet.
CivBase 6y ago
It really would be nice if we could get some legally-recognized standards for data schemas and interfaces so projects like this wouldn't be so necessary. But I have absolutely no idea if there's any precedent for that or how we'd get there.
dang 6y ago
https://news.ycombinator.com/item?id=23384556 https://news.ycombinator.com/item?id=23384556

78 points by polm23 6y ago

“Are you affiliated with a political party or have a political agenda? No. Our only motivation is to provide trusted data in an age of disinformation.” If true, this is an awesome project.

chaps 6y ago
They're already working with one political candidate: https://pdap.atlassian.net/browse/PDAP-149 https://pdap.atlassian.net/browse/PDAP-149
- zepto 6y ago
  You have linked to a message saying: “Any chance you guys have data on NJ? I'm running for office there and was looking to get some police misconduct data to double check a couple things” It is not clear how this means that they are working with a political candidate, or what that means to you. Can you explain what conclusions you are drawing from this message?
  - chaps 6y ago
    From their slack, a political candidate posted that they were interested One of two of their volunteer scrapers said that he would drop his work in California to work on New Jersey instead. I guess "working with" is bad language, though "interested in dropping other work in favor of requests by political candidates" is more accurate.
    - zepto 6y ago
      Providing data to people who want it seems like the reason the project exists, so it seems natural for them to respond to users asking for data. It’s not clear to me how you think this relates to their claim: “Are you affiliated with a political party or have a political agenda?“
      - chaps 6y ago
        I can see why others would disagree with me here, but I personally see no difference between having party affiliation and having 50% of your relevant volunteers doing work that helps a politician achieve their goals.
        deleted 6y ago
        [deleted]
        zepto 6y ago
        It seems reasonable to assume that 100% of their work will help politicians achieve their goals. Data on NJ was clearly already part of their plan. I don’t see how this has anything to do with party affiliation. Can you explain how it does?
        luxuryballs 6y ago
        I can take a stab at it, there are potentially political motives to why a politician might want data that would bolster the perception of police misconduct.
        zepto 6y ago
        Of course, but that’s an inevitable use of this data, and is about the politician not the project.
        chaps 6y ago
        As far as I can tell, you're being intentionally uncharitable. There's a difference between doing work in isolation that a politician uses for their own political goals and doing work that specifically and intentionally helps a politician on their request. If you were to drop all of your ongoing side projects to work with a political candidate that asked for your help, would you really try to argue that you're not at least somewhat tied to that candidate?
        6y ago
- jjulius 6y ago
  "Working with" feels like a loaded phrase. Someone created that ticket requesting that data 14 minutes ago, and the ticket has yet to even be assigned to anyone (at the time of my post). I'm not sure how you can conflate this project's lack of a response to a request as "already working with".
- Hasnep 6y ago
  A political candidate asked for help accessing some data, not the same thing as affiliating with a political party.

ClosedPistachio 6y ago

Some States have comprehensive databases. Here's one for New Jersey: https://force.nj.com/ https://force.nj.com/

psychlops 6y ago
"most comprehensive" is different from comprehensive. Curious, I went to take a look. The latest set of data was release November 2018 and covered the years 2012-2016 purchasable for $200.
electricBllue 6y ago
Do you happen to have the links for any other states?
rhcom2 6y ago
Just to clarify, The Force Report was the result of over a year of work by journalists that had to fight tooth and nail for every FOIA request and they freely acknowledge it is imperfect. To me at least it is something our government should be already doing.

glenda 6y ago

An issue I see is that the public data police departments put out is often only what they choose to, meaning any analysis of that data is potentially biased to an unknown degree. I suppose adding FOIA data into the mix could improve that, but doesn’t sound easy since those are usually delivered as PDFs.

zepto 6y ago
They could and should document and account for this somewhere in the data set. It wouldn’t solve the problem but it would allow the bias to be visible.
deleted 6y ago
[deleted]
mistermann 6y ago
I think this "the dog that didn't bark" phenomenon may be a much more important phenomenon than meets the eye. People's perception of the world is a function of their observations of reality - for starters, the vast majority of people don't even have any exposure to things like police data (other than the typically misleading narratives that they might encounter in the news), but then even the small majority of people who do dig deeper into the details, these people too will leave with a distorted perception of reality if only a subset of the data is revealed, and they do not know that it is only a subset. The end result of all of this complexity is that people think the world is one way, but it is actually another, and the vastness of the gap between the two could be anywhere from small and insignificant, to gigantic and crucially significant, if one applies this idea to not just police data, but everything (the entirety of reality is subject to this flaw, and many others that we do not realize).
- TeMPOraL 6y ago
  Add the fact that even the businessmen and regulators, influential people, politicians and judges, they all build their image of the world mostly on the same sources as everyone else - and it's becoming worryingly apparent that all the control systems in societies are operating with very low visibility into the things they're controlling. Imagine a PID controller keeping a boiler from overheating. Now imagine the temperature input to that controller is a thermometer mounted to the door of a metal cabinet next to the boiler, and thus it reports anything related to the boiler temperature only when somebody is looking for something in that cabinet (when the opened metal doors touch the boiler and conduct heat). Doesn't sound like a good control system.

cyberlab 6y ago

Is this an American project that sources & aggregates data from American police forces, or is it some general thing that wants to draw from multiple countries? (Because the site doesn't mention anything about locale).

_joel 6y ago
I thought the same, but there are apparently 17,985 police forces in USA[1] so it looks to be American centric. I think they should list this in their FAQ or make it more obvious. [1] https://en.wikipedia.org/wiki/Law_enforcement_in_the_United_States https://en.wikipedia.org/wiki/Law_enforcement_in_the_United_...

jimbob45 6y ago

The police force is losing much of its ability to enforce the law. Young casual political observers praise this but we’ve already seen this happen in the 1960s and then culminate in Bernhard Goetz once everyone was fed up with drugged low-lifes bringing down the quality of life of otherwise hard-working individuals. There must surely be a middle road here.

alexilliamson 6y ago
There are lots of hard working drug addicts as well as lazy sober people. Quality of life is complicated.
throwawaysea 6y ago
The reduction of quality of life is already happening in permissive cities like SF, Portland, and Seattle. Public spaces are taken over by those breaking the law, and yes that very often includes “drugged low-lifes”, and it brings a host of other problems with it. As examples: a popular jogging path near me is now being avoided by women I know because men from the encampments that have taken over the park the path runs through, and sexually harass them sometimes. There are frequent posts on Next Door where some homeless person will be on camera stealing things out of peoples porches or yards, to feed their lifestyle, and police simply don’t have the resources to solve those crimes and deter further criminal acts. Graffiti and littering have increased substantially, despite littering in green spaces carrying a big fine for any law abiding citizens. Criminals that are arrested are mostly simply released in the spirit of restorative justice or similar policies, going on to repeat the same crimes against new victims. This last year has involved a lot of blind hatred of policing and poorly thought out calls to reduce police funding all based on very rare anecdotal incidents which have been spun as data. The reality is that police forces in America have hundreds of millions of interactions with citizens and almost none of them are problematic. It do surprise that surveys show public perceptions of police brutality are incredibly skewed, far from resort (https://nypost.com/2021/02/27/cases-of-police-brutality-against-black-people-are-overestimated/ https://nypost.com/2021/02/27/cases-of-police-brutality-agai...). I can only describe the current trend as a mass hysteria driven by biased news media and dedicated true believers, who organize on social media and real life, and end up influencing the masses into believing the same things.

tyingq 6y ago

On the flip side, this will likely expose a lot more arrest data, and complicate the "innocent until proven guilty" thing that's mostly not true.

chaps 6y ago
This group seems to have turned into a group that's only interested in scraping "criminal records" and "court records" over police accountability records, and believes that cop names should be removed from scraped records. They've also openly discussed inviting cops and FBI employees into their slack to discuss scraping. Consider that "Police Data Accessibility Project" is another way of saying "We only make available data that the police want us to make available.". The project itself has tons of managerial types (who happily put PDAP on their linkedin pages..) trying to manage the 2,500 people who have joined their slack, however only a small number of contributors are actually working on scrapers (~2), who like I said -- are focusing on "criminal" datasets and not datasets around accountability. In doing so, they're effectively ignoring the core of the data that exists to aid in police accountability, eg complaints and misconduct records themselves. This shouldn't be a surprise, since police agencies tend NOT to provide information online that could be used against them in court, and they WILL fight to the teeth to prevent the release of damning records. Souce: I spend a lot of time with FOIA trying to get misconduct records.. The group's foundation is also shaky. The group was started by owners of a marketing company, frac.tl that specializes in exploiting emotions to turn something viral, which on its own should raise flags. The blog post that "started" their supposed movement has three listed writers over the course of several months [1][2][3]. The data from this blog post (afaict the only dataset actually released that's tied to this project) was found to contain private information and had to be cleaned up - 6mo after it was pushed onto github and forked many times. If you are looking for a group to support, please consider either volunteering with or donating to local police accountability groups. Your time/donations would be much better served with them. [1] https://web.archive.org/web/20191118214540/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20191118214540/https://lawsuit.o... [2] https://web.archive.org/web/20200518181855/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20200518181855/https://lawsuit.o... [3] https://web.archive.org/web/20200527213804/https://lawsuit.org/keeping-cops-accountable/ https://web.archive.org/web/20200527213804/https://lawsuit.o...

bigth 6y ago

So where's the data? All I see is a FAQ page and a donation button. Edit: Found the answer, they dont have any publicily available. Lol. > Our data isn't hosted anywhere yet.

CivBase 6y ago

It really would be nice if we could get some legally-recognized standards for data schemas and interfaces so projects like this wouldn't be so necessary. But I have absolutely no idea if there's any precedent for that or how we'd get there.

dang 6y ago

https://news.ycombinator.com/item?id=23384556 https://news.ycombinator.com/item?id=23384556