36 ms·

I accidentally started a movement – Policing the Police by scraping court data

650 points by kristintynski 4y ago

Almost 3 years ago, I posted a story of how a post I wrote about utilizing county level police data to "police the police" to r/privacy and hackernews. https://old.reddit.com/r/privacy/comments/gr11aw/i_think_i_accidentally_started_a_movement/ https://old.reddit.com/r/privacy/comments/gr11aw/i_think_i_a... The idea quickly evolved into a real goal, to make good on the promise of free and open policing data. By freeing policing data from antiquated and difficult-to-access county data systems, and compiling that data in a rigorous way, we could create a valuable new tool to level the playing field and help provide community oversight of police behavior and activity. In the almost 3 years since the first post, something amazing has happened. The idea turned into something real. Something called The Police Data Accessibility Project. (https://www.pdap.io https://www.pdap.io) More than 2,000 people joined the initial community, and while those numbers dwindled after the initial excitement, a core group of highly committed and passionate folks remained. In these 3 years, this team has worked incredibly hard to lay the groundwork necessary to enable us to realistically accomplish the monumental data collection task ahead of us. Let me tell you a bit about what the team has accomplished in these 3 years. Established the community and identified volunteer leaders who were willing and able to assume consistent responsibility. -Gained a pro-bono law firm to assist us in navigating the legal waters. Arnold + Porter is our pro-bono law firm. -Arnold + Porter helped us to establish as a legal entity and apply for 501c3 status -501c3 status granted -We've carefully defined our goals and set a clear roadmap for the future -Hired first full-time staff. -PDAP was awarded a $250,000 grant by The Heinz Endowments So now, I'm asking for help, because scraping, cleaning, and validating 18,000 police departments is no easy task. The first is to join us and help the team. Perhaps you joined initially, realized we weren't organized yet, and left? Now is the time to come back. Or, maybe you are just hearing of it now. Either way, the more people we have working on this, the faster we can get this done. Those with scraping experience are especially needed. The second is to either donate, or help us spread the message. The more donations, the more data we can gather. I want to thank the r/privacy community especially. It was here that things really began. TL;DR: I accidentally started a movement from a blog post I wrote about policing the police with data. The movement turned into something real because of r/privacy and hackernews: (Police Data Accessibility Project). 3 years later, the groundwork has been laid, non-profit established, full-time staff hired, and $250,000 in grant money and donations so far! Scrapers so far Github https://github.com/Police-Data-Accessibility-Project/Scrapers https://github.com/Police-Data-Accessibility-Project/Scraper... Discord if you would like to join the efforts: https://discord.com/invite/wMqex8nKZJ https://discord.com/invite/wMqex8nKZJ *This is US centric

meteor333 4y ago
Thanks for sharing about your project! Do you mind giving us brief on what kind of data you are collecting and highlight any interesting findings so far?
treis 4y ago
- llbeandata 4y ago
  On their github [0] I found https://app.pdap.io https://app.pdap.io but it is not loading for me. [0] - https://github.com/Police-Data-Accessibility-Project/PDAP-Data-Sources-App https://github.com/Police-Data-Accessibility-Project/PDAP-Da...
  - josh-pdap 4y ago
    This was an early experiment, we don't have a published app right now.
- kristintynski 4y ago
  You can find the roadmap, goals, and a lot of detail on the organizational structure and the scrapers that have been written so far are here https://docs.pdap.io/activities/our-process https://docs.pdap.io/activities/our-process and here https://github.com/Police-Data-Accessibility-Project https://github.com/Police-Data-Accessibility-Project
  - kylewatson 4y ago
    With all due respect, you really need to be able to explain the goals in a sentence or two, and then follow up with a link to the pdf. An elevator pitch, a vision statement, a clearly written user statement, all would go a long ways towards explaining why this is a worth while use of ones time and energy. I don't know whether the goal is to find bad cops who have been fired and get hired one town over and try to prevent those hirings? Or if the idea is to find people who have been arrested and ruin their lives? Or if the goal is simply to make the data available and let people use it for whatever they wish, from deepfakes to erotic fan fiction. Data wants to be free. The data is the end goal. Public data should be public, etc all sound great. But, I've seen the mug shot database ruin people's lives, and eventually its founders had mugshots of their own. Scraping and publishing is one thing, but knowing the goals is even more important because it would let me know why we're scraping, what we need to scrape, how to make that available, etc. What problem are you trying to solve? And Why? Police data is not easily searchable? Okay, so what? What good is it to make this available? What uses could it have? Even if the goal is just to build it and see how people use it, it would be helpful to know that. I strongly suggest you consider working backwards. It's been 3 years. What does 3 years from now look like? What is the press release? How do you know you were successful, what does that mean? Availability is nice. It could lead to transparency. That may lead to accountability. Is that the goal? Right now this data is hard to access. If you succeed it will be easier to access. And so what? What changes in the world could come because of this? If you fail, what lost opportunity do we mourn? You gotta have some sort of easily conveyed reason for doing this. If the data accessibility alone is the end goal, that's fine, not enough for me, but at least make that clear and convince me that's worth the effort.

650 points by kristintynski 4y ago

meteor333 4y ago

Thanks for sharing about your project! Do you mind giving us brief on what kind of data you are collecting and highlight any interesting findings so far?

treis 4y ago

llbeandata 4y ago
On their github [0] I found https://app.pdap.io https://app.pdap.io but it is not loading for me. [0] - https://github.com/Police-Data-Accessibility-Project/PDAP-Data-Sources-App https://github.com/Police-Data-Accessibility-Project/PDAP-Da...
- josh-pdap 4y ago
  This was an early experiment, we don't have a published app right now.
kristintynski 4y ago
You can find the roadmap, goals, and a lot of detail on the organizational structure and the scrapers that have been written so far are here https://docs.pdap.io/activities/our-process https://docs.pdap.io/activities/our-process and here https://github.com/Police-Data-Accessibility-Project https://github.com/Police-Data-Accessibility-Project
- kylewatson 4y ago
  With all due respect, you really need to be able to explain the goals in a sentence or two, and then follow up with a link to the pdf. An elevator pitch, a vision statement, a clearly written user statement, all would go a long ways towards explaining why this is a worth while use of ones time and energy. I don't know whether the goal is to find bad cops who have been fired and get hired one town over and try to prevent those hirings? Or if the idea is to find people who have been arrested and ruin their lives? Or if the goal is simply to make the data available and let people use it for whatever they wish, from deepfakes to erotic fan fiction. Data wants to be free. The data is the end goal. Public data should be public, etc all sound great. But, I've seen the mug shot database ruin people's lives, and eventually its founders had mugshots of their own. Scraping and publishing is one thing, but knowing the goals is even more important because it would let me know why we're scraping, what we need to scrape, how to make that available, etc. What problem are you trying to solve? And Why? Police data is not easily searchable? Okay, so what? What good is it to make this available? What uses could it have? Even if the goal is just to build it and see how people use it, it would be helpful to know that. I strongly suggest you consider working backwards. It's been 3 years. What does 3 years from now look like? What is the press release? How do you know you were successful, what does that mean? Availability is nice. It could lead to transparency. That may lead to accountability. Is that the goal? Right now this data is hard to access. If you succeed it will be easier to access. And so what? What changes in the world could come because of this? If you fail, what lost opportunity do we mourn? You gotta have some sort of easily conveyed reason for doing this. If the data accessibility alone is the end goal, that's fine, not enough for me, but at least make that clear and convince me that's worth the effort.