4 ms·

Is it possible to see the data the PDAP has scraped? I visited the website but I don't see any actual data.

josh-pdap 4y ago
We don't have any scraped data yet. I replied to the parent post addressing some of this, but mostly if people need the data they run a scraper locally and use the data that way. At the moment, our energy is going into building an app to help people submit and manage our database of data sources: https://docs.pdap.io/activities/data-sources/what-is-a-data-source https://docs.pdap.io/activities/data-sources/what-is-a-data-...
- VWWHFSfQ 4y ago
  Ah very good! This sounds like it would be a perfect application of https://datasette.io/ https://datasette.io/
  - josh-pdap 4y ago
    We’ve looked at this and it’s pretty exciting! Have you used it? Know of any particularly relevant case studies we should look at?
veritas3241 4y ago
https://docs.pdap.io/activities/data-sources/explore-data-sources https://docs.pdap.io/activities/data-sources/explore-data-so... It's in the docs.
- s1artibartfast 4y ago
  It still isn't very clear what the focus and direction they want to go in is from that page. What would be an example of a core data set they are trying to compile? police involved shootings? police budgets? everything.
  - jasonlotito 4y ago
    Seems pretty obvious from the page: > Our mission is to make data from every U.S. police agency accessible via a single public resource. More precisely: > There are over 18,000 police organizations, and each has a unique way to publish information. i.e. Police are publishing data? Let's organize it and make it easily accessible.
    - s1artibartfast 4y ago
      I am curious and want to know more. "Police data" is incredibly vague. What types of data are most interest and available from most of every agency? If the only answer to what kind of data is "police data" then I'm not sure if I should care, support, or contribute. Is this data on how many toilet paper roles departments purchase or police involved shootings? neither? Certainly you can see where the question is coming from right?
      - josh-pdap 4y ago
        Your question makes sense. I’ve addressed this elsewhere but I’m currently just answering on my phone so I’ll summarize by saying that our focus is on complete, solid local data one municipality at a time (full context is really helpful when actually using the data) as opposed to any particular type of data. That said, we still have work to do pointing people in this direction and helping them understand why. This whole thread is going to really affect our website and docs :)
        s1artibartfast 4y ago
        Thanks for responding! What is complete data? Arrests by time and geolocation? Quarterly stats by call or response? I have gone through some of the public info for my department so I am curious.
        jasonlotito 4y ago
        Again, it seems to me that they if police are publishing data, they want to make it accessible. Are police publishing arrests by time and geolocation? Quarterly stats by call or response? Then yes. Hell, I'm sure if they are publishing coffee consumption numbers, that would go up as well.

We don't have any scraped data yet. I replied to the parent post addressing some of this, but mostly if people need the data they run a scraper locally and use the data that way. At the moment, our energy is going into building an app to help people submit and manage our database of data sources: https://docs.pdap.io/activities/data-sources/what-is-a-data-source https://docs.pdap.io/activities/data-sources/what-is-a-data-...

VWWHFSfQ 4y ago
Ah very good! This sounds like it would be a perfect application of https://datasette.io/ https://datasette.io/
- josh-pdap 4y ago
  We’ve looked at this and it’s pretty exciting! Have you used it? Know of any particularly relevant case studies we should look at?

veritas3241 4y ago

https://docs.pdap.io/activities/data-sources/explore-data-sources https://docs.pdap.io/activities/data-sources/explore-data-so... It's in the docs.

s1artibartfast 4y ago
It still isn't very clear what the focus and direction they want to go in is from that page. What would be an example of a core data set they are trying to compile? police involved shootings? police budgets? everything.
- jasonlotito 4y ago
  Seems pretty obvious from the page: > Our mission is to make data from every U.S. police agency accessible via a single public resource. More precisely: > There are over 18,000 police organizations, and each has a unique way to publish information. i.e. Police are publishing data? Let's organize it and make it easily accessible.
  - s1artibartfast 4y ago
    I am curious and want to know more. "Police data" is incredibly vague. What types of data are most interest and available from most of every agency? If the only answer to what kind of data is "police data" then I'm not sure if I should care, support, or contribute. Is this data on how many toilet paper roles departments purchase or police involved shootings? neither? Certainly you can see where the question is coming from right?
    - josh-pdap 4y ago
      Your question makes sense. I’ve addressed this elsewhere but I’m currently just answering on my phone so I’ll summarize by saying that our focus is on complete, solid local data one municipality at a time (full context is really helpful when actually using the data) as opposed to any particular type of data. That said, we still have work to do pointing people in this direction and helping them understand why. This whole thread is going to really affect our website and docs :)
      - s1artibartfast 4y ago
        Thanks for responding! What is complete data? Arrests by time and geolocation? Quarterly stats by call or response? I have gone through some of the public info for my department so I am curious.
        jasonlotito 4y ago
        Again, it seems to me that they if police are publishing data, they want to make it accessible. Are police publishing arrests by time and geolocation? Quarterly stats by call or response? Then yes. Hell, I'm sure if they are publishing coffee consumption numbers, that would go up as well.