5 ms·

PACER's antiquated interface and dirty data have fueled a wildly profitable legal services industry. The public deserves more than the base line of free access

PACER's antiquated interface and dirty data have fueled a wildly profitable legal services industry. The public deserves more than the base line of free access to court records. Give the public access to clean normalized case data like in LexisNexis and Westlaw's outrageously-priced products, including a full database dump for civic hackers to improve. Aside: I worked on a product which relied on scraping PACER. A foolish integration test on our CI server once racked up a $50,000 PACER bill which we didn't notice until the end of the quarter.

rayiner 8y ago
The concept of “civic hacking” is an error of projection. Because user-generated data is valuable in other areas of tech, techies assume its valuable in civics. Unlike with Internet technology, user-generated data has little value in politics and the law. Knowing that 76% of district courts did something one way has little persuasive value compared to knowing two or three well-reasoned court of appeals opinions. The data is in fact out there-all federal cases are available for free on Google Scholar and various other sites. And nobody has done anything interesting with that information. That is not to say that politics and law can’t be data driven. But the data we need in those areas is not data you can derive by analyzing user or government generated databases. We need “hard data” such as how different tax policies impact economic growth. That’s not the sort of data analysis “civic hacking” can give us.
- buildbuildbuild 8y ago
  The users are generating the data already: getting arrested and sued. This data is not yet a URL you can share. Sign up for PACER and see how hard it is to associate a credit card and look up one case. It is not accessible to the casual user. Perhaps we have witnessed different types of citizen contribution. I am not suggesting data science. Many would happily join a volunteer endeavor to clean up our country's case database and build a great UI on top of it which encourages public interest in the judicial system. > The data is in fact out there It is not. There is no complete, real-time, or normalized open case data out there, and LexisNexis and Westlaw's products are not priced to be accessible to individual citizens.
  - rayiner 8y ago
    > Many would happily join a volunteer endeavor to clean up our country's case database and build a great UI on top of it which encourages public interest in the judicial system. What would be the value of that? What problem is that trying to solve that is worth giving up $140 million a year in revenue that’s currently coming almost entirely out of the pockets of lawyers and law firms? > The data is in fact out there It is not. There is no complete, real-time, or normalized open case data out there, and LexisNexis and Westlaw's products are not priced to be accessible to individual citizens. I’m not sure what you mean by “normalized” in this context, but Lexis and Westlaw do a ton of additional analysis on top of public cases at their own expense. That information isn’t in PACER. To generate it you’re talking about even more expense. And while the existing databases are not updated in real time, they’re quite complete for federal cases, which is all that PACER covers. And I’ve never seen an interesting application of that data beyond simply allowing you to read the cases.
    - cheriot 8y ago
      I don't mind advocating against regulatory capture in general. If nothing else, improved access to this data will make the Westlas/LexisNexis industry more competitive. There are a lot of ways governments generate revenue even if we want to restrict it to the legal industry. Obfuscating public information to the point that there's an oligopoly controlling access is fucked up.
      - dd36 8y ago
        Exactly. Sounds like maybe courts should have higher filing fees.
      - rayiner 8y ago
        What exactly is the “regulatory capture” here?
        Nasrudith 8y ago
        The higher than needed fees locking out competitors. Even if they didn't actually 'capture' them it is still regulation supporting their business model without a justifying good. Seat belts being compulsory may make more money for seat belt manufacturers but it isn't regulatory capture because it serves an actual purpose. Requiring all seatbelts be sourced domestically to shut out competitors would be regulatory capture however.
- 420codebro 8y ago
  Why are you so against the data being open? Do you work for/on PACER? Are you a lawyer shill?
- heartbreak 8y ago
  It's not just techies who think this is valuable. Yale Law's Information Society Project uses it.
- Nasrudith 8y ago
  The concept of precedent in itself disagrees with you there - it isn't just a techie thing because a fundamental principle of law is consistency in application. The value may not be financial per say (nobody gets rich directly out of ensuring equal application of the law) but it has a major impact upon quality of life. While it may be used as an excuse to avoid thinking there is a value to precedent in itself in making the laws predictable and better in bounds of certainty. Gathering the data of trials and judgement can prove when something isn't working and prove how things are actually working in the field better than even the best rhetorician. The question of 'should we be doing something' is separate from data. But being able to get the cases together and point out that 'minimum/maximum sentences are constraining judges and juries since in 70% of the cases they note that they would go with lower/higher but they are constrained from it'. It can point out real problems with inequality in execution or corruption. Civic hacking is meant for accountability essentially to show how the system is or isn't working. Which is part of why resistance to it is so worrying.
  - rayiner 8y ago
    That data is already out there. Almost the entire corpus of federal cases is available in data dumps online. I’ve never seen anything interesting done with it. Don’t confuse my opposition to techno-optimism with opposition to accountability. The issue is not that we shouldn’t have accountability. It’s that there is already a ton of data out there, and civic hackers have had approximately zero impact with it. Because for the most part what you actually need is carefully controlled studies from real institutions, not “civic hacking.” Civic hacking is the techie version of “raising awareness.”

rayiner 8y ago

The concept of “civic hacking” is an error of projection. Because user-generated data is valuable in other areas of tech, techies assume its valuable in civics. Unlike with Internet technology, user-generated data has little value in politics and the law. Knowing that 76% of district courts did something one way has little persuasive value compared to knowing two or three well-reasoned court of appeals opinions. The data is in fact out there-all federal cases are available for free on Google Scholar and various other sites. And nobody has done anything interesting with that information. That is not to say that politics and law can’t be data driven. But the data we need in those areas is not data you can derive by analyzing user or government generated databases. We need “hard data” such as how different tax policies impact economic growth. That’s not the sort of data analysis “civic hacking” can give us.

buildbuildbuild 8y ago
The users are generating the data already: getting arrested and sued. This data is not yet a URL you can share. Sign up for PACER and see how hard it is to associate a credit card and look up one case. It is not accessible to the casual user. Perhaps we have witnessed different types of citizen contribution. I am not suggesting data science. Many would happily join a volunteer endeavor to clean up our country's case database and build a great UI on top of it which encourages public interest in the judicial system. > The data is in fact out there It is not. There is no complete, real-time, or normalized open case data out there, and LexisNexis and Westlaw's products are not priced to be accessible to individual citizens.
- rayiner 8y ago
  > Many would happily join a volunteer endeavor to clean up our country's case database and build a great UI on top of it which encourages public interest in the judicial system. What would be the value of that? What problem is that trying to solve that is worth giving up $140 million a year in revenue that’s currently coming almost entirely out of the pockets of lawyers and law firms? > The data is in fact out there It is not. There is no complete, real-time, or normalized open case data out there, and LexisNexis and Westlaw's products are not priced to be accessible to individual citizens. I’m not sure what you mean by “normalized” in this context, but Lexis and Westlaw do a ton of additional analysis on top of public cases at their own expense. That information isn’t in PACER. To generate it you’re talking about even more expense. And while the existing databases are not updated in real time, they’re quite complete for federal cases, which is all that PACER covers. And I’ve never seen an interesting application of that data beyond simply allowing you to read the cases.
  - cheriot 8y ago
    I don't mind advocating against regulatory capture in general. If nothing else, improved access to this data will make the Westlas/LexisNexis industry more competitive. There are a lot of ways governments generate revenue even if we want to restrict it to the legal industry. Obfuscating public information to the point that there's an oligopoly controlling access is fucked up.
    - dd36 8y ago
      Exactly. Sounds like maybe courts should have higher filing fees.
    - rayiner 8y ago
      What exactly is the “regulatory capture” here?
      - Nasrudith 8y ago
        The higher than needed fees locking out competitors. Even if they didn't actually 'capture' them it is still regulation supporting their business model without a justifying good. Seat belts being compulsory may make more money for seat belt manufacturers but it isn't regulatory capture because it serves an actual purpose. Requiring all seatbelts be sourced domestically to shut out competitors would be regulatory capture however.
420codebro 8y ago
Why are you so against the data being open? Do you work for/on PACER? Are you a lawyer shill?
heartbreak 8y ago
It's not just techies who think this is valuable. Yale Law's Information Society Project uses it.
Nasrudith 8y ago
The concept of precedent in itself disagrees with you there - it isn't just a techie thing because a fundamental principle of law is consistency in application. The value may not be financial per say (nobody gets rich directly out of ensuring equal application of the law) but it has a major impact upon quality of life. While it may be used as an excuse to avoid thinking there is a value to precedent in itself in making the laws predictable and better in bounds of certainty. Gathering the data of trials and judgement can prove when something isn't working and prove how things are actually working in the field better than even the best rhetorician. The question of 'should we be doing something' is separate from data. But being able to get the cases together and point out that 'minimum/maximum sentences are constraining judges and juries since in 70% of the cases they note that they would go with lower/higher but they are constrained from it'. It can point out real problems with inequality in execution or corruption. Civic hacking is meant for accountability essentially to show how the system is or isn't working. Which is part of why resistance to it is so worrying.
- rayiner 8y ago
  That data is already out there. Almost the entire corpus of federal cases is available in data dumps online. I’ve never seen anything interesting done with it. Don’t confuse my opposition to techno-optimism with opposition to accountability. The issue is not that we shouldn’t have accountability. It’s that there is already a ton of data out there, and civic hackers have had approximately zero impact with it. Because for the most part what you actually need is carefully controlled studies from real institutions, not “civic hacking.” Civic hacking is the techie version of “raising awareness.”