3 ms·

Any idea how complete the COLD dataset is compared to the Caselaw Access Project? Unfortunately Caselaw limits access to the full text bulk data of most jurisd

Any idea how complete the COLD dataset is compared to the Caselaw Access Project? Unfortunately Caselaw limits access to the full text bulk data of most jurisdictions without a research account and I’m trying to find an alternative.

chefandy 3y ago
> Unfortunately Caselaw limits access to the full text bulk data of most jurisdictions without a research account That's only true until Feb of 2024! It should be totally unrestricted in a few months.
- civilitty 3y ago
  Great to know, thank you! Any idea why they're time limited? I assumed that was a license restriction from reporters or FastCase, et. al. which would have been permanent.
  - chefandy 3y ago
    To be clear, they are all in the public domain— Fastcase updates included. All of the proprietary info was redacted by hand and the opinions themselves are not copyrightable. The throttling is a contractual obligation to a project partner that limits Harvard's distribution of the cases until Feb of 2024, but that's it. There are also exceptions— cases where the publication is no longer in copyright, and jurisdictions that already publish their opinions online... There are 3 or 4. Those are accessible without throttling through the API and through bulk downloads right now. This should have more up-to-date and accurate information than I do: https://case.law/about https://case.law/about
mdellabitta 3y ago
COLD is bigger than Caselaw Access Project. It's over 8 million cases vs 6.9 on case.law. Please let me know how you find using the data and if you'd like to see any additions or changes!
- civilitty 3y ago
  > Please let me know how you find using the data and if you'd like to see any additions or changes! I only just started yesterday but so far so good! Having it in Parquet files on Git LFS makes a huge difference. It only took a few lines to add the entire dataset to our CI/CD cache which is an improvement over the ingestion scripts we have to normally write with change detection and all that. It took less than an hour to start running the cases through our pipeline - I wish all of the GovInfo bulk data were available this way!
  - mdellabitta 3y ago
    I'm really glad you appreciate that!
- chefandy 3y ago
  That's rad. I'll have to dig into it.

chefandy 3y ago

> Unfortunately Caselaw limits access to the full text bulk data of most jurisdictions without a research account That's only true until Feb of 2024! It should be totally unrestricted in a few months.

civilitty 3y ago
Great to know, thank you! Any idea why they're time limited? I assumed that was a license restriction from reporters or FastCase, et. al. which would have been permanent.
- chefandy 3y ago
  To be clear, they are all in the public domain— Fastcase updates included. All of the proprietary info was redacted by hand and the opinions themselves are not copyrightable. The throttling is a contractual obligation to a project partner that limits Harvard's distribution of the cases until Feb of 2024, but that's it. There are also exceptions— cases where the publication is no longer in copyright, and jurisdictions that already publish their opinions online... There are 3 or 4. Those are accessible without throttling through the API and through bulk downloads right now. This should have more up-to-date and accurate information than I do: https://case.law/about https://case.law/about

mdellabitta 3y ago

COLD is bigger than Caselaw Access Project. It's over 8 million cases vs 6.9 on case.law. Please let me know how you find using the data and if you'd like to see any additions or changes!

civilitty 3y ago
> Please let me know how you find using the data and if you'd like to see any additions or changes! I only just started yesterday but so far so good! Having it in Parquet files on Git LFS makes a huge difference. It only took a few lines to add the entire dataset to our CI/CD cache which is an improvement over the ingestion scripts we have to normally write with change detection and all that. It took less than an hour to start running the cases through our pipeline - I wish all of the GovInfo bulk data were available this way!
- mdellabitta 3y ago
  I'm really glad you appreciate that!
chefandy 3y ago
That's rad. I'll have to dig into it.