4 ms·
Gatekeepers of Law: Inside the Westlaw and LexisNexis Duopoly
- shicholas 10mo agoFree Law project is amazing and the best org to counter this duopoly. Please check them out if you haven’t!
- witte 10mo ago[dead]
- wcarss 10mo agohttps://www.courtlistener.com/ https://www.courtlistener.com/ is run by freelaw.
- divbzero 10mo agoPlainSite is another project worth checking out. I have come across both when exploring in the space. [1]: https://free.law/ https://free.law/ [2]: https://www.plainsite.org/ https://www.plainsite.org/
- showerst 10mo agoLexis especially makes even scraping public state laws and admin codes difficult, which is extra frustrating because they are the legal publisher of record in a number of states. I've been considering trying to launch an OpenStates style scraper project for US laws and admin codes, but haven't had the time to attack 100 more scrapers. Even with AI help, the volume is significant.
- shicholas 10mo agoAlso the states themselves can’t update their own laws. In Nevada the code (NRS) from Nevada’s website is out of date. Very embarrassing imo and hard to get it to work bc AI can’t have a trusted source of data.
- rockskon 10mo agoWhy can't Nevada update its own laws on its own website?
- singleshot_ 10mo agoThe answer is probaly embedded within the concept of codification of acts. Legislatures pass acts, which are kind of like diffs for statutory law. But there is no base document, just a series of diffs from the beginning. Somewhere along the way, someone did a lot of work to “codify” the law, and when you go look up 18 USC 1001, and then click “next,” you are taking advantage of the codification process. But the person who did the codification has some rights thereto, meaning that while NV can post every act that passed the legislature, they can’t publish someone else’s codification of the statutes. This matters very little because everyone just has Westlaw and no one uses the state legislature’s website to cite statutes.
- mothballed 10mo agoI would argue it does matter because the public has to know all the law as ignorance of the law is no excuse. If Westlaw is limited or an unreasonable monetary burden on the populace (or possibly, depending on your argument, costs anything at all) then the argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal. IMO, this should also extend to opinions -- if there is precedent that guides what the law is, it needs to be publicly published free of charge so that the public is put under notice what the law is. (someone might mention something like PACER is free in small quantities, I would counter it would cost you a gazillion dollars to be fully informed of all the precedent that forms the full common law meaning of the laws.) This is especially important in mala prohibita crimes since there's no way to even guess through moral/ethical deduction.
- singleshot_ 10mo ago> argument kind of evaporates of how you can prosecute someone, because an essential part of law is that the party under criminal penalty must be put under clear notice of what is illegal. I reckon that’s why the sixth amendment exists but if you want to make a free PACER, go for it.
- google234123 10mo agoGoogle scholar tried and gave up
- toomuchtodo 10mo agohttps://openlaws.us/ https://openlaws.us/ https://public.resource.org/ https://public.resource.org/ https://www.eff.org/press/releases/publicresourceorg-can-keep-freeing-law-court-allows-posting-public-laws-and https://www.eff.org/press/releases/publicresourceorg-can-kee... https://blog.archive.org/2022/10/19/2022-internet-archive-hero-award-carl-malamud/ https://blog.archive.org/2022/10/19/2022-internet-archive-he...
- showerst 10mo agoNeither openlaws nor public.resource actually let you just pull the laws in a common format (or the scrapers) as far as I can tell? I was thinking something more along the lines of a git repo per state.
- toomuchtodo 10mo agoWho will maintain the git repo per state [1] [2]? There is value in a pipeline that continually ingests this data from various sources and pushes it into the Internet Archive, but if you wish to treat it as authoritative, it must have a human minding it, because of entropy and decay. Even the Python Software Foundation has a budget of ~$5M/year. Hence my openlaws.us example. If it was as easy as writing a scraper and dumping it all in a bucket or repo, it'd already be done. It's just the usual thankless hard work over time grind. [1] https://xkcd.com/2347/ https://xkcd.com/2347/ [2] https://en.wikipedia.org/wiki/Free-rider_problem https://en.wikipedia.org/wiki/Free-rider_problem
- showerst 10mo agoI know a thing or two about that, 2,400 commits to the scrapers powering openstates over the past 9 years. Even with openstates, we have an API but don't "just" dump the bills to git for legacy nerd reasons. The nice thing about laws is that the host websites (or PDFs) don't change templates _that_ often, so generally you can rescrape quarterly (or in some states, annually) without a ton of maintenance. With administrative codes you need to scrape more often, but the websites are still pretty stable. The downside is that codes in particular are often big, so a single scrape might need to make 20,000 or more requests, so you have to be very careful about rate limiting and proxies, which goes to my original point that it sucks that accessing this stuff is such a mess.
- deleted 10mo ago[deleted]
- deleted 10mo ago[deleted]
- chaps 10mo agoI do a lot of IL criminal courts research and it's really, really fucking bad out there. Transcripts: Multiple dollars per pages. Want that expedited? Multiply that amount by four. Don't know the court date? Can't get your transcript. Clerks put in the wrong date? Tough luck. Payment for those transcripts? Over Zelle because the court reporters themselves are contractors and get paid independently. FOIA: IL Judicial system isn't FOIAable. There's a "data request" process that you can go down, but last time I tried by requesting through the chief judge of Cook County, they told me to request the data from the Clerk of Courts. The Clerk of Courts told me I need to speak to the chief judge's office. That went on for 6 months. The docket-level data I've seen from them is... beyond indecipherable. Literally never seen a dataset so indecipherable in my life -- a combination of esoteric codes, hidden encoding schemes, and length-delimited lines. Court docs: Gotta go downtown if you're not a lawyer or if you're not a "criminal justice partner" to get any, then you have to print them (don't print past 4 or you'll get yelled at). Some journalists have access, but the IL Supreme Court's policies disallow journalists from getting it. So we have a system where only some journalists get access to court documents and fuck everyone else, I guess. Civil Asset Forfeiture cases: the courts and state's attorneys do everything in their part to separate it from the initiating criminal case. So peoples' cars are being taken by the cops under dubious constitutional grounds and they can't get it back until the criminal case is over. And I've yet to find a consistent way to figure out the initiating criminal case tied to a civil asset forfeiture. I've yet to find anything like a policy doc that says it's intentional to obfuscate the criminal case from the civil case, but I can't think of any reason why it would be like this if it wasn't intentional. FOIA through the State's Attorney? Nah, they don't respond, or they give nonsense denials that require litigation. I could go on, but. Fuck. It's bad out there and it hides so much suffering. I highly encourage folk to give court watching a try. You'll be shocked how much you'll learn.
- mothballed 10mo agoFamily court is even worse. You can't even know what's illegal for your kid to do because the civil standards on things like 'neglect' are incredibly nebulous and left for agencies like CPS and family law judges to review, so to find out how they're actually interpreted you would need to see cases and how they played out. Those are ~all wrapped up in secret court hearings. So basically you have no fucking idea what the law actually is, how it's being enforced, what will happen to you if you don't follow their interpretation, and god forbid you wind up there they can use a lot of methods to keep you from talking about it.
- deleted 10mo ago[deleted]
- aadraple 10mo agoHere's a great X thread with additional reporting giving an inside look on this: https://x.com/ROSSIntel/status/2006429363034824709?s=20 https://x.com/ROSSIntel/status/2006429363034824709?s=20
- deleted 10mo ago[deleted]
- atlanta90210 10mo agoThe state of Georgia tried to copyright its public written laws. They lost. https://www.bfvlaw.com/supreme-court-rules-georgia-cannot-claim-copyright-its-annotated-code/ https://www.bfvlaw.com/supreme-court-rules-georgia-cannot-cl...
- akudha 10mo agoHow does one decide what should be in the public domain for the good of society and what should be commercialized? These are a couple of examples that I learned recently 1. In the UK, Royal Mail owns the postal addresses data. I was looking at UK's open datasets - apparently lot of datasets that have addresses can't be used without paying Royal Mail. There are some exceptions - but I am no lawyer. It is depressing to learn that Royal Mail is no longer a public institution, it was sold against public will by the UK government to a private entity, and sold again and as of last year it is owned by a Czech billionaire. Similarly, Canadian postal code database is also not free. 2. CPT code descriptions are owned by AMA (apparently they're super litigious?). Sure they took the time to write them, they should be compensated - but imagine how many interesting projects can be built if this data was freely available On one hand, multi Billion dollar companies like Bloomberg exist, thanks to free and open data. But also things that should be free (dictionaries, postal codes etc) aren't.
- deleted 10mo ago[deleted]
- AlotOfReading 10mo agoIt's not a question of "how do you decide which public standards should be freely accessible". That's easy: all of them. The public benefits more from freely accessible standards whether they're building codes, legal codes, ISO standards, or HDMI. The effect of not having them publicly available is that people make-do without having read the standard and the public has no way to validate things against the standard afterwards. The question we don't have an easy answer for is how to incentivise the people behind these things without locking their work behind paywalls? Compliance marks, homologation regimes, copyright, and other strategies all have their own downsides.
- esbranson 10mo agoIf you're trying to fix anything relating to public policy, politics, &c., without fixing this you are wasting your time. You will make a bigger difference adopting a local stretch of highway and picking up litter. It is maddening to see generation after generation learn this the hard way. FLP's CourtListener RECAP hits hard, but it has no counterparts either worldwide or within the US. If it goes offline, democracy and rule of law across the entire world go offline with it. Federal statutes and regulations are a very small part of black letter law.
- rockskon 10mo agoThere is little worse action you can take than discouraging others from doing anything. Far worse problems than PACER have been solved. Just because you've been unsuccessful doesn't necessarily mean others will be.
- esbranson 10mo agoPACER is currently a "lesser concern" because of FLP, compared to state court dockets in backwards states like California and New York (affecting some 60 million people alone), though by no means solved. And if by discouraging you mean encouraging people to adopt freeways and pick up litter, I definitely intended to set expectations about how much might be achieved without addressing fundamental free law movement issues. We do need help on Wikipedia though, but that is more ancillary and general.
- sjy 10mo agoNo counterparts worldwide? Here in Australia, all significant case law has been published on AustLII, which is freely accessible and searchable, since about 2000. I think CanLII and NZLII are equally comprehensive. Government law publishers provide free and authoritative access to all statutes. Lexis and Westlaw still make money by publishing commentary and summaries, but paywalled primary sources are an American problem.
- troad 10mo agoAustLII, CanLII and NZLII are all clones of the original Legal Information Institute (LII) from Cornell University. [0] Most Australian courts require or prefer that case law be cited using exact page numbers from the 'authorised reports', i.e. the ones published by Westlaw or LexisNexis. [0] https://www.law.cornell.edu https://www.law.cornell.edu
- TZubiri 10mo ago"The fees the federal government charges for court documents are significant. PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document." It's funny how these numbers sound very reasonable on their face, especially if used for human review. But if you consider programmatic searches, 10 cents per page is a lot. Consider the CPU cost of cycling through each pdf, assume 1000 letters, that's 1Khz which is like a billionth of a cent, in terms of electricity and hardware ammortization. I'd bet that this incentivizes a 'hoarding' and reuse of information, rather than a "query when necessary" practice, which would in fact give place to monopolies like OP argues. A newcomer legal analysis company would have to pay, say 200$ for research from their first customer. If they save those documents, then it is unlikely that their second customer will benefit from those documents. However huge firms would make a business out of the byproduct of searches, they spend 200$ on documents, and they serve a client, but they add those documents to their own internal database, which of course has costs much lower than the nominal fees stipulated by the federal steward (PACER). An argument could be that you can't eliminate the fees because there's costs that need to be paid. But isn't that what taxes are typically for? Especially corporate income tax. The costs are probably a nominal fee to avoid overbearing systems, similar to franchise taxes and incorporation clerical fees. I don't think the idea of charging for documents is bad, who said that the system would need to be adopted for huge data hungry systems, I'm all for giving human analysis an advantage instead of adapting systems to play into the data munching AI fad. I'd rather see balancing that rewards human analysis rather than bringing data munching to the masses.
- bicepjai 10mo agoYour point makes sense, but for law-abiding citizens, it’s hard to justify: people are expected to understand and comply with the law, yet the practical record of how it’s enforced (dockets/filings) is paywalled. If the goal is to fund the system or deter abuse, that’s what general taxation and targeted anti-scraping limits are for, metering “public” records mostly punishes ordinary users and small competitors, not heavy institutional users.
- 15155 10mo ago> PACER charges users $0.10 for every page users search and access, and $30 per name or item searched plus $0.10 per page per document per month, albeit with a max of $3 per individual document This is intentionally misleading and obviously wrong for anyone who has ever used PACER. If you have the PACER Service Center execute the search for you - a human being - it costs $30. If you use the web interface, the rates are extremely low, and citizens are given $30/quarter in credit to use. https://www.uscourts.gov/court-programs/fees/electronic-public-access-fee-schedule https://www.uscourts.gov/court-programs/fees/electronic-publ...
- burner420042 10mo agoIf someone wanted to do business research for Mexico or any other LATAM country, while in the US and using English source data what resources would you look to?
- yxhuvud 10mo agoDepends on what kind of business research you need. But there are at least collections of all tax treaties that exist available from a single source somewhere. I don't remember what the source is off the top of my head unfortunately.
- burner420042 10mo agoThank you. If it ever comes to you please let me know.
- pyuser583 10mo agoAnd this doesn't include the messiness in the decisions themselves. Judges often edit the decisions for some time after they are published. For example, Souter was editing his published opinion in Lee v. Weisman nine years after the case was decided. https://balkin.blogspot.com/2025/05/a-justice-souter-anecdote-or-seeing.html https://balkin.blogspot.com/2025/05/a-justice-souter-anecdot...
- squigz 10mo ago> Having received my letter, Justice Souter notified the Reporter of the error, and at 535 U.S. i appears an erratum, directing that “Homer nodded” be inserted—eight or nine years after the opinion was delivered. I would guess corrections so long after publishing are relatively rare - but if they're also marked as such, it seems a slight stretch to say it's still being "edited" 9 years later
- bicepjai 10mo agoIts crazy insane that citizens who are expected to follow the law cannot have access to law ? what does that even mean ? Do we at-least know some high level numbers on how many law/opinions/cases texts are not freely available to citizens ?
- FireBeyond 10mo agoTangentially the State of Florida will charge you $75/day for your time in jail, regardless of whether you were found not guilty, the case was dismissed or charges dropped completely. And not paying those fees is a second degree felony.
- pstuart 10mo agoOn a tangential note, this seems ripe for LLM analysis/summarization to digest the legal code down so mere mortals could better understand it. It would also be great for pointing out all the loopholes that have been built in by our owners.
- yxhuvud 10mo agoThere is, but that depends a whole lot on access to content that can be hard to get - both case data and books in law are essential.
- handfuloflight 10mo agoRiches are in the niches.
- esbranson 10mo agoOne would need the data.
- squigz 10mo agoI'd be interested to see some information on LLMs being used for legal research. I expect it would not go well. I don't imagine it's just a simple matter of summarizing the text that's there - one has to interpret it with legalese, put it in context of the jurisdiction, look up related cases, etc.
- Terr_ 10mo agoI wonder if anything would happen from a law stating that government (judges, clerks, legislators, etc.) may not pay less for access than the public. Could it align incentives for future reform, or would it just blow to the bottom line of these companies?