3 ms·
- "To complete the download of the full dataset, please fill out this form, which helps us report usage back to our funders, and we'll immediately provide you w
by perihelions 5y ago
- "To complete the download of the full dataset, please fill out this form, which helps us report usage back to our funders, and we'll immediately provide you with a download link."
https://unpaywall.org/products/snapshot https://unpaywall.org/products/snapshot
Is that dataset different from this un-gated one? They're both indexes of Crossref DOI's, and they're both 120 million records long.
https://www.crossref.org/blog/new-public-data-file-120-million-metadata-records/ https://www.crossref.org/blog/new-public-data-file-120-milli...
- IanCal 5y agoUnpaywall is different to crossref. The size will be similar as a lot of scientific DOIs are from crossref, and they're a common dataset to base initial data on. https://unpaywall.org/data-format https://unpaywall.org/data-format is the format for unpaywall https://api.crossref.org/swagger-ui/index.html https://api.crossref.org/swagger-ui/index.html for crossref info
- snarkerson 5y agoWould not be surprised if the Crossref DB was their starting point.
- hpiwowar 5y agoIt is indeed :) (COI: Unpaywall cofounder and dev)
- bnewbold 5y agoUnpaywall is based on Crossref DOIs (one-to-one records), and adds information about publicly accessible versions of each work. In theory publishers can register metadata about whether articles are OA or not with Crossref, but the quality and coverage of this metadata is poor in general. Unpaywall will both check if articles are actually available from the publisher (by following the DOI and parsing the landing page), and by looking for other versions elsewhere on the web (eg, a pre-print). It is simple in theory, but doing this reliably for millions of DOIs from thousands of publishers is a lot of work!