10 ms·
Exciting work. I hope they consider releasing the 138 million scans. The FAQ at the end is a bit confusing, because it says: Was the “raw” data saved in add
by jakeogh 2y ago
Exciting work. I hope they consider releasing the 138 million scans.
The FAQ at the end is a bit confusing, because it says:
Was the “raw” data saved in addition to the
preprocessed/cleaned/labeled data (e.g., to support
unanticipated future uses)? If so, please provide a link or
other access point to the “raw” data.
All data is in the dataset.
- defrost 2y agoThe raw scans are in the Linrary of Congress digital collection. (^- oops, example transcription error) The "end of described process" dataset descrption is at https://huggingface.co/datasets/dell-research-harvard/newswire https://huggingface.co/datasets/dell-research-harvard/newswi... and for each record there's "newspaper_metadata", "year", "date", and "article" fields that link back to _a_ LoC newspaper scan. I stress _a_ singular as much is made of their process to identify articles with multiple reprints and multiple scans across multiple newspapers as these repititions of content (to a degree mitigated by local sub editors) with varying layouts are used to robust the conversion of the scans. I haven't investigated whether every duplicate article has a seperate record to a distinct scan source ..