5 ms·
After the greader*-grab programs upload data to the target server, it is removed from your machine. All of the data eventually ends up in WARCs at https://arch
by ivank 13y ago
After the greader*-grab programs upload data to the target server, it is removed from your machine. All of the data eventually ends up in WARCs at https://archive.org/details/archiveteam_greader https://archive.org/details/archiveteam_greader
As for an API, someone will hopefully write one to directly seek into a megawarc in that archive.org collection, or import everything into their feed reading service.
- tmzt 13y agoAny way I could patch the program to stop it from deleting the data after it is uploaded? Among other things I would like to set up an ElasticSearch cluster for my own feeds. Is the WARC format defined somewhere? I haven't looked at any other ArchiveTeam projects so I'm not informed if this format is used elsewhere.
- ivank 13y agoYeah, you could patch seesaw-kit to not delete local data. Note that greader-grab just gets a random work item from the tracker. There's an ISO spec for WARC and tools linked at http://www.archiveteam.org/index.php?title=The_WARC_Ecosystem http://www.archiveteam.org/index.php?title=The_WARC_Ecosyste...