3 ms·
One solution might be to collaborate with, build upon, and donate to an existing reputable organization like archive.org or Wikipedia which takes snapshots of w
by minhmeoke 5y ago
One solution might be to collaborate with, build upon, and donate to an existing reputable organization like archive.org or Wikipedia which takes snapshots of websites.
Given these snapshots, you could write manual extractors or build a machine learning system [1] to extract the main content of each page as plaintext. Then load these timestamped text file snapshots into git, which will give you a hash of the content and let you easily track changes.
Push the git repo to a few places like github, bitbucket, and maybe IPFS where people can mirror it.
[1] https://joyboseroy.medium.com/an-overview-of-web-page-content-extraction-5e0e2c62855d https://joyboseroy.medium.com/an-overview-of-web-page-conten...
Alternatively you could use the built-in reader mode in Firefox or Chrome which do this automatically, but then you'd have to figure out how to maintain stability of the extraction algorithm between new browser releases.