3 ms·

I think you might be rather successful in gaining initial market traction by scraping off sites which are closing down. This way, you will also be able to suppo

I think you might be rather successful in gaining initial market traction by scraping off sites which are closing down. This way, you will also be able to support people who would potentially be loosing a lot of their data otherwise.

memset 5y ago
Yes! This was the reason our first integration was with Chartio. I wonder if it is easy to find companies that are shutting down - maybe even an HN search would give us some direction.
- betamaxthetape 5y ago
  Join Archive Team [1]! As that site says, "Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage". We have a number of volunteers actively monitoring various sites and searching for others that are shutting down (there's a lot), which are noted on the Deathwatch [2]. Typically the process is as follows: 1. A site announces it is shutting down. 2. Archive Team discovers the site is closing and adds it to the Deathwatch. 3. Members of Archive Team collaborate to write scripts to extract user data from the website, using an existing archiving framework [3] developed for / by Archive Team. 4. Members of the public are invited to run instances of the archiving software [4], resulting in a distributed archiving network. This is often quite powerful, and has to be managed carefully to avoid overloading the site being archived. 5. Once completed, the data is (in many cases) uploaded to the Internet Archive to be stored permanently. Note that Archive Team is not affiliated with the Internet Archive, although we greatly appreciate their willingness to host this sort of data. I'm not particularly actively involved with Archive Team at the moment (too busy with real life!) but I was heavily involved with a few projects, the most significant being the project to archive Yahoo Groups in 2019, which was quite complex due to the site structure (and Yahoo being less than helpful [5]). We got a lot (much more than I ever expected to get) but the majority of all that data was deleted. [1] http://wiki.archiveteam.org/ http://wiki.archiveteam.org/ [2] http://wiki.archiveteam.org/index.php/Deathwatch http://wiki.archiveteam.org/index.php/Deathwatch [3] http://wiki.archiveteam.org/index.php/Dev/Seesaw http://wiki.archiveteam.org/index.php/Dev/Seesaw [4] http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior [5] https://arstechnica.com/tech-policy/2019/12/verizon-reportedly-blocks-archivists-from-yahoo-groups-days-before-deletion/ https://arstechnica.com/tech-policy/2019/12/verizon-reported...

memset 5y ago

Yes! This was the reason our first integration was with Chartio. I wonder if it is easy to find companies that are shutting down - maybe even an HN search would give us some direction.

betamaxthetape 5y ago
Join Archive Team [1]! As that site says, "Archive Team is a loose collective of rogue archivists, programmers, writers and loudmouths dedicated to saving our digital heritage". We have a number of volunteers actively monitoring various sites and searching for others that are shutting down (there's a lot), which are noted on the Deathwatch [2]. Typically the process is as follows: 1. A site announces it is shutting down. 2. Archive Team discovers the site is closing and adds it to the Deathwatch. 3. Members of Archive Team collaborate to write scripts to extract user data from the website, using an existing archiving framework [3] developed for / by Archive Team. 4. Members of the public are invited to run instances of the archiving software [4], resulting in a distributed archiving network. This is often quite powerful, and has to be managed carefully to avoid overloading the site being archived. 5. Once completed, the data is (in many cases) uploaded to the Internet Archive to be stored permanently. Note that Archive Team is not affiliated with the Internet Archive, although we greatly appreciate their willingness to host this sort of data. I'm not particularly actively involved with Archive Team at the moment (too busy with real life!) but I was heavily involved with a few projects, the most significant being the project to archive Yahoo Groups in 2019, which was quite complex due to the site structure (and Yahoo being less than helpful [5]). We got a lot (much more than I ever expected to get) but the majority of all that data was deleted. [1] http://wiki.archiveteam.org/ http://wiki.archiveteam.org/ [2] http://wiki.archiveteam.org/index.php/Deathwatch http://wiki.archiveteam.org/index.php/Deathwatch [3] http://wiki.archiveteam.org/index.php/Dev/Seesaw http://wiki.archiveteam.org/index.php/Dev/Seesaw [4] http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior http://wiki.archiveteam.org/index.php/ArchiveTeam_Warrior [5] https://arstechnica.com/tech-policy/2019/12/verizon-reportedly-blocks-archivists-from-yahoo-groups-days-before-deletion/ https://arstechnica.com/tech-policy/2019/12/verizon-reported...