3 ms·
Good luck to them blocking any well executed, geographically distributed and well architected scraping/mirroring effort. If you're willing to ignore a robots.tx
by walrus01 7y ago
Good luck to them blocking any well executed, geographically distributed and well architected scraping/mirroring effort. If you're willing to ignore a robots.txt file and do whatever is necessary to mirror content from publicly accessible http daemons on the internet, there's a lot of ways to do it.
It sounds like the initial effort from archive team was a bit too concentrated and strained server resources.
- bdcravens 7y agoThey have 10 days before the deletion takes place. "On Nov 18, we are planning on permanently removing anonymous content that meets the following criteria: 1) less than 200 views, 2) older than one year, and 3) anonymous (made w/o an account)" https://twitter.com/GfycatHelp/status/1191770073259577344?s=20 https://twitter.com/GfycatHelp/status/1191770073259577344?s=...