4 ms·
IIRC, wayback machine has a pretty wicked flaw in that if a domain adds a restrictive robots.txt at some point (say, a domain changes hands), then all prior arc
by Karunamon 5y ago
IIRC, wayback machine has a pretty wicked flaw in that if a domain adds a restrictive robots.txt at some point (say, a domain changes hands), then all prior archives for that domain are retroactively taken offline. Not a problem for archiving things just to keep them online under heavy load, but it's a big problem for future use.
I'd prefer something like archive.is for longer-term storage.