3 ms·

The wayback machine is the only reason I use robots.txt: User-agent: ia_archiver Disallow: / There is no good reason to let old (and possibly embarras

• gojomo 15y ago
  Few users prefer the archived versions; only when they need information that has disappeared (or is temporarily unavailable) do they try the Wayback Machine. Many, many web designers are delighted to be able to see their old creations – or even recover past work from the Archive when all other copies are lost to organizational upheaval and system problems. Finally, note that by blocking archiving, you're opting out of history. Those who look back to see the web of today won't see your contributions. Alternatives, competitors, and other projects that are open to archiving will be seen and vividly remembered. Being invisible to the future may be OK for your projects, but many others prefer to part of the shared memory. (FYI: I work on web archiving at the Internet Archive.)