archive.org respects robots.txt so you can exclude what you want I guess, I'm not sure whether or not that's a great thing though. Preferrably people would thi

archive.org respects robots.txt so you can exclude what you want I guess, I'm not sure whether or not that's a great thing though. Preferrably people would think more carefully before disallowing / I guess. I have many times been disappointed that info is entirely gone because archive.org respects robots.txt and the site is now offline forever.