3 ms·
> Agree with their position 100%. Including the part where they blamed anyone but themselves for hiding old snapshots when robots.txt changed?
by pseudalopex 3y ago
> Agree with their position 100%.
Including the part where they blamed anyone but themselves for hiding old snapshots when robots.txt changed?
- npunt 3y agono I didn't see that in there, I just agree that robots.txt makes sense for web crawlers but not archival purposes