3 ms·
Once they acquire old domains, some websites start blocking archive.org through robots.txt. It would be better if there's any elegant solution for this problem.
by johnx123-up 14y ago
Once they acquire old domains, some websites start blocking archive.org through robots.txt. It would be better if there's any elegant solution for this problem.