4 ms·
I think part of the confusion is that they were referring to archive.org, which is a service of the Internet Archive. Archive.is is a separate site run by someo
by hoppyhoppy2 5y ago
I think part of the confusion is that they were referring to archive.org, which is a service of the Internet Archive. Archive.is is a separate site run by someone else, operating under different guidelines. There is no need to be rude over a misunderstanding.
- geokon 5y agowell I didn't mean to be rude. but it was an off the cuff remark that was wrong. People say things without posting a source and that's how disinformation starts... first thing that came up in search suggests archive.org also is moving away from listening to robot.text files (granted this is an old blog post) https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea... I couldn't find anything stating their policy clearly. but maybe I didn't search hard enough. I find it hard to believe news papers wouldn't block them if it was that easy