4 ms·
You're quoting "agreement" as if its literally in their robots file. It's not. They're telling the public that it does not have permission to crawl the site wh
by MichaelApproved 13y ago
You're quoting "agreement" as if its literally in their robots file. It's not.
They're telling the public that it does not have permission to crawl the site which try have the right to do. What is the problem with that?
- yeukhon 13y agoWhat is the point of having such silly prohibition? It's silly because anyone can crawl it if they want, Facebook may block such DDoS attack, but why would they bother to put up such sign when they know it's useless?
- sfall 13y agomy guess lawyers
- MichaelApproved 13y agoThey have such a large network, anything they could do to prevent unwanted crawling is probably helpful.