2 ms·
After 3 years of pirating and scraping the entire world by doing the above, I guess they have everything that they now need or want. So then it's better to sta
by deaux 9mo ago
After 3 years of pirating and scraping the entire world by doing the above, I guess they have everything that they now need or want.
So then it's better to start obeying ROBOTS.txt as a ladder pull through a "nicely behaved" image advantage.
- skybrian 9mo agoObeying robots.txt (now) is still better than not obeying it, regardless of what they did before. The alternative is to say that bugs shouldn’t be fixed because it’s a ladder pull or something. But that’s crazy. What’s the point of complaining if not to get people to fix things?