4 ms·
Bytedance seems to have increased its scraping efforts significantly. I've posted a canary token[1] URL as a Mastodon post, to check how scrape-resistant Masto
by miki123211 2y ago
Bytedance seems to have increased its scraping efforts significantly.
I've posted a canary token[1] URL as a Mastodon post, to check how scrape-resistant Mastodon actually is (it is not resistant at all), and have been getting quite a few hits from the ByteDance spider recently.
Last hit is from 47.128.114.151, Mozilla/5.0 (Linux; Android 5.0) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; Bytespider; spider-feedback@bytedance.com)
Edit: added missing footnote.
[1] https://canarytokens.org https://canarytokens.org
- diggan 2y ago> to check how scrape-resistant Mastodon actually is (it is not resistant at all) That's expected, no? It's a social network that is explicitly designed to be as open as possible, as it's using ActivityPub. To be "scraping resisting" would be to go against the very goal of Mastodon.
- Aachen 2y agoExactly, this is how I want it to be. I post there because it's not another walled garden that profits from lock-in
- miki123211 2y agoYes and no. If you look at the technical side of things, you're absolutely right. If you look at the social side, however, there's a lot of talk on there about opting out of scraping, scrapers being bad, not wanting to be part of AI training and so on. Naming-and-shaming people who have been caught scraping is a routine practice. I think that many Mastodonians believe that defederating from scraper-friendly instances and blocking scraper-like requests on their own protects them, this was a way to show that this very much isn't true.