2 ms·
Some time ago I noticed the ByteDance spider very aggressively scraping my modest side project and, more importantly, modest server. I wrote to them to please
by conradfr 2y ago
Some time ago I noticed the ByteDance spider very aggressively scraping my modest side project and, more importantly, modest server.
I wrote to them to please stop (I think the address was in the user agent or something), they replied sorry and actually stopped.
Not sure why all these crawlers can't pace themselves.
- throwaway98797 2y agodevs are promoted on how fast they get done faster, bigger, MOAR sometimes it’s hard to have nice things