3 ms·
I've found that Bytedance ignores robots.txt. After discovering that, I used a PHP program to generate "content" in image and text formats for Bytespider to con
by bediger4000 3y ago
I've found that Bytedance ignores robots.txt. After discovering that, I used a PHP program to generate "content" in image and text formats for Bytespider to consume. Turns out Bytespider will consume as much lorem ipsum as you can generate. Now I just have an Apache config that gives Bytespider 404 responses on everything.