3 ms·
> Question: do these bots not respect robots.txt? No they don't, because there is no potential legal liability for not respecting that file in most countries.
by throw_m239339 2y ago
> Question: do these bots not respect robots.txt?
No they don't, because there is no potential legal liability for not respecting that file in most countries.