3 ms·
I am using Linkchecker: https://github.com/linkchecker/linkchecker/ https://github.com/linkchecker/linkchecker/ Unfortunately, many web servers will block bots
by vbernat 3y ago
I am using Linkchecker: https://github.com/linkchecker/linkchecker/ https://github.com/linkchecker/linkchecker/
Unfortunately, many web servers will block bots, either immediately or after a few requests.