3 ms·

Then I'll need to make some crawler, which is also not that easy to implement. Maybe it's a good idea to have some kind of complex functionality, broken down t

by Akcium 6y ago

Then I'll need to make some crawler, which is also not that easy to implement. Maybe it's a good idea to have some kind of complex functionality, broken down to parts: - screenshots - links - DNS - ... so that user can enable/disable them as he needs too

rovr138 6y ago
You can also use something that’s built already and extend it. I have used in the past linkchecker, https://github.com/linkchecker/linkchecker https://github.com/linkchecker/linkchecker to do crawling too. You can get the report and analyze it however you want.