3 ms·
Yeah, if it is just one articles or a blog post I crawl at depth 0, and if it is someone's personal website who I enjoy reading always, no matter what they writ
by rasulkireev 4y ago
Yeah, if it is just one articles or a blog post I crawl at depth 0, and if it is someone's personal website who I enjoy reading always, no matter what they write, I do an infinite crawl on that specific domain.