2 ms·
I just started with my website and did a crawl. Subsequently I've been seeding it with the best results form my previous crawls. It's a directed search so it d
by marginalia_nu 5y ago
I just started with my website and did a crawl. Subsequently I've been seeding it with the best results form my previous crawls.
It's a directed search so it doesn't seem to need a particularly solid seed to get decent results.
- c0wb0yc0d3r 5y agoSo how long did it take to get to 17 million URLs?