3 ms·
That was really a very useful reply messel. However in our case the crawler code etc is ready. The only thing we need is some powerful infrastructure to run it
by digamber_kamat 16y ago
That was really a very useful reply messel. However in our case the crawler code etc is ready. The only thing we need is some powerful infrastructure to run it on.
We have certainly considered all the factors you have mentioned while implementing our crawler however unlike typical crawlers the objectives of our crawler is purely a research goal and completely different than what most have done.