4 ms·
Im confused. Do they do the scraping crawling and we just search their database? Do we have to do the scraping/crawling and we dump the results into a mixnode
by nickthemagicman 8y ago
Im confused.
Do they do the scraping crawling and we just search their database?
Do we have to do the scraping/crawling and we dump the results into a mixnode server running locally?
- amingilani 8y agoI'm slightly confused too. They say that the web is a database, but it looks like we're SQL querying their database of the web. I'm also interested in how often they rescrape their pages, and if they have rate-limit bypassing tech (for the Amazon scrapers). So far, I think they're calling the Web a database because you can use SQL to query their database — which makes me feel like they're missing the point. But they've done such hard work and they look like they're really excited about it — but I just don't understand why