4 ms·
Hey, I was just curious how a tool like this is built. I'm not saying I want to know every intricate detail, but how would one crawl the entire web over and ov
by eldorad0 7y ago
Hey,
I was just curious how a tool like this is built. I'm not saying I want to know every intricate detail, but how would one crawl the entire web over and over for such a cheap price ?
- lyjackal 7y agoMy guess would be scraping search engines. Still doesn't seem very cost efficient
- JamesGreene 7y agoA lot of it is because I have been involved in prior projects that involve scraping for the past decade. I have built up a lot of code, skill, API access, libraries, and thoughts on crawling and scraping different websites. Also, this won't stay almost $7 forever. But I wanted to test the waters and see demand. I also don't think it will go much higher either so "cheap" will be relative to each person.