3 ms·
Make crawlers not scrapers.
by KasianFranks 7y ago
Make crawlers not scrapers.
- fredley 7y agoMake open APIs, not scrapers.
- bitfhacker 7y agoThe problem is that most companies/developers still aren't "Open API first"... So, people must build scrapers.
- bransonf 7y agoDepends on your use case. If you just need specific data from a known location, build a scraper. If you’re looking to index as much data as possible, build a crawler. The complexity of a crawler is significantly more than a scraper, so it’s not always the right choice.