3 ms·

What type of business can afford being down every time the site they're scrapping is tweaked?

• tantalor 14y ago
  1. Any business that scrapes many sites. 2. Who says it goes down? Might just have stale or incorrect data before the tweak is patched.
• hartleybrody 14y ago
  How often do websites really change the layout of their DOM? Sure, some sites that are actively developed (ie social networks) but for a lot of business websites where information doesn't change that often, like restaurants, reviews or clothing, websites are pretty consistent. Plus, building a parser isn't always that much more work than trying to build a client for a convoluted API. It's obviously case-by-case, but I think a lot of people dismiss scraping outright when there are a lot of useful applications.