3 ms·
if u run a small site and ppl scrape it aggresively that can rack up ur bills.depending on where u host ur site. ofcourse not an issue for billion dollar compan
by sim7c00 3y ago
if u run a small site and ppl scrape it aggresively that can rack up ur bills.depending on where u host ur site. ofcourse not an issue for billion dollar companies, but a line needs to be drawn somewhere. also, whats the purpose of the scraping? usually its greedy capatalist purposes, so.then ur point is a bit moot, dont u think?
- codedokode 3y agoLet's say someone makes a site that compares prices in different stores. It needs a lot of scraping, but is useful for consumers. Obviously, the only ones who are against it will be the store owners.
- noAnswer 3y agoUnless you go full blown DoS you will not be able to scrap 100.000 of articles multiple times a day. Geizhals for example compares prices every 10 minutes. It does this by working with the stores (they provide a price list) not against them. If a store owner doesn't want the reach, it's their loss. IMHO no need for a DoS attack.
- zztop44 3y agoAnd then someone else makes another site that also compares prices in different stores. Instead of scraping all the stores they can just scrape the first comparison site and invest the time saved into SEO :)