3 ms·
Whether or not scrapping publically available websites is unethical is probably up for debate. In some cases at least, courts have found it to be legal, even wh
by mikeocool 4mo ago
Whether or not scrapping publically available websites is unethical is probably up for debate. In some cases at least, courts have found it to be legal, even when the site is throwing up technical barriers or issues cease and desists.
What is likely unethical is the fact that they offer residential proxies. The residential providers of those proxies are frequently not aware they’ve been opted in to provide such a service.
- eab- 4mo ago> courts have found it to be legal ≠ ethical
- OhSoHumble 4mo agoI don't know if this falls under good ethical behavior but the one time I needed to mass scrape sites was to build an affordable housing directory for the Bay Area. Whether or not a unit of housing was under rent control was only available through private entities or apartment hosting sites and it was scattered and hard to find. In my opinion, a directory of subsidized housing should have been provided by local governments and not through a plethora of real estate websites.
- inigyou 4mo agoSome are aware. I know some providers that pay you to run a proxy, but the pay isn't very much.