4 ms·
I think that you're building an unethical business, and should not get advice or publicity from this community. Doubly so given you already had a very high vis
by jsnell 3y ago
I think that you're building an unethical business, and should not get advice or publicity from this community.
Doubly so given you already had a very high visibility Show HN for this just a few months ago.
- fastball 3y agoI don't think this community agrees with you that web scraping is inherently unethical. In fact I think many (most?) in this community would argue that web scraping is an almost fundamental feature of the web itself, and that attempts at preventing it are more unethical than scraping.
- mellosouls 3y agoI doubt the objections are about scraping per se, but about unethical scraping where no consideration is given to etiquette. "proxy rotation" in the first line does not bode well for ethics.
- fastball 3y agoWhat's wrong with proxy rotation? Big Tech attempts to prevent any scraping of their content whatsoever. In the context of that web, proxy rotation is table stakes.
- mellosouls 3y agoWithout any documentation about how etiquette will be respected and sites won't be hammered, it's fair to be sceptical about the ethics. Not every crawled site is "big tech" or even commercial.
- fastball 3y agoI don't disagree, but I also don't think "proxy rotation" immediately implies sites will be hammered either.
- mellosouls 3y agoFair enough. I may be too sceptical. But I note the OP has ignored prior comments instead of taking the opportunity to clarify. It doesn't reassure.
- fakedang 3y agoAhh yes, the guy who worked at LinkedIn, I presume?