4 ms·
I run a property website that lists properties for sale, rent, etc. A big part of my job is importing feeds, scraping sites (with permission) - and preventing o
by blowski 9y ago
I run a property website that lists properties for sale, rent, etc. A big part of my job is importing feeds, scraping sites (with permission) - and preventing others scraping our site. I know that some people will scrape, but I make I try to make it unprofitable for them to do so.
We do a bunch of other stuff, like adding fake properties so we can check who is scraping our content, and using tarpits.
Developers always argue with me that it's pissing in the wind, that "content wants to be free", that you shouldn't bother even trying to prevent it since it's inevitable. And yet it has helped. We did a split A/B test on preventing scrapers, and it turns out that it's quite effective.
- pharrington 9y agoYou are deliberately devoting energy and resources towards removing value that already existed in your product. If you want to charge rent (via a subscription service etc), then do that, and be clear that you're in the business of charging rent. Don't conflate selling with renting - that just leads to a product gimping death spiral.
- blowski 9y agoValue to whom? If a competitor scrapes all my content and then gets the leads instead of me, they've removed value from my bank account, and that's the main value I'm concerned about. This is not a hypothetical concern - it happens a lot.
- pharrington 9y agoThe competitor did not remove anything from your bank account. If your assertion is that you will lose in economic competition if you make certain information publicly accessible, why are you publicly providing that information while expecting to make money from it in the first place? To me, it sounds like your business lacks some fundamentals.
- jklein11 9y agoIf your secret sauce is the data that you have why not sell it to your competitor?