3 ms·
No, the difference is that theres nothing forcing you to abide by a robots.txt As long as that content is valuable it will be scraped and used. Theres no solu
by bnchrch 2y ago
No, the difference is that theres nothing forcing you to abide by a robots.txt
As long as that content is valuable it will be scraped and used.
Theres no solution to this. So the structure of content sites will have to adapt.
How? I dont know, likely will vary by publication and content type.
Is it fair? Probably not
Is it stoppable? No. Not without heavy, global legislation. (read impossible)
- jaredwiener 2y agoThis is such a weird take. If a house is robbed, you don't give the burglars a pass and say "well their security system wasn't very good." Yes, the defensive measures probably need improvement. But that doesn't mean that the act they are defending against is wrong.
- bnchrch 2y agoMy point is that it isnt a security system. Its a sign on the door that says "Please don't rob me" while the door is unlocked and no body is home.
- jaredwiener 2y agoRight. And my point is that it would still be wrong to rob that person.
- bnchrch 2y agoWe don't disagree there. In such a large ungoverned system as the internet morals don't dictate what people do / dont do. So it doesnt matter whats wrong / right. What does matter is what system participants will / wont do And they will scrape your site if you don't put up any real defences.