4 ms·
While I have substantial agreement with your point of view, when web content is substantially ad-funded, automated scraping effectively bypasses the “payment”.
by sokoloff 6y ago
While I have substantial agreement with your point of view, when web content is substantially ad-funded, automated scraping effectively bypasses the “payment”.
I say this as someone who runs an adblocker, installs them for family, and doesn’t derive income from hosting ads, so I’m not pro-ad; I just realize that while it’s not a crime for me to dump the whole bowl of waiting room candies into my backpack, that’s going to be frowned upon.
- simplecto 6y agoThis is orthogonal to the purpose of scraping. If I run a search engine which potentially links real human eyes back to you, then should I pay the "ad" toll as well? I don't believe so. I do believe there is an agreeable middle-ground, but Google walked away from that conversation years ago.
- krageon 6y agoAdvertisements are harmful to every layer of society, mostly because they prey on you to instill desires that you probably wouldn't have had on your own (it being the case that this is their entire added value proposition). They should not be tolerated, and the fact that they ever were is a travesty. That being the case, any argument founded on "but advertisements" does not hold water.
- LunaSea 6y agoIt's not founded on "but advertisements" it's founded on "but paying for content". The fact that websites are developed by people (wages), hosted on infrastructure (hardware renting) and require countless other jobs is somehow magically forgotten in these discussions.
- perched_robin 6y agoWhatever can be killed by adblockers should be. 95% of the web consists of bloated javascript made by overpaid frontend devs that's less functional than a plain text file. So the ad money runs dry and nobody can afford to push 20 megabytes of javascript and fonts with every page view anymore. What a tragedy.
- krageon 6y agoIt's not up to me or anyone consuming content to help with figuring out the right way to pay for it. What is up to me is the choice to not be potentially exposed to malware any time I go to read the news. While you may have a point, that point just doesn't matter within the context that we're talking about. Making companies money is not my responsibility.
- LunaSea 6y agoIt absolutely is your responsibility if you are consuming the content. Consuming the content means agreeing to the premise that the content is paid for using ads. If you disagree with said premise you may happily browse another website. Malware is already illegal to install and you are free to sue the website for damages. I don't like ads either but trying to justify that their should be a choice of browsing a website without the ads it hosts is ludicrous.
- krageon 6y ago> Consuming the content means agreeing to the premise that the content is paid for using ads. It is in fact not this way, because the content arrives with or without the ads. In the EU EULAs (the dystopian construct that you'd expect to enforce that bit of lunacy) that purport to apply to content you have already accessed are not legally valid. Leaving the legal interpretation aside, me doing one thing doesn't mean consent for something else. Believing otherwise is both unethical and amoral, stances I don't hugely feel like interacting with.
- LunaSea 6y ago> It is in fact not this way, because the content arrives with or without the ads But that's simply a technical implementation detail. If the articles you read had first party ads or videos with embedded ads in them, that choice wouldn't exist like it already doesn't when you watch TV. > Leaving the legal interpretation aside, me doing one thing doesn't mean consent for something else. Believing otherwise is both unethical and amoral, stances I don't hugely feel like interacting with. Yes and you stealing content (consuming without paying for it) is somehow moral? It's easy to find all kinds of free content, should it be music, movies or series but lets not kid ourselves in thinking that it is some great human right to have access to things we didn't pay for.
- shadowprofile77 6y agoWhen you dump the bowl of candies into your backpack, there are no more candies left for anyone else. On the other hand, when you scrape, all the original data and its underlying use case remain just as they are. You simply having scraped took nothing away from anyone.
- mikem170 6y agoI wrote a scraper for myself only to keep an eye on several used truck websites. I was able to do this for craigslist, so I could run it every couple days and check the nearest 75 cities, the list of truck ads, filtering on/out certain keywords, and the pages for the interesting trucks, making a report for myself to help find the vehicle I want. I could not do this for any of other several large sites advertising used trucks, like commercialtrucker.com, truckpaper.com, and machinio.com. I kept bumping into artificial javascript and captcha limitations that were not worth my time to try to work around. The thing is that I will probably find what I want on a craigslist site that I can scrape, I'm getting a lot of great info from them, and I'm not going to bother with any of the ad-based sites, it takes to olong to run all the manual searches. They have effectively done a disservice to their (presumably paying) customers who want to sell their vehicles.
- sokoloff 6y agoIf AutoTempest covers your search criteria, I’ve found them to be excellent to do repetitive searches. Somewhat fortunately for you, they suck for craigslist searches, but have great coverage for all the other major sites.