4 ms·

"New services shouldn’t expect to be welcomed in the same way, " Why exactly? So megacorporations can keep owning everything of value? +I don't see how downlo

• taskforcegemini 3y ago
  > being downloaded once the issue isn't "downloaded once" but "downloaded at once". agressive crawlers often turn into dos attacks
• Riverheart 3y ago
  I guess how I feel is, if there are 50 new services that spring up overnight with different opt outs that would be annoying for people to deal with. If the website operator isn’t getting value out of the service they might just block you or put content behind an paywall and we all lose. =/ Also, the article claims the scraping generated so much traffic that it got reported to the owner as an attack. I’m not claiming anything other than what’s being reported. They should have at least throttled their application to reduce the load. If the owner didn’t notice the scraping we might not even be having this conversation.
• shadowgovt 3y ago
  > Why exactly? Because when the megacorporations were just corporations the web was young. Now the web is old, and what was interesting and novel at the small scale is tedious and taxing at the large scale. It's much the same as how it's now a death march to write a new standards-compliant browser, or how you can't really found a new country unless you want to start a bloody war or carve a chunk out of Antarctica. Sometimes, the rules have changed by the time new things come along.