6 ms·

I disagree. The person who owns the server should get to decide who has access and under what circumstances. Joe Scraper, having invested nothing, has no claim

by hsod 10y ago

I disagree. The person who owns the server should get to decide who has access and under what circumstances. Joe Scraper, having invested nothing, has no claim or rights to it whatsoever. Furthermore, the fact that the server gives 200 responses is not sufficient implied permission IF a no-scraping policy has been communicated in some other way such as robots.txt or (clearly communicated) TOS. The techno-nihilist argument of "i'm just throwing bits on the wire, they're choosing to respond" is pretty unconvincing in a world where intent and context matters.

elmigranto 10y ago
Is there a law requiring me to read and accept TOS or robots.txt before opening a website? What is the legal status of those? What if TOS says "you owe us $100 for every byte included in this TOS"? What if I have my own TOS about opening their links where they owe me $100 per byte sent?
- deleted 10y ago
  [deleted]
- brilliantcode 10y ago
  No. TOS is not mutually binding agreement. You can't invite people to your garage sales and tell people that by looking at the items they've made a final sale automatically without their consent.
  - pdabbadabba 10y ago
    Not true (in the U.S.). Just ask Aaron Swartz. The fact that it does (or, at least, has been interpreted to) criminalize TOS violations is one of the must important flaws in the Computer Fraud and Abuse Act. https://www.eff.org/deeplinks/2013/01/rebooting-computer-crime-law-part-1-no-prison-time-for-violating-terms-of-service https://www.eff.org/deeplinks/2013/01/rebooting-computer-cri...
    - brilliantcode 10y ago
      Aaron Swartz was a weird one. Some zealous IT guy exaggerated it as if national security data were being scraped when in fact most were just shoddy research papers written at 1 AM before the due date. All people that landed a hand in this man's death should be charged with manslaughter. Truly a shame and how draconian laws end up killing the innocent in the United States.
brilliantcode 10y ago
This feels a lot like the "downloading mp3 is stealing". If you don't want people stealing your music, don't put it other people's hands. The minute you release it to the world, it can't be reversed. See streisand effect. Likewise, you cannot place burden on your visitors to read and analyze ToS with their lawyers and submit an official request via fax. If you don't want people access your server, put it behind a paywall or a login screen at least so that you can easily ban people who don't play by your rules. Otherwise, you have no excuse. If it's publicly accessible, then you cannot enforce any legally binding agreement as you've left the front door wide open and expect people to read tiny letters you put in front of your mailbox. ToS is a one way street.
- hsod 10y ago
  Seems like your argument is based on the idea that it's unreasonable to expect the end user to understand the terms under which a service is offered. And that's fair in some cases, but it's also kind of a cop out. "How was I supposed to know they didn't want me to scrape it? I'm just an innocent passerby dropping bits on a wire" is bullshit in many, many cases. You do know, or at least could easily find out if you wanted to, but choose to maintain a fiction of ignorance to avoid responsibility. Sure, if someone makes it difficult or arcane to read and understand their TOS, you're probably not (morally) bound by it. But if you close your eyes and plug your ears, you don't have much of a leg to stand on.
  - brilliantcode 10y ago
    If someone breaks the TOS, you can block them or send them C&D letters. If you have the cash to take someone to court but you are going to be on the losing side. ToS is not a mutually binding agreement. Even if you have a checkbox that says "I have read the terms", it's often thrown out because nobody expects you to read EULA down to the letter.
  - codedokode 10y ago
    Scraping a website can be compared to only downloading mp3 file, not redistributing it to other people. And I don't see how electronic ToS have any legal power. It is not a contract one has signed before visiting a site. I think ToS should only bind the website owner, not its visitors.
codedokode 10y ago
The person who owns a server can restrict access by adding a login form and creating user accounts. Or taking other measures (for example, banning IP networks or countries or implementing a captcha). I think that the only thing that has to be regulated is the load (number of request per unit of time) on the server. So that it doesn't prevent server from serving pages to other visitors. Regarding robots.txt, I am not sure if ignoring it should have legal consequences. It is just a hint to robots, don't visit these pages or we will ban you. If there is no harm done to the server I don't see any problem with scraping.
- pdabbadabba 10y ago
  > The person who owns a server can restrict access by adding a login form and creating user accounts. Or, say, returning "403 Forbidden" or using a CAPTCHA?
- deleted 10y ago
  [deleted]

by hsod 10y ago

elmigranto 10y ago

Is there a law requiring me to read and accept TOS or robots.txt before opening a website? What is the legal status of those? What if TOS says "you owe us $100 for every byte included in this TOS"? What if I have my own TOS about opening their links where they owe me $100 per byte sent?

deleted 10y ago
[deleted]
brilliantcode 10y ago
No. TOS is not mutually binding agreement. You can't invite people to your garage sales and tell people that by looking at the items they've made a final sale automatically without their consent.
- pdabbadabba 10y ago
  Not true (in the U.S.). Just ask Aaron Swartz. The fact that it does (or, at least, has been interpreted to) criminalize TOS violations is one of the must important flaws in the Computer Fraud and Abuse Act. https://www.eff.org/deeplinks/2013/01/rebooting-computer-crime-law-part-1-no-prison-time-for-violating-terms-of-service https://www.eff.org/deeplinks/2013/01/rebooting-computer-cri...
  - brilliantcode 10y ago
    Aaron Swartz was a weird one. Some zealous IT guy exaggerated it as if national security data were being scraped when in fact most were just shoddy research papers written at 1 AM before the due date. All people that landed a hand in this man's death should be charged with manslaughter. Truly a shame and how draconian laws end up killing the innocent in the United States.

brilliantcode 10y ago

This feels a lot like the "downloading mp3 is stealing". If you don't want people stealing your music, don't put it other people's hands. The minute you release it to the world, it can't be reversed. See streisand effect. Likewise, you cannot place burden on your visitors to read and analyze ToS with their lawyers and submit an official request via fax. If you don't want people access your server, put it behind a paywall or a login screen at least so that you can easily ban people who don't play by your rules. Otherwise, you have no excuse. If it's publicly accessible, then you cannot enforce any legally binding agreement as you've left the front door wide open and expect people to read tiny letters you put in front of your mailbox. ToS is a one way street.

hsod 10y ago
Seems like your argument is based on the idea that it's unreasonable to expect the end user to understand the terms under which a service is offered. And that's fair in some cases, but it's also kind of a cop out. "How was I supposed to know they didn't want me to scrape it? I'm just an innocent passerby dropping bits on a wire" is bullshit in many, many cases. You do know, or at least could easily find out if you wanted to, but choose to maintain a fiction of ignorance to avoid responsibility. Sure, if someone makes it difficult or arcane to read and understand their TOS, you're probably not (morally) bound by it. But if you close your eyes and plug your ears, you don't have much of a leg to stand on.
- brilliantcode 10y ago
  If someone breaks the TOS, you can block them or send them C&D letters. If you have the cash to take someone to court but you are going to be on the losing side. ToS is not a mutually binding agreement. Even if you have a checkbox that says "I have read the terms", it's often thrown out because nobody expects you to read EULA down to the letter.
- codedokode 10y ago
  Scraping a website can be compared to only downloading mp3 file, not redistributing it to other people. And I don't see how electronic ToS have any legal power. It is not a contract one has signed before visiting a site. I think ToS should only bind the website owner, not its visitors.

codedokode 10y ago

The person who owns a server can restrict access by adding a login form and creating user accounts. Or taking other measures (for example, banning IP networks or countries or implementing a captcha). I think that the only thing that has to be regulated is the load (number of request per unit of time) on the server. So that it doesn't prevent server from serving pages to other visitors. Regarding robots.txt, I am not sure if ignoring it should have legal consequences. It is just a hint to robots, don't visit these pages or we will ban you. If there is no harm done to the server I don't see any problem with scraping.

pdabbadabba 10y ago
> The person who owns a server can restrict access by adding a login form and creating user accounts. Or, say, returning "403 Forbidden" or using a CAPTCHA?
deleted 10y ago
[deleted]