5 ms·

If the data is made available to me as a human, then I am free to delegate the job of retrieving it to a machine if I choose, and I will be doing it whether you

If the data is made available to me as a human, then I am free to delegate the job of retrieving it to a machine if I choose, and I will be doing it whether you like it or not.

madeofpalk 8y ago
Unfortunately "whether you like it or not" doesn't carry much legal weight.
- Rjevski 8y ago
  Tor and proxies solve that problem pretty nicely from experience. Not saying this is right, but if the service provider wants to play cat and mouse then I’m happy to take part.
  - jasongill 8y ago
    I don't see how using Tor would make your legal case any stronger
    - curiousgal 8y ago
      It would prevent the legal case from ever taking place, ideally.
      - toomuchtodo 8y ago
        The service provider will just block Tor.
        Rjevski 8y ago
        And there are plenty of other proxies to choose from. My point is, you can’t just put content online and expect to put restrictions on how I send the HTTP requests for it on my side. And if you think you can well I’ll do my best to prove you wrong. Edit: I totally agree that the data they’re giving me is their property and I merely have a license to use it. I’m not advocating for copyright infringement or anything like that. But if the license allows me to use the content for X purpose, then the way I request the content (whether through a browser or a scraper) shouldn’t matter.
        toomuchtodo 8y ago
        Just because you can do something doesn’t mean you should. Just because you can make an HTTP request doesn’t mean you have a right to do whatever you want with that data. You may not agree with that, but that’s what at least US law outlines.
        gazarsgo 8y ago
        There are plenty of fair use exemptions to copyright. That said, US law is not the highest of moral standards to aspire to. Especially US copyright law. We've been flaunting it online forever, because it's obviously ridiculous and still hasn't caught up with what's fair and equitable in 30 years.
        toomuchtodo 8y ago
        If you're staying within the boundaries of fair use, you only have a Terms of Service to contend with. What you want to aspire to is meaningless versus what the law of the land is (which is what is enforced). That is my point. I don't disagree that most first world copyright law is overzealous and in a lot of scenarios, unreasonable.
- LarryPage 8y ago
  "Whether you like it or not" isn't the legal part, it's the result.
- merinowool 8y ago
  That's why it is important to root out corruption from law making, nowadays called "lobbying" - this should be illegal. If you expose an endpoint to the public you can't restrict who or what can consume it. You can do throttling on your side but that's it. Otherwise this is just racism towards machines.
  - Rjevski 8y ago
    Not to mention, this still leaves a hole open as a scraping company can outsource the “scraping” to low-wage data-entry slaves in third-world countries.
  - notduncansmith 8y ago
    How do you outlaw lobbying?
    - qbrass 8y ago
      It probably involves lobbying.
  - jjeaff 8y ago
    I don't think machines are yet considered a "protected class".
  - zo1 8y ago
    Why are we even arguing about this, is what I want to know. Call a referendum and let the people decide.
- carbocation 8y ago
  This brings up the delightful distinction between malum prohibitum and malum in se.
zbentley 8y ago
Sure, but the question was about legality, not ability. Of course there are ways to reliably circumvent some laws. Some laws are stupid. Sometimes those two intersect. But those serve only to answer questions about indictment risk and whether something should be legal, not whether or not it is.
- fulafel 8y ago
  Legal rephrasing: Is there a reason that as a user of a user-agent like Firefox you would be in a legally different position than as a user of this user-agent app?
  - moarcoffee 8y ago
    Yes, because your intent is to circumvent the provided mechanisms of the Instagram API and you’re doing it with a program instead of just browsing around the site with the intent of using it like a normal user who isn’t scraping. “But the program did the exact same thing that a human could do!” is a really good, pedantic CS argument, but it’s a terrible legal argument.
    - tacotime 8y ago
      You're being down-voted, and I disagree with your use of the word pedantic here, but I believe your answer is mostly correct. "Browse wrapped" usage agreements are very real and enforceable under certain conditions. At best this is presently a legal gray area. https://en.wikipedia.org/wiki/Browse_wrap#Summary https://en.wikipedia.org/wiki/Browse_wrap#Summary Remember, anyone can sue you for anything and it is then your burden to defend yourself if they are persistent and wealthy enough.
    - deleted 8y ago
      [deleted]
anonu 8y ago
I second this opinion. The other point I would make - and of course this doesn't necessarily condone the use of scraping sites that don't want to be scraped - but I suspect scraping is being done on a massive scale - especially with sites such as IG. The influencer marketing model has become such a big one that theres a lot of valuable data to be had that you cant access via the API.
- gsich 8y ago
  >massive scale just take every search engine.

If the data is made available to me as a human, then I am free to delegate the job of retrieving it to a machine if I choose, and I will be doing it whether you like it or not.

madeofpalk 8y ago

Unfortunately "whether you like it or not" doesn't carry much legal weight.

Rjevski 8y ago
Tor and proxies solve that problem pretty nicely from experience. Not saying this is right, but if the service provider wants to play cat and mouse then I’m happy to take part.
- jasongill 8y ago
  I don't see how using Tor would make your legal case any stronger
  - curiousgal 8y ago
    It would prevent the legal case from ever taking place, ideally.
    - toomuchtodo 8y ago
      The service provider will just block Tor.
      - Rjevski 8y ago
        And there are plenty of other proxies to choose from. My point is, you can’t just put content online and expect to put restrictions on how I send the HTTP requests for it on my side. And if you think you can well I’ll do my best to prove you wrong. Edit: I totally agree that the data they’re giving me is their property and I merely have a license to use it. I’m not advocating for copyright infringement or anything like that. But if the license allows me to use the content for X purpose, then the way I request the content (whether through a browser or a scraper) shouldn’t matter.
        toomuchtodo 8y ago
        Just because you can do something doesn’t mean you should. Just because you can make an HTTP request doesn’t mean you have a right to do whatever you want with that data. You may not agree with that, but that’s what at least US law outlines.
        gazarsgo 8y ago
        There are plenty of fair use exemptions to copyright. That said, US law is not the highest of moral standards to aspire to. Especially US copyright law. We've been flaunting it online forever, because it's obviously ridiculous and still hasn't caught up with what's fair and equitable in 30 years.
        toomuchtodo 8y ago
        If you're staying within the boundaries of fair use, you only have a Terms of Service to contend with. What you want to aspire to is meaningless versus what the law of the land is (which is what is enforced). That is my point. I don't disagree that most first world copyright law is overzealous and in a lot of scenarios, unreasonable.
LarryPage 8y ago
"Whether you like it or not" isn't the legal part, it's the result.
merinowool 8y ago
That's why it is important to root out corruption from law making, nowadays called "lobbying" - this should be illegal. If you expose an endpoint to the public you can't restrict who or what can consume it. You can do throttling on your side but that's it. Otherwise this is just racism towards machines.
- Rjevski 8y ago
  Not to mention, this still leaves a hole open as a scraping company can outsource the “scraping” to low-wage data-entry slaves in third-world countries.
- notduncansmith 8y ago
  How do you outlaw lobbying?
  - qbrass 8y ago
    It probably involves lobbying.
- jjeaff 8y ago
  I don't think machines are yet considered a "protected class".
- zo1 8y ago
  Why are we even arguing about this, is what I want to know. Call a referendum and let the people decide.
carbocation 8y ago
This brings up the delightful distinction between malum prohibitum and malum in se.

zbentley 8y ago

Sure, but the question was about legality, not ability. Of course there are ways to reliably circumvent some laws. Some laws are stupid. Sometimes those two intersect. But those serve only to answer questions about indictment risk and whether something should be legal, not whether or not it is.

fulafel 8y ago
Legal rephrasing: Is there a reason that as a user of a user-agent like Firefox you would be in a legally different position than as a user of this user-agent app?
- moarcoffee 8y ago
  Yes, because your intent is to circumvent the provided mechanisms of the Instagram API and you’re doing it with a program instead of just browsing around the site with the intent of using it like a normal user who isn’t scraping. “But the program did the exact same thing that a human could do!” is a really good, pedantic CS argument, but it’s a terrible legal argument.
  - tacotime 8y ago
    You're being down-voted, and I disagree with your use of the word pedantic here, but I believe your answer is mostly correct. "Browse wrapped" usage agreements are very real and enforceable under certain conditions. At best this is presently a legal gray area. https://en.wikipedia.org/wiki/Browse_wrap#Summary https://en.wikipedia.org/wiki/Browse_wrap#Summary Remember, anyone can sue you for anything and it is then your burden to defend yourself if they are persistent and wealthy enough.
  - deleted 8y ago
    [deleted]

anonu 8y ago

I second this opinion. The other point I would make - and of course this doesn't necessarily condone the use of scraping sites that don't want to be scraped - but I suspect scraping is being done on a massive scale - especially with sites such as IG. The influencer marketing model has become such a big one that theres a lot of valuable data to be had that you cant access via the API.

gsich 8y ago
>massive scale just take every search engine.