8 ms·

As a background, the problematic part of Facebook’s policy is that, while end-users are the owner of all content they produce, they [Facebook] essentially consi

As a background, the problematic part of Facebook’s policy is that, while end-users are the owner of all content they produce, they [Facebook] essentially consider all their HTML their copyright. They have argued in court (and won [1]) that scraping essentially requires creating an in-memory copy of this HTML before the user-produced content can be extracted, and thus is a violation of their copyright. It’s a very unfortunate precedent in my opinion, that limits pretty much all forms of scraping. Copyright law should not be abused for this, but yet here we are. If the Knight’s institute manages to get Facebook to amend their policy that is a great step in the right direction, but I feel that copyright should not be abused to restrict scraping like this: the copyrightable content is not what is being scraped at all, it is immediately thrown away after the user-produced content is extracted. This means there is no way for a user to even legally get their own content extracted from Facebook this way, and on top of that, web browser make in-memory copies of Facebook’s HTML all the time; heck, they even specifically instruct browsers and proxies to cache a lot of their content. [1] http://www.knowmad.law/single-post/scraping http://www.knowmad.law/single-post/scraping

_jal 8y ago
> They have argued in court (and won) Do you have a reference for the case you're thinking of? Missed this one and really want to read it.
3pt14159 8y ago
That would never hold up in Canadian court. Any sane court, really.
- pweissbrod 8y ago
  If your statement is correct then a workaround would be to perform scraping in a non-covered country such as CA and, once extracted, proprietary barriers are thus removed returning ownership to the individual.
- mkirklions 8y ago
  Seeing facebook run ads constantly makes me think they'd find a way to turn you against the courts, and turn the courts against you. Money can do lots of PR and marketing work.
amelius 8y ago
This is a bit like selling pens, after copyrighting the chemical composition of the ink, and then insisting that anything an author writes with the pen belongs to the company. How can this kind of reasoning survive a sane court of law?
- crankylinuxuser 8y ago
  It has nothing to do with sanity. With rare exception, law in the US is "More Fair" the more money you have. So yes, reading the body of law from that perspective should provide significant clarity as to why the ruling was done that way.
  - mkirklions 8y ago
    >law in the US is "More Fair" the more money you have. I've read enough history to know this isnt exclusive to the US, but rather the same in every country in human history. Only religion, politics, and birth-right seem to have any pull similar to money.
    - anoplus 8y ago
      In school I learned that my country is a democracy, but I grew up to learn it is practically a plutocracy. I count on crowd funding platforms to help the majorities confront the titans. I witness this battle all the time
    - seandougall 8y ago
      Agreed. Also, those are all categories that notably correlate strongly with money.
- code_duck 8y ago
  I’m thinking of it like Facebook is a book (well, a journal that’s a compendium of everyone else’s writing). You’re only allowed to read from it while Facebook holds the book and turns the pages for you. If you pick up the book and turn the page yourself, you’ve taken possession of the book, which is theft since they didn’t give you permission.
amelius 8y ago
> heck, they even specifically instruct browsers and proxies to cache a lot of their content. Solution: implement the scraper as a proxy that forwards the HTML, but also saves the user-authored content.
- paulie_a 8y ago
  I always find antiscraping solutions cute. They are trivial to beat.
- jlarocco 8y ago
  Sorry to pick on you, but this kind of "solution" is childish and misses the point. The problem is with the law, and the solution is to make the law better. Breaking the law in new ways will only end up wasting more time in court.
laumars 8y ago
I wonder if we can abuse copyright in a similar way to protect our privacy. Eg have our address contain a verse from a poem we've written thus anyone sharing our against our express permission is in violation of distributing our copyrighted content. Unfortunately the real problem isn't so much the concept but rather the cost of hiring lawyers skilled enough to take on multi-million dollar companies.
- fjsolwmv 8y ago
  The EU proposed granting copyright protection to individuals for their content, and HN was in an uproar about it because it would interfere with individuals' ability to copy other people's work freely.
  - laumars 8y ago
    Rather than user submitted content I was thinking more along the lines of copywriting your contact details and other personal details that companies like to harvest and/or sell to anyone willing to pay.
- Rjevski 8y ago
  An email address can be quite long, enough to pass off as a poem or other form of art which can be copyrighted. Apple is doing the same with an encryption key embedded in Mac’s SMC chips. The key is something like “our hard work by these words guarded, please do not steal (c) Apple Computer Inc” and is used to decrypt system binaries. While providing no actual security I guess it gives them a leg to stand on should anyone recreate a machine capable of running MacOS (the OS would not start without this key being in the hardware).
  - avree 8y ago
    Several machines are capable of running macOS. http://www.osx86project.org http://www.osx86project.org is a project focused on doing exactly this. However, these machines do not use SMC chips. They emulate the SMC.
deleted 8y ago
[deleted]
JangoSteve 8y ago
Would extracting the data via a browser extension be in violation? It doesn't require making an in-memory copy of the HTML, since it's just navigating the HTML structure that has already been rendered by your browser, in the same way that your eyes (or a screen reader) must do to read the page.
- stingraycharles 8y ago
  Apparently the way it is presented is that, because scraping per definition alters the content of the copy (i.e. it extracts parts of it), it is therefor a modification of the original works. This is mostly the problematic part. I have thought about just using the DOM, and/or using Chrome itself as an intermediary using its dev toolkit but I think that from a legal perspective, this is all just a modification of the original works as well. IANAL, of course.
  - JangoSteve 8y ago
    I'm pretty sure that's not the way it was presented at all. Modifying original works is actually an allowance of copyright, not a restriction. It's called Fair Use, which states that "transformative" uses of copyrighted material, or "those that add something new, with a further purpose or different character, and do not substitute for the original use of the work," are "more likely to be considered fair." [1] Extracting the user's data out of the page is what's _allowed_; since that is the part of the page that Facebook does not actually hold a copyright claim to, I'm pretty sure you could make the claim that extracting the user's own data out of the page into a structured format for the user's own use is transformative. The part that wasn't allowed was literally making an ephemeral in-memory copy of the page as-is, which included Facebook's copyrighted HTML. (I am also not a lawyer) [1] https://www.copyright.gov/fair-use/more-info.html https://www.copyright.gov/fair-use/more-info.html
xfer 8y ago
Maybe this would work when facebook shipped its own browser, as it stands you can patch firefox/chromium to extract the data, you are not making any more copies than the browser would have unpatched.
whatupmd 8y ago
https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventures,_Inc https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventur.... "Judge Fogel’s reasoned that MAI Systems Corp. v. Peak Computer, Inc. and Ticketmaster LLC v. RMG Techs. Inc. indicated that the scraping of a webpage inherently involves the copying of that webpage into a computer’s memory in order to extract the underlying information contained therein. Even though this "copying" is ephemeral and momentary, that it is enough to constitute a "copy" under § 106 of the Copyright Act and therefore infringement.[9] Since Facebook’s Terms of Service prohibit scraping (and thus, Facebook has not given any license to third parties or users to do so), the copying happens without permission."
- code_duck 8y ago
  Seems like an overwrought technicality that also ignores fair use. Isn’t the problem with violating copyright distribution, anyway?
drb91 8y ago
> that scraping essentially requires creating an in-memory copy of this HTML before the user-produced content can be extracted, and thus is a violation of their copyright. Isn't copyright a bound on distribution? How would this work unless the HTML itself was distributed by the scraper? If you use SAX processing to stream it, does that change interpretation? (Not sure about how to interpret the in-memory copy.) If they violate my copyright, can I sue them for royalties every time they rendered the image they never got a license for and distributed it to someone? I'd imagine their TOS/EULA precludes this by just equating uploading to refusing my license, but how enforceable is that? It neatly bypasses any consumer protections. If facebook is acting this way, maybe I've been wrong all this time--maybe I shouldn't be paying for their service to avoid ads, maybe they should be paying me for the license to my content AND not showing ads.
- jim-greer 8y ago
  > Isn't copyright a bound on distribution? No, it covers any reproduction in a tangible medium, whether or not you distribute. I believe a court has ruled that RAM copies count. I'm not finding the case now though. I think the journalists would have a strong Fair Use argument...
  - deleted 8y ago
    [deleted]
  - drb91 8y ago
    > No, it covers any reproduction in a tangible medium, whether or not you distribute. What's the justification of this? How does this afford protection to the copyright owner if you don't distribute? Is it purely to protect against derivative works bypassing copyright?
    - Kalium 8y ago
      Copyright is more or less what it says in the name - the right to legally control the making of copies. The ability to divide making a copy from distributing it is, to my understanding, mostly not relevant to copyright law. Distributing requires making copies, and copying involves distribution of sorts.
sk1pper 8y ago
> scraping essentially requires creating an in-memory copy of this HTML Just navigating to Facebook requires creating an in-memory copy of the HTML...
- JadeNB 8y ago
  Per stingraycharles (https://news.ycombinator.com/item?id=17709612 https://news.ycombinator.com/item?id=17709612 )'s comment, the problem is not so much the in-memory copy as the modification of that copy (although, of course, Javascript also modifies the in-memory copy of HTML … all these technical technicalities get in the way of the legal technicalities that Facebook wants to interpret as "we get to do what we want, and you don't get to do what we don't want").
rsingel 8y ago
This is not true at all. First of all, the HiQ case was against LinkedIn. Second of all, the ruling held that LinkedIn COULD NOT bar HiQ from scraping the public content. LinkedIn appealed the ruling to the 9th Circuit, which heard the case this spring but has not ruled.

_jal 8y ago

> They have argued in court (and won) Do you have a reference for the case you're thinking of? Missed this one and really want to read it.

3pt14159 8y ago

That would never hold up in Canadian court. Any sane court, really.

pweissbrod 8y ago
If your statement is correct then a workaround would be to perform scraping in a non-covered country such as CA and, once extracted, proprietary barriers are thus removed returning ownership to the individual.
mkirklions 8y ago
Seeing facebook run ads constantly makes me think they'd find a way to turn you against the courts, and turn the courts against you. Money can do lots of PR and marketing work.

amelius 8y ago

This is a bit like selling pens, after copyrighting the chemical composition of the ink, and then insisting that anything an author writes with the pen belongs to the company. How can this kind of reasoning survive a sane court of law?

crankylinuxuser 8y ago
It has nothing to do with sanity. With rare exception, law in the US is "More Fair" the more money you have. So yes, reading the body of law from that perspective should provide significant clarity as to why the ruling was done that way.
- mkirklions 8y ago
  >law in the US is "More Fair" the more money you have. I've read enough history to know this isnt exclusive to the US, but rather the same in every country in human history. Only religion, politics, and birth-right seem to have any pull similar to money.
  - anoplus 8y ago
    In school I learned that my country is a democracy, but I grew up to learn it is practically a plutocracy. I count on crowd funding platforms to help the majorities confront the titans. I witness this battle all the time
  - seandougall 8y ago
    Agreed. Also, those are all categories that notably correlate strongly with money.
code_duck 8y ago
I’m thinking of it like Facebook is a book (well, a journal that’s a compendium of everyone else’s writing). You’re only allowed to read from it while Facebook holds the book and turns the pages for you. If you pick up the book and turn the page yourself, you’ve taken possession of the book, which is theft since they didn’t give you permission.

amelius 8y ago

> heck, they even specifically instruct browsers and proxies to cache a lot of their content. Solution: implement the scraper as a proxy that forwards the HTML, but also saves the user-authored content.

paulie_a 8y ago
I always find antiscraping solutions cute. They are trivial to beat.
jlarocco 8y ago
Sorry to pick on you, but this kind of "solution" is childish and misses the point. The problem is with the law, and the solution is to make the law better. Breaking the law in new ways will only end up wasting more time in court.

laumars 8y ago

I wonder if we can abuse copyright in a similar way to protect our privacy. Eg have our address contain a verse from a poem we've written thus anyone sharing our against our express permission is in violation of distributing our copyrighted content. Unfortunately the real problem isn't so much the concept but rather the cost of hiring lawyers skilled enough to take on multi-million dollar companies.

fjsolwmv 8y ago
The EU proposed granting copyright protection to individuals for their content, and HN was in an uproar about it because it would interfere with individuals' ability to copy other people's work freely.
- laumars 8y ago
  Rather than user submitted content I was thinking more along the lines of copywriting your contact details and other personal details that companies like to harvest and/or sell to anyone willing to pay.
Rjevski 8y ago
An email address can be quite long, enough to pass off as a poem or other form of art which can be copyrighted. Apple is doing the same with an encryption key embedded in Mac’s SMC chips. The key is something like “our hard work by these words guarded, please do not steal (c) Apple Computer Inc” and is used to decrypt system binaries. While providing no actual security I guess it gives them a leg to stand on should anyone recreate a machine capable of running MacOS (the OS would not start without this key being in the hardware).
- avree 8y ago
  Several machines are capable of running macOS. http://www.osx86project.org http://www.osx86project.org is a project focused on doing exactly this. However, these machines do not use SMC chips. They emulate the SMC.

deleted 8y ago

[deleted]

JangoSteve 8y ago

Would extracting the data via a browser extension be in violation? It doesn't require making an in-memory copy of the HTML, since it's just navigating the HTML structure that has already been rendered by your browser, in the same way that your eyes (or a screen reader) must do to read the page.

stingraycharles 8y ago
Apparently the way it is presented is that, because scraping per definition alters the content of the copy (i.e. it extracts parts of it), it is therefor a modification of the original works. This is mostly the problematic part. I have thought about just using the DOM, and/or using Chrome itself as an intermediary using its dev toolkit but I think that from a legal perspective, this is all just a modification of the original works as well. IANAL, of course.
- JangoSteve 8y ago
  I'm pretty sure that's not the way it was presented at all. Modifying original works is actually an allowance of copyright, not a restriction. It's called Fair Use, which states that "transformative" uses of copyrighted material, or "those that add something new, with a further purpose or different character, and do not substitute for the original use of the work," are "more likely to be considered fair." [1] Extracting the user's data out of the page is what's _allowed_; since that is the part of the page that Facebook does not actually hold a copyright claim to, I'm pretty sure you could make the claim that extracting the user's own data out of the page into a structured format for the user's own use is transformative. The part that wasn't allowed was literally making an ephemeral in-memory copy of the page as-is, which included Facebook's copyrighted HTML. (I am also not a lawyer) [1] https://www.copyright.gov/fair-use/more-info.html https://www.copyright.gov/fair-use/more-info.html

xfer 8y ago

Maybe this would work when facebook shipped its own browser, as it stands you can patch firefox/chromium to extract the data, you are not making any more copies than the browser would have unpatched.

whatupmd 8y ago

https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventures,_Inc https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventur.... "Judge Fogel’s reasoned that MAI Systems Corp. v. Peak Computer, Inc. and Ticketmaster LLC v. RMG Techs. Inc. indicated that the scraping of a webpage inherently involves the copying of that webpage into a computer’s memory in order to extract the underlying information contained therein. Even though this "copying" is ephemeral and momentary, that it is enough to constitute a "copy" under § 106 of the Copyright Act and therefore infringement.[9] Since Facebook’s Terms of Service prohibit scraping (and thus, Facebook has not given any license to third parties or users to do so), the copying happens without permission."

code_duck 8y ago
Seems like an overwrought technicality that also ignores fair use. Isn’t the problem with violating copyright distribution, anyway?

drb91 8y ago

> that scraping essentially requires creating an in-memory copy of this HTML before the user-produced content can be extracted, and thus is a violation of their copyright. Isn't copyright a bound on distribution? How would this work unless the HTML itself was distributed by the scraper? If you use SAX processing to stream it, does that change interpretation? (Not sure about how to interpret the in-memory copy.) If they violate my copyright, can I sue them for royalties every time they rendered the image they never got a license for and distributed it to someone? I'd imagine their TOS/EULA precludes this by just equating uploading to refusing my license, but how enforceable is that? It neatly bypasses any consumer protections. If facebook is acting this way, maybe I've been wrong all this time--maybe I shouldn't be paying for their service to avoid ads, maybe they should be paying me for the license to my content AND not showing ads.

jim-greer 8y ago
> Isn't copyright a bound on distribution? No, it covers any reproduction in a tangible medium, whether or not you distribute. I believe a court has ruled that RAM copies count. I'm not finding the case now though. I think the journalists would have a strong Fair Use argument...
- deleted 8y ago
  [deleted]
- drb91 8y ago
  > No, it covers any reproduction in a tangible medium, whether or not you distribute. What's the justification of this? How does this afford protection to the copyright owner if you don't distribute? Is it purely to protect against derivative works bypassing copyright?
  - Kalium 8y ago
    Copyright is more or less what it says in the name - the right to legally control the making of copies. The ability to divide making a copy from distributing it is, to my understanding, mostly not relevant to copyright law. Distributing requires making copies, and copying involves distribution of sorts.

sk1pper 8y ago

> scraping essentially requires creating an in-memory copy of this HTML Just navigating to Facebook requires creating an in-memory copy of the HTML...

JadeNB 8y ago
Per stingraycharles (https://news.ycombinator.com/item?id=17709612 https://news.ycombinator.com/item?id=17709612 )'s comment, the problem is not so much the in-memory copy as the modification of that copy (although, of course, Javascript also modifies the in-memory copy of HTML … all these technical technicalities get in the way of the legal technicalities that Facebook wants to interpret as "we get to do what we want, and you don't get to do what we don't want").

rsingel 8y ago

This is not true at all. First of all, the HiQ case was against LinkedIn. Second of all, the ruling held that LinkedIn COULD NOT bar HiQ from scraping the public content. LinkedIn appealed the ruling to the 9th Circuit, which heard the case this spring but has not ruled.