5 ms·

One of the difficulties with these training datasets is the currently understood rules around web scraping. The current legal precedent [0] is that web scraping

One of the difficulties with these training datasets is the currently understood rules around web scraping. The current legal precedent [0] is that web scraping is perfectly legal, despite what is in the websites terms of service, "licence" or robots.txt. If a human can navigate to it freely, you can scrape it using automated means. What you can't do with scraped data is republish it verbatim. Doing a data analysis on scraped data is permitted by law, and you can publish your analysis of that data. The question is, is an AI model trained on scraped data a derived analysis that is therefore legal? Or is it republishing of the original data? We need a test case to find out. In the case of this dataset, I don't think the CC license applies to people using it. It "may" apply to redistribution of it for free. If the dataset was sold, that would be a violation. I suspect (after tested in court) a model trained on this dataset would be allowed despite the CC license on the photos. Personally, in this case I think the ethics committee of the University should have put up barriers to the project. The morals of this are questionable at best. 0: https://techcrunch.com/2022/04/18/web-scraping-legal-court/ https://techcrunch.com/2022/04/18/web-scraping-legal-court/

KRAKRISMOTT 4y ago
You can do the scraping in a jurisdiction where it is legal.
- pbhjpbhj 4y ago
  Importing (in the geographical sense) the data would still be infringing, you've just scraped it in a convoluted way -- legal systems in my limited experience take account of such things.
- traceroute66 4y ago
  > You can do the scraping in a jurisdiction where it is legal. No such thing with GDPR. Why do you think so many US websites take the lazy-ass approach and block EU visitors to their websites ? Simple, its because either you comply with GDPR or you don't process the information of citizens of GDPR covered countries. End of story.
  - pixl97 4y ago
    If I'm in China and I scrape/collect data I don't think the GDPR is going to do anything to me. This really only affects businesses that some the EU has some means of reaching.
  - deleted 4y ago
    [deleted]
  - laingc 4y ago
    Well, no, only if you’re under the jurisdiction of the EU courts. They can rule against you as much as they like, but it’s not enforceable outside of the EU or a jurisdiction that chooses to enforce EU judgements.
    - traceroute66 4y ago
      > Well, no, only if you’re under the jurisdiction of the EU courts. That is an awfully naïve argument my friend. If it were that simple then there would, for example, be no need for a 68 page document entitled "The Sedona Conference Commentary on the Enforceability in U.S. Courts of Orders and Judgments Entered under GDPR"[1]. Allow me to quote from the Conclusion on page 68: "As the Commentary shows, the enforceability of GDPR orders and judgments in a U.S. court will depend on several factors, including the nature of the relief sought through the order or judgment, the nature of the underlying violation and the process through which the order or judgment was initially obtained in the EU, and the U.S. organization’s contacts with the EU." I would say that makes it pretty darn clear that it's far from being a simple argument about the jurisdiction in which the defendant is based. [1]https://www.dorsey.com/~/media/files/newsresources/publications/2021/01/cattanach-commentary-on-us-enforcement-of-gdpr-jan.pdf https://www.dorsey.com/~/media/files/newsresources/publicati...
      - int_19h 4y ago
        It's not about jurisdiction in which the defendant is based, of course, but rather in jurisdiction where it has presence. If foreign laws were enforceable against actors who never operated in those other countries, we'd have to enforce Saudi laws against atheism and Russian laws against "gay propaganda".
the_duke 4y ago
It's not as easy as that. Pictures are clearly personally identifiable data, so storing them violates the GDPR if you don't have permission to do so. Some "data analysis company" got fined a hefty sum for doing so with EU citizens. I forgot the name, but they were recently in the news for helping Ukraine identify Russian soldiers by picture. Of course they were also aggregating other data including names, so just pictures might be a more complicated case, but as a company with EU exposure I wouldn't do it. It's pretty clearly against the law.
- samwillis 4y ago
  You are quite right, forgot that one. Point is though, we need a test case to go through the courts to clarify all of this. There are companies betting billions on the outcome that they are ok to do what they are doing.
  - voakbasda 4y ago
    They are betting they can make billions of dollars while the courts screw around trying to decide the legality. They don’t care if it is legal; they only care about making money today. This is the standard playbook for big money these days: profit until the courts decide you can’t.
- fleddr 4y ago
  "Pictures are clearly personally identifiable data, so storing them violates the GDPR if you don't have permission to do so." Wouldn't a Creative Commons license express this permission?
  - kixiQu 4y ago
    IANAL, but I believe no; the CC license handles the rights that a photographer can hand out, but doesn't come with any kind of model release guarantees.
    - fleddr 4y ago
pbhjpbhj 4y ago
>Or is it republishing of the original data? If it's publishing _data_ then you're fine under regular copyright as it only protects artistic works and not things like data. You might fall shy of other IP legislation but not copyright. YMMV, this is not legal advice and represents my personal opinion unrelated to my employment.
- fragmede 4y ago
  The CFAA would be the thing to look out for.
- cmeacham98 4y ago
  The "data" here is photographs, which all jurisdictions I'm aware of treat as coprightable.
  - pbhjpbhj 4y ago
    FWIW in UK it's possible for works to be too generic to attract copyright, or to be slavish reproductions (eg as photos). ID images have their parameters dictated by technical needs - no smiling, plain background, even lighting, no eyewear, head only, face-on - and so leave no room for artistry. An ID photo might lack copyright. I know of no caselaw here (on copyright in ID shots) and am projecting from eg the "red bus" case (Temple Island v New English Tea).
  - Mtinie 4y ago
    Which makes this case even more interesting to me. Some percentage of those photos’ copywrites’ are owned by corporations rather than the pictured individuals. If it was simply a large group of selfies, I don’t expect much legal challenge from the allegedly aggrieved. But when companies with legal counsel get involved…
- messe 4y ago
  Yep, in the EU you can run afoul of database rights. This is separate from copyright.
satvikpendem 4y ago
There was an updated ruling in November 2022 showing that HiQ was ruled against and that they reached a settlement with LinkedIn, so I'm not sure that web scraping is entirely legal. https://www.natlawreview.com/article/hiq-and-linkedin-reach-proposed-settlement-landmark-scraping-case https://www.natlawreview.com/article/hiq-and-linkedin-reach-...
- EMIRELADERO 4y ago
  That was because Linkedin added a no-scraping clause to their ToS and also put up a login wall for viewing profiles in the first place. If you scraped from a web page without actually signing up for an account you wouldn't be accepting the terms and would thus be legally in the clear.
JumpCrisscross 4y ago
Beyond copyright, how would these requirements work with Illinois’ biometrics law?
charcircuit 4y ago
A ML model would be considered transformational.
- janalsncm 4y ago
  Depends? What about Naive Bayes?
  - contravariant 4y ago
    Why would that be less transformational than other models?
drusepth 4y ago
>One of the difficulties with these training datasets is the currently understood rules around web scraping. The current legal precedent [0] is that web scraping is perfectly legal, despite what is in the websites terms of service, "licence" or robots.txt. If a human can navigate to it freely, you can scrape it using automated means. So... the HN crowd seems perfect to ask, since everyone's always clamoring for a Goodreads alternative: where is the line between "data" and "content" (on public pages)? AFAICT, Goodreads has been entrenched for so long because their ToS doesn't allow anyone else to use their book/reviews data, which makes quite a large moat. This comment would make me think it's legal to scrape that moat.
langsoul-com 4y ago
There was an article talking about how big companies fund researchers to harvest datasets and train models. Then those companies use those models. Even though, in the tos, it's not for commercial use.

KRAKRISMOTT 4y ago

You can do the scraping in a jurisdiction where it is legal.

pbhjpbhj 4y ago
Importing (in the geographical sense) the data would still be infringing, you've just scraped it in a convoluted way -- legal systems in my limited experience take account of such things.
traceroute66 4y ago
> You can do the scraping in a jurisdiction where it is legal. No such thing with GDPR. Why do you think so many US websites take the lazy-ass approach and block EU visitors to their websites ? Simple, its because either you comply with GDPR or you don't process the information of citizens of GDPR covered countries. End of story.
- pixl97 4y ago
  If I'm in China and I scrape/collect data I don't think the GDPR is going to do anything to me. This really only affects businesses that some the EU has some means of reaching.
- deleted 4y ago
  [deleted]
- laingc 4y ago
  Well, no, only if you’re under the jurisdiction of the EU courts. They can rule against you as much as they like, but it’s not enforceable outside of the EU or a jurisdiction that chooses to enforce EU judgements.
  - traceroute66 4y ago
    > Well, no, only if you’re under the jurisdiction of the EU courts. That is an awfully naïve argument my friend. If it were that simple then there would, for example, be no need for a 68 page document entitled "The Sedona Conference Commentary on the Enforceability in U.S. Courts of Orders and Judgments Entered under GDPR"[1]. Allow me to quote from the Conclusion on page 68: "As the Commentary shows, the enforceability of GDPR orders and judgments in a U.S. court will depend on several factors, including the nature of the relief sought through the order or judgment, the nature of the underlying violation and the process through which the order or judgment was initially obtained in the EU, and the U.S. organization’s contacts with the EU." I would say that makes it pretty darn clear that it's far from being a simple argument about the jurisdiction in which the defendant is based. [1]https://www.dorsey.com/~/media/files/newsresources/publications/2021/01/cattanach-commentary-on-us-enforcement-of-gdpr-jan.pdf https://www.dorsey.com/~/media/files/newsresources/publicati...
    - int_19h 4y ago
      It's not about jurisdiction in which the defendant is based, of course, but rather in jurisdiction where it has presence. If foreign laws were enforceable against actors who never operated in those other countries, we'd have to enforce Saudi laws against atheism and Russian laws against "gay propaganda".

the_duke 4y ago

It's not as easy as that. Pictures are clearly personally identifiable data, so storing them violates the GDPR if you don't have permission to do so. Some "data analysis company" got fined a hefty sum for doing so with EU citizens. I forgot the name, but they were recently in the news for helping Ukraine identify Russian soldiers by picture. Of course they were also aggregating other data including names, so just pictures might be a more complicated case, but as a company with EU exposure I wouldn't do it. It's pretty clearly against the law.

samwillis 4y ago
You are quite right, forgot that one. Point is though, we need a test case to go through the courts to clarify all of this. There are companies betting billions on the outcome that they are ok to do what they are doing.
- voakbasda 4y ago
  They are betting they can make billions of dollars while the courts screw around trying to decide the legality. They don’t care if it is legal; they only care about making money today. This is the standard playbook for big money these days: profit until the courts decide you can’t.
fleddr 4y ago
"Pictures are clearly personally identifiable data, so storing them violates the GDPR if you don't have permission to do so." Wouldn't a Creative Commons license express this permission?
- kixiQu 4y ago
  IANAL, but I believe no; the CC license handles the rights that a photographer can hand out, but doesn't come with any kind of model release guarantees.
  - fleddr 4y ago

pbhjpbhj 4y ago

>Or is it republishing of the original data? If it's publishing _data_ then you're fine under regular copyright as it only protects artistic works and not things like data. You might fall shy of other IP legislation but not copyright. YMMV, this is not legal advice and represents my personal opinion unrelated to my employment.

fragmede 4y ago
The CFAA would be the thing to look out for.
cmeacham98 4y ago
The "data" here is photographs, which all jurisdictions I'm aware of treat as coprightable.
- pbhjpbhj 4y ago
  FWIW in UK it's possible for works to be too generic to attract copyright, or to be slavish reproductions (eg as photos). ID images have their parameters dictated by technical needs - no smiling, plain background, even lighting, no eyewear, head only, face-on - and so leave no room for artistry. An ID photo might lack copyright. I know of no caselaw here (on copyright in ID shots) and am projecting from eg the "red bus" case (Temple Island v New English Tea).
- Mtinie 4y ago
  Which makes this case even more interesting to me. Some percentage of those photos’ copywrites’ are owned by corporations rather than the pictured individuals. If it was simply a large group of selfies, I don’t expect much legal challenge from the allegedly aggrieved. But when companies with legal counsel get involved…
messe 4y ago
Yep, in the EU you can run afoul of database rights. This is separate from copyright.

satvikpendem 4y ago

There was an updated ruling in November 2022 showing that HiQ was ruled against and that they reached a settlement with LinkedIn, so I'm not sure that web scraping is entirely legal. https://www.natlawreview.com/article/hiq-and-linkedin-reach-proposed-settlement-landmark-scraping-case https://www.natlawreview.com/article/hiq-and-linkedin-reach-...

EMIRELADERO 4y ago
That was because Linkedin added a no-scraping clause to their ToS and also put up a login wall for viewing profiles in the first place. If you scraped from a web page without actually signing up for an account you wouldn't be accepting the terms and would thus be legally in the clear.

JumpCrisscross 4y ago

Beyond copyright, how would these requirements work with Illinois’ biometrics law?

charcircuit 4y ago

A ML model would be considered transformational.

janalsncm 4y ago
Depends? What about Naive Bayes?
- contravariant 4y ago
  Why would that be less transformational than other models?

drusepth 4y ago

>One of the difficulties with these training datasets is the currently understood rules around web scraping. The current legal precedent [0] is that web scraping is perfectly legal, despite what is in the websites terms of service, "licence" or robots.txt. If a human can navigate to it freely, you can scrape it using automated means. So... the HN crowd seems perfect to ask, since everyone's always clamoring for a Goodreads alternative: where is the line between "data" and "content" (on public pages)? AFAICT, Goodreads has been entrenched for so long because their ToS doesn't allow anyone else to use their book/reviews data, which makes quite a large moat. This comment would make me think it's legal to scrape that moat.

langsoul-com 4y ago

There was an article talking about how big companies fund researchers to harvest datasets and train models. Then those companies use those models. Even though, in the tos, it's not for commercial use.