5 ms·

Yes, but I suppose you can't use them in a publication on Nature if they are scrapped illegally. This dataset directly from the dating website

sillysaurusx 6y ago
Ha. Hahaha. I wish. I'm sorry to laugh, but a ton of ML papers are based on illegally-scraped datasets of one form or another, unless they use strictly blessed datasets (Imagenet2012 being the gold standard mostly-useless-in-the-real-world dataset). OpenAI's Jukebox is based on illegal large-scale gathering of copyrighted material, for example.
freebreakfast 6y ago
What does "scrapped illegally" mean? I've never encountered this term. I can see how scrapping might be a violation of some websites terms of use, but I've never seen "scrapped illegally" used. Do you have any examples?
- emteycz 6y ago
  Well, pictures of faces could be considered personal data per GDPR. Scraping that data without each person's approval could be illegal regardless of any terms of use.
- Dayshine 6y ago
  - I have personal information on linkedin - I have agreed with LinkedIn that they may use my personal information for a set of well-defined uses (basically things on the LinkedIn website/service, and some 3rd party services they use to run the website/service). - LinkedIn promise that they will not share my identifiable personal information with 3rd parties for any use - LinkedIn's terms of use state that nobody may scrape personal information from their website without their consent. This is how they enforce the previous promise to me - Some business comes along and scrapes my personal information for their own business use. - That business knows that LinkedIn prohibit this, and they know that I have only consented for my personal information to be used for LinkedIn itself. - This is probably "unlawful" (as they're interfering in my contract with LinkedIn), and certainly violating my GDPR rights. Sadly, it's hard to point at a specific example as guidance doesn't have a section titled "Can I ignore individual's explicit opting out of my usage?". Hence, illegal scraping, as willing violating the GDPR is illegal. Just to head-off the very common response: Personal, individual, use is not covered by the GDPR. So there is nothing wrong with you going and using my LinkedIn data for any personal reasons. The moment you try to use it for business purposes though, that's illegal.
  - FDSGSG 6y ago
    Just like the US isn't the entire world, neither is EU.
astura 6y ago
1) Web scraping is not "illegal" 2) I haven't the slightest clue why you think scraped data can't be used in a publication - some results from first page of Google Scholar: https://www.sciencedirect.com/science/article/pii/S1878029617300464 https://www.sciencedirect.com/science/article/pii/S187802961... https://journals.sagepub.com/doi/abs/10.1177/0042098020918196 https://journals.sagepub.com/doi/abs/10.1177/004209802091819...

sillysaurusx 6y ago

Ha. Hahaha. I wish. I'm sorry to laugh, but a ton of ML papers are based on illegally-scraped datasets of one form or another, unless they use strictly blessed datasets (Imagenet2012 being the gold standard mostly-useless-in-the-real-world dataset). OpenAI's Jukebox is based on illegal large-scale gathering of copyrighted material, for example.

freebreakfast 6y ago

What does "scrapped illegally" mean? I've never encountered this term. I can see how scrapping might be a violation of some websites terms of use, but I've never seen "scrapped illegally" used. Do you have any examples?

emteycz 6y ago
Well, pictures of faces could be considered personal data per GDPR. Scraping that data without each person's approval could be illegal regardless of any terms of use.
Dayshine 6y ago
- I have personal information on linkedin - I have agreed with LinkedIn that they may use my personal information for a set of well-defined uses (basically things on the LinkedIn website/service, and some 3rd party services they use to run the website/service). - LinkedIn promise that they will not share my identifiable personal information with 3rd parties for any use - LinkedIn's terms of use state that nobody may scrape personal information from their website without their consent. This is how they enforce the previous promise to me - Some business comes along and scrapes my personal information for their own business use. - That business knows that LinkedIn prohibit this, and they know that I have only consented for my personal information to be used for LinkedIn itself. - This is probably "unlawful" (as they're interfering in my contract with LinkedIn), and certainly violating my GDPR rights. Sadly, it's hard to point at a specific example as guidance doesn't have a section titled "Can I ignore individual's explicit opting out of my usage?". Hence, illegal scraping, as willing violating the GDPR is illegal. Just to head-off the very common response: Personal, individual, use is not covered by the GDPR. So there is nothing wrong with you going and using my LinkedIn data for any personal reasons. The moment you try to use it for business purposes though, that's illegal.
- FDSGSG 6y ago
  Just like the US isn't the entire world, neither is EU.

astura 6y ago

1) Web scraping is not "illegal" 2) I haven't the slightest clue why you think scraped data can't be used in a publication - some results from first page of Google Scholar: https://www.sciencedirect.com/science/article/pii/S1878029617300464 https://www.sciencedirect.com/science/article/pii/S187802961... https://journals.sagepub.com/doi/abs/10.1177/0042098020918196 https://journals.sagepub.com/doi/abs/10.1177/004209802091819...