3 ms·

I just don't think the whole "you can do X but only if you use the data in ways we want" is a good idea. It's completely unenforceable, and there are millions

I just don't think the whole "you can do X but only if you use the data in ways we want" is a good idea. It's completely unenforceable, and there are millions of loopholes like the one you just said that will let them skate around the restrictions both legally and in action. This seems like an addition because of the "GeekedIn leak"[0] of a bunch of scraped GitHub data from Nov of last year, but I just don't get what it's trying to do. Bad actors are just going to ignore it like normal (or find loopholes) and good actors weren't doing this in the first place. Personally, I wish they left out the "what you can do with the data" and focused their ToS only on the "how you are allowed to scrape it". Give solid limits on amounts and speeds and other various aspects of the scraping, make them restrictive enough to prevent a service like that from working, then provide a point of contact so that an "archivist" can be allowed to do more on a case-by-case basis. [0]https://www.troyhunt.com/8-million-github-profiles-were-leaked-from-geekedins-mongodb-heres-how-to-see-yours/ https://www.troyhunt.com/8-million-github-profiles-were-leak...

koolba 10y ago
I would imagine the idea is to have a legal recourse to go after a company built atop breaking the terms of service. An example of that would be Craigslist going after Padmapper. There's tons of scraping of Craiglist and I'm pretty sure it's a violation of the terms, but they'd reserve legal enforcement to specific circumstance.
- Klathmon 10y ago
  I just think that there are better ways of limiting that (however I'll freely admit that I don't really have any experience in the area, especially the legal aspect, so I could be horribly off on this). I've seen many very cool, creative, and useful tools pop up on top of GitHub and sites like it, and I just fear that changes like this will have the (perhaps unintended) side effect of shutting them down. Things like CommitPrint [https://commitprint.com/ https://commitprint.com/], or Commits.io [https://commits.io/ https://commits.io/] come to mind.
  - delinka 10y ago
    "...up on top of GitHub..." GitHub has an API. Use of an API is not 'scraping'. The act of scraping involves pulling full HTML pages and parsing out data. Because scraping and API usage are different, there are different ToS for each. I suggest that useful tools are sticking to the API.
kriro 10y ago
Maybe not enforceable (I dunno) but the academic scraping clause seems very good. You can't really hide that you scraped from GH if you want to publish it according to accepted standards (transparency, i.e. there will be a "data was collected this way" section). So if you don't open access your work you risk writing a worse paper if you want to hide where you got the data. At the very least this gives us a nice opportunity to shame people for not OAing their Github-data based papers. I like it :)

koolba 10y ago

I would imagine the idea is to have a legal recourse to go after a company built atop breaking the terms of service. An example of that would be Craigslist going after Padmapper. There's tons of scraping of Craiglist and I'm pretty sure it's a violation of the terms, but they'd reserve legal enforcement to specific circumstance.

Klathmon 10y ago
I just think that there are better ways of limiting that (however I'll freely admit that I don't really have any experience in the area, especially the legal aspect, so I could be horribly off on this). I've seen many very cool, creative, and useful tools pop up on top of GitHub and sites like it, and I just fear that changes like this will have the (perhaps unintended) side effect of shutting them down. Things like CommitPrint [https://commitprint.com/ https://commitprint.com/], or Commits.io [https://commits.io/ https://commits.io/] come to mind.
- delinka 10y ago
  "...up on top of GitHub..." GitHub has an API. Use of an API is not 'scraping'. The act of scraping involves pulling full HTML pages and parsing out data. Because scraping and API usage are different, there are different ToS for each. I suggest that useful tools are sticking to the API.

kriro 10y ago

Maybe not enforceable (I dunno) but the academic scraping clause seems very good. You can't really hide that you scraped from GH if you want to publish it according to accepted standards (transparency, i.e. there will be a "data was collected this way" section). So if you don't open access your work you risk writing a worse paper if you want to hide where you got the data. At the very least this gives us a nice opportunity to shame people for not OAing their Github-data based papers. I like it :)