3 ms·
I am curious about this too, from a copyright perspective: 1) Whether it is legal for Common Crawl to collect this data in the first place? Presumably they don
by nodamage 6y ago
I am curious about this too, from a copyright perspective:
1) Whether it is legal for Common Crawl to collect this data in the first place? Presumably they don't have explicit permission from every site they have crawled?
and
2) Whether it is legal to use this data to train ML algorithms for commercial use (e.g. GPT-3)? They provide a bunch of examples of this on their site but the terms of use are kind of vague about this.
- visarga 6y agoThe data is open they don't crawl inside password protected areas of the internet. Why shouldn't they be able to read data put into the open? If the authors didn't want their data openly accessible they should put it in a private place.
- nodamage 6y agoExcept just because someone published something on the internet doesn't mean you have the legal right to reproduce and redistribute it yourself.
- nonameiguess 6y agoIt has to be legal to use legally acquired published information for a commercial purpose. That is not nearly the same thing as selling someone else's work. Training a ML algorithm on published work is no different from me reading a tutorial on the web and using the new skills I learned to do something that makes me money. That is very different from reselling the tutorial as if I authored it.