7 ms·
> “Do you ask for permission when you train your mind on copyrighted books?” I’m not able to read billions of books in less than an hour. Even if we agree tha
by pavlov 4y ago
> “Do you ask for permission when you train your mind on copyrighted books?”
I’m not able to read billions of books in less than an hour.
Even if we agree that machine learning is like human learning, scale commonly matters in law.
- kirse 4y agoI’m not able to read billions of books in less than an hour. I think you underestimate the sheer volume of data + conclusions the brain ingests and processes on a daily basis, primarily through unconscious experience.
- ben_w 4y agoThe training set for GPT-3 is about 500e9 tokens; any given synapse in a human in their lifetime is going to fire about 2e9s * (10% * {100Hz to 1000Hz}) = 20e9 to 200e9 times.
- sillysaurusx 4y agoIt sounds like you agree with them, since there are a lot of synapses.
- ben_w 4y agoAu contraire. While our brains are more complex than the networks, this has never been in dispute. The quantity of experiences needed to train GPT-3, however, is many more than we are capable of experiencing in a lifetime.
- kennyloginz 4y agoNot not mention the millions of years building our nervous system.
- jacquesm 4y ago100's of millions of years.
- sinity 4y agoGPT-3 is learning from ridiculously dense data through. Also, about volume of data processed by the brain: https://gwern.net/Differences https://gwern.net/Differences
- visarga 4y agoI recently worked on information extraction from 10K documents. GPT-3 needs about 7 days of operation in batch mode on one thread. It takes 40..70s to read one single document and report the extracted data. One MINUTE per page. But I think you meant GPT-3 has seen many books during training, not during inference. You should know that training on millions of books is not the only way GPT-3 learns. It is just the foundation of its knowledge. GPT-3 learns "in-context", that means it can learn a new word or a new task at first sight. It just needs a description or a few examples. This is the most powerful feature of GPT-3 - in-context learning. And when it comes to ICL, it is much like humans - only sees a few examples, not millions of books. > “Do you ask for permission when you train your mind on copyrighted books?” The nature of ICL is that it happens at prediction time. So GPT-3 would have to explicitly be instructed to learn a specific skill. Should it reject instructions if they are sourced from copyrighted books?
- pavlov 4y ago> “You should know that training on millions of books is not the only way GPT-3 learns. It is just the foundation of its knowledge.” I’m not a lawyer, but to me it seems within the realm of possibility that a U.S. court eventually finds strongly in favor of the copyright holders, the Supreme Court agrees (because Big Tech has so few friends left), and OpenAI will be required to destroy the GPT-3 model and all copies of the training data because they can’t filter out copyrighted works.
- 1248 4y ago[dead]
- notahacker 4y agoYeah, I think "we can't actually tell which bits of our model are derived from your work which we copied without your permission onto our system for training purposes probably makes AI companies more vulnerable rather than less. Other platforms like search and social media have successfully defended themselves by showing willingness to promptly remove copyrighted material, let copyright holders opt out of indexing or even negotiating stuff like ContentID so copyright holders get paid each time their work is used.
- usrbinbash 4y ago> scale commonly matters in law I am not a lawyer, so the following is only my opinion. Scale matters, but so does the legality of the thing that scales. Reading two dozen books by other authors, or studying hundreds of artworks, or visiting the museum of awesome statues every week, in order to get inspired for ones own novel/painting/scuplture, isn't illegal. So a lawsuit will have a really hard time argueing that it somehow is a problem if its two dozen billion books/paintings/sculptures. Because, such a lawsuit would suddely need to explain why the smaller scale is also problematic, only less so. And given that this is basically how art worked ever since the first human had the idea to paint pictures on a cave wall, that's a hard sell.
- foota 4y agoYou make it sound like we should assume that getting inspiration from a few hundred or thousand art works that are very famous and highly public is the same as training over nearly every available public piece of art. I see no reason why that should be our null hypothesis. Humans either learn art by being natural art geniuses, or by receiving instruction and learning through an iterative process (where, again, they might create thousands of art works, but nowhere near the scale here), which is very different.
- usrbinbash 4y ago> which is very different. Why is it different? The only difference that matters, is scale. And again, if I want to argue that something done 10000000000 times is legally problematic, I have to be prepared to explain why doing it 10 times is problematic as well, only less so.
- RandomLensman 4y agoNothing new. People differentiate between genocide and murder, for example, or poisoning water supply vs an individual poisoning. Criminal law in quite a few places definitely has scale considerations.
- 4y ago
- i_like_apis 4y agoIs this about jealousy then? Or it’s just too fast so let’s stop it?
- manholio 4y agoIt's about a law designed by humans to give everyone a chance to make a living and contribute to the common good. You think you have found a loophole in that law that lets you use that work for free and deny author's any compensation. Bear in mind, AI is not making artists or creative types obsolete - that would be fair game, just like computers made human calculators obsolete. No, this is about abusing other people's work.
- i_like_apis 4y agoI’m not seeing a salient argument here. Something about an allegation of abuse? Where is the abuse happening?
- unusualmonkey 4y agoCopyright never guaranteed anyone compensation, nor is it loophole to not pay for copyright just because you saw and learned from something in the public domain. If using someone else work for learning is infringement, then that's going to cause a lot of difficulty for all artists. Try making a rock song without listening to rock, or paint some modern art without viewing it etc.
- manholio 4y agoCopyright exists to protect human authors and promote creation, which in turn leads to learning - from other human beings. Algorithms are not learning, they are automated tools which are consuming creative works and outputing derivative works. The loophole is to use copyrighted works for free despite no learning taking place - no human being observing and developing their skills based on that work - rather, an algorithm transforming those works into some other useful interpretation of them.
- allturtles 4y agoI think the Aereo case is an interesting precedent [0] [1]. An individual DVR-ing over-the-air broadcasts with an antenna was fine. A corporation DVR-ing over-the-air-broadcasts for thousands of customers by using thousands of tiny antennas was not fine. [0]: https://en.wikipedia.org/wiki/American_Broadcasting_Cos.,_Inc._v._Aereo,_Inc https://en.wikipedia.org/wiki/American_Broadcasting_Cos.,_In.... [1]: https://www.vox.com/2018/11/7/18073200/aereo https://www.vox.com/2018/11/7/18073200/aereo