4 ms·
Do you have a credible source that says an LLM like the ones trained by OpenAI can perfectly memorize millions of books? Or be trained in a single day?
by Kim_Bruning 3y ago
Do you have a credible source that says an LLM like the ones trained by OpenAI can perfectly memorize millions of books? Or be trained in a single day?
- vkoskiv 3y agoNo, but the point still stands.
- chankstein38 3y agoWhat point? To me it sounds like they shot down your entire comment
- nickthegreek 3y agoNo it doesnt. Kim just took your entire thesis away.
- vkoskiv 3y agoAre you really trying to argue that there exist humans that can learn facts at a similar pace to a datacenter running GPT training software on petabytes of scraped data? My point still stands.
- freejazz 3y agoDoes anyone have a credible source indicating that LLM learning is anything like human learning? That's the first step in this argument
- ghaff 3y agoWell, we would probably have to understand human learning a lot better first.
- freejazz 3y ago"Your honor, I don't know human minds work, but clearly LLMs work the same way" The legal burden of evidence is on LLM proponents to establish that what they are doing is the same as the human mind and therefore should be treated the same way.
- Kim_Bruning 3y agoThat's a bit of a false dilemma. To address copyright issues, we needn't prove that machine learning models learn exactly like humans or not. The more relevant point is that neither human learning nor machine learning has the intent to store or replicate copyrighted material; both aim to generalize from data to produce new content. It is in this way that they are similar.
- freejazz 3y agoIt's the argument that is being made. Intent isn't a requisite for copyright infringement. Your re-characterization of the argument is so general that it's useless.
- Kim_Bruning 3y agoHeh, we seem to be talking past each other. I wonder if this might be because Slavboj might be a monist/physicalist, and you might be a dualist[2]? If that's the case we'd all argue until we're blue in the face, if we don't at least recognize this underlying difference. For the record, since I've studied biology, I'm probably closest to some form of mechanism[3] (due to the rejection of vis vitalis [4] in early 20th c. ) Of course, you could also just be a very skeptical monist mindful of the kluger hans effect[5] and working from there. Let me know which (if any), maybe we can still find middle ground! [1] https://en.wikipedia.org/wiki/Physicalism https://en.wikipedia.org/wiki/Physicalism [2] https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism [3] https://en.wikipedia.org/wiki/Mechanism_(philosophy) https://en.wikipedia.org/wiki/Mechanism_(philosophy) [4] https://en.wikipedia.org/wiki/Vitalism https://en.wikipedia.org/wiki/Vitalism [5] https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans
- throwawaymaths 3y agoIt's actually provably impossible for openAI to perfectly memorize millions of books.
- gkbrk 3y agoIt's provably impossible for a model with 1.76 trillion floating-point parameters (like GPT-4) to memorize millions of books? How many bytes do you think a million compressed books takes? Consider that the way these models are trained is basically completing the next symbol based on the previous words, which is how most compressors are made.