3 ms·
I thought it was even worse than that: learning any of the corpus verbatim would actually reduce model utility.
by seanmcdirmid 2y ago
I thought it was even worse than that: learning any of the corpus verbatim would actually reduce model utility.
- edude03 2y agoYes although how close to verbatim is debatable. For example there are questions that you’d ask that other people have asked many times before that you’d like the exact answer for (e.g. when does daylight saving time end?)
- startupsfail 2y ago> that make it forget specific facts. These are often meant to satisfy copyright claims Facts are not copyrightable. To quote copyright.gov: “Copyright does not protect facts, ideas, systems, or methods of operation, although it may protect the way these things are expressed.”
- PittleyDunkin 2y agoWhat is a fact without expression? It's not clear under what interpretation might be necessary to get the quoted sentiment to be considered sensical.
- deleted 2y ago[deleted]
- seanmcdirmid 2y agoWouldn't that be stuffed in the prompt anyways? No reason for the LLM to learn that.
- int_19h 2y agoIt really depends on which part of the corpus, though. I do expect my LM to be able to reproduce culturally important citations, for example.