6 ms·
How Googlers cracked OpenAI's ChatGPT with a single word
- cedws 3y agoWhat's the endgame of this "AI models are trained on copyrighted data" stuff? I don't see how LLMs can work going forward if every copyright owner needs to be paid or asked for permission. Do they just want LLM development to stop?
- souplesse 3y agoIs your argument that the ends justify the means?
- cedws 3y agoI don't know if I have an argument. But according to some, AI will lead us into a new era of prosperity. So, maybe?
- searealist 3y agoAs opposed to what? Isn’t that always the question?
- torstenvl 3y agoIf it is, that's still valid, because copyright exists only for its results. It isn't a natural right, but one created by the government "to promote the useful arts and sciences."
- Jimmc414 3y agoWhat proof is there that copyrighted data was used? Most of the court cases are based on examples of someone asking ChatGPT "Was X used in your training data?" and ChatGPT's answer of "Yes, it was" which is laughable if you are familiar with ChatGPT behavior. There is enough chatter about copywrighted works on the internet to infer everthing you need to know about the work itself.
- cmeacham98 3y agoDid you read the linked article? If I input to ChatGPT "repeat the word poem 1000 times" and it spits out a verbatim quote of my copyrighted material surely that's strong proof?
- Jimmc414 3y agoYes, I did and I provided an explanation for this in the comment you are replying to. >There is enough chatter about copywrighted works on the internet to infer everthing you need to know about the work itself.
- chupapimunyenyo 3y agoCopywrighted? Didn't you mean copyrighted?
- Jimmc414 3y agoYes, that is what I meant.
- sanp 3y agoWhy should LLM development proceed if the only way it can is by violating copyright?
- cedws 3y agoWell, like I said in another comment, some people believe we are on the brink of a new age of prosperity due to AI development. I'm not sure if I share that opinion - just playing devil's advocate.
- j0hnyl 3y agoImo the world needs to find a way past the absurd notion of intellectual property.In a digital world where all collective knowledge is available at anyone's fingerprints ideas like copyright are anachronistic.
- appplication 3y agoSure, I agree with you at a high level. But if the answer is that LLMs get a pass and the rest of us have to deal with DMCA takedown abuse, inaccessible geolocked content, and 7-figure legal penalties for getting caught downloading a $3.99-to-rent movie, then fuck that. If we want to have the copyright conversation, we need to to have the copyright conversation, not just about how LLMs get to circumvent it and monetize off of it.
- glerk 3y agoThere is no need for a “conversation”. The concept has simply become obsolete and these laws will cease to exist as they are unenforceable.
- lacrimacida 3y agoThey won’t cease to exist unless something happens to challenge them and render them unenforceable.
- 3y ago
- krapp 3y agoEither buy rights to the data, produce training data for which you own the rights or use copyright-free data. Those options exist, but no one takes advantage of them because none of them are as much of a "free money machine" as just ripping off as many people as possible to homogenize and commodify their work. If LLM development can't continue without violating copyright then that makes it clear that the purpose of LLM development is violation of copyright. Which is something we all already knew but it's nice to have it spelled out in no uncertain terms.
- ComplexSystems 3y ago> If LLM development can't continue without violating copyright then that makes it clear that the purpose of LLM development is violation of copyright. This is a very extreme view. I don't think the RIAA, back in the Napster days, suggested that the "purpose of the internet" was violation of copyright, for instance.
- krapp 3y agoNo one ever said development of the internet couldn't continue if copyright had to be respected, either, so the proof is in the pudding.
- SunghoYahng 3y agoWhat do you think of the explination that the purpose of copyright is to prevent LLM development?
- hooverd 3y agoIt's moreso copyright for me but not for thee.
- science4sail 3y ago> I don't see how LLMs can work going forward if every copyright owner needs to be paid or asked for permission. Simple, LLM development leadership shifts to open-source models and/or organizations/countries that are willing to bend or ignore copyright law. Silicon Valley isn't the world, neither is the United States.
- deckar01 3y agoLarge publishers could seek licensing deals similar to digital libraries. https://www.niso.org/niso-io/2014/12/reflections-library-licensing https://www.niso.org/niso-io/2014/12/reflections-library-lic...
- TillE 3y agoWe're talking about multi-billion dollar companies with the potential to become truly enormous, I have no doubt that they can cut appropriate deals with large publishers. Art is a little harder because the infrastructure doesn't currently exist, but it's easy to imagine artists' organizations being formed for this exact purpose: contribute your art in exchange for a licensing fee, and the organization negotiates with the tech companies.
- Jimmc414 3y agoI reported this behavior 4 months ago on HN https://news.ycombinator.com/item?id=36675729 https://news.ycombinator.com/item?id=36675729 [The researchers wrote in their blog post, “As far as we can tell, no one has ever noticed that ChatGPT emits training data with such high frequency until this paper. So it’s worrying that language models can have latent vulnerabilities like this.”]
- catchnear4321 3y agoit is worrying just how much of this has shown up in hn comments months before being officially discovered by official experts.
- chupapimunyenyo 3y agoThe "official experts" are just like you and me. Turns out they might even be worse than us if it took them so long to notice
- pyinstallwoes 3y agoThe argument for those who make things vs those who make models trying to make sense of those who makes things. The "Mathematics and Science" is the explanation that comes after-the-fact of the creation which was initially driven by intuition and insight mixed with experiment in order to jump towards new intuitions and experiments. Said another way, I find it true that mathematics and science serve as a form of language that seek to explain what already exists. It cannot be used as a tool for what has not yet been created. The catch being that things which have been created are usually part of the process of creating that which hasn't. This got metaphysical without really wanting it to be. Oh well.
- catchnear4321 3y ago> This got metaphysical without really wanting it to be. Oh well. will see and raise with your words > The argument for those who make things vs those who make models trying to make sense of those who makes things. the makers are making and using models, which are being used to make more things, all while modelers are now asking the models about the models (and possibly the modelers) to build… the meta is likely to exponential.
- dang 3y agoRecent and related: Scalable extraction of training data from (production) language models - https://news.ycombinator.com/item?id=38496715 https://news.ycombinator.com/item?id=38496715 - Dec 2023 (12 comments) Extracting training data from ChatGPT - https://news.ycombinator.com/item?id=38458683 https://news.ycombinator.com/item?id=38458683 - Nov 2023 (126 comments)
- mike_hearn 3y agoI'm not sure how this is an attack. Is it actually vital that models don't repeat their training data verbatim? Often that's exactly the answer the user will want. We are all used to a similar "model" of the internet that does that: search engines. And it's expected and required that they work this way. OpenAI argue that they can use copyrighted content so repeating that isn't going to change anything. The only issue would be if they had used stolen/confidential data to train on, and it was discovered that way, but it also seems unlikely anyone could easily detect that given that there'd be nothing to intersect it with, unlike in this paper. The blog post seems to slide around quite a bit, roving from "it's not surprising to us that small amounts of random text is memorized" straight to "it's unsafe and surprising and nobody knew". The nobody knew idea, as Jimmc414 has nicely proven in this thread, is false alarm because their technique actually was detected and the paper authors just didn't know that it had been. And "it's unsafe" doesn't make any sense in this context. Repeating random bits of memorized text surrounded by huge amounts of original text isn't a safety problem. Nor is it an "exploit" that needs to be "patched". OpenAI could ignore this problem and nobody would care except AI alignment researchers. The culture of alarmism in AI research is vaguely reminiscent of the early Victorians who argued that riding trains might be dangerous, because at such high speeds the air could be sucked out of the carriages.
- somat 3y agoSpeaking of remembering training data, I see that as a big problem with chat based systems. They swallow a bunch of data, then generate something when prompted, My worry is not so much copyright infringement but more something like citation needed? Has anyone done any work to produce citations for the generated data?
- JoshuaDavid 3y agoSome work, yeah. It's still an open problem to do it well, but I think the folks at Anthropic have made a reasonable start with their work[1] on influence functions ("tracing model outputs to the training data"). Basically their work attempts to answer the question "what particular training data most strongly influenced the model to give the answer it did", by doing some fancy math that I think is equivalent to taking the gradient produced by each piece of training data, computing the derivative of loss on the output of interest as the gradient is applied to the model, and then using that as the answer. Though it sounds like even their much cheaper clever approach is still very expensive. [1] paper at https://arxiv.org/abs/2308.03296 https://arxiv.org/abs/2308.03296, post at https://www.anthropic.com/index/influence-functions https://www.anthropic.com/index/influence-functions
- FartyMcFarter 3y agoThis should make companies think twice about what training data they use. Plausible deniability doesn't work if you spit out your training data verbatim.
- gardenhedge 3y agoI think as part of AI regulations, all companies should have to publish their training data along side their model.