5 ms·
"Hack" is defined as "using deceptive prompts" which is a very misleading use of the word - using creative prompts is absolutely how people use ChatGPT, even if
by klik99 3y ago
"Hack" is defined as "using deceptive prompts" which is a very misleading use of the word - using creative prompts is absolutely how people use ChatGPT, even if it isn't "expected behaviour". There is zero evidence that AI isn't leaky and that there will always be a way of returning the original corpus. Even if it takes a thousand tries, that means one out of thousand users would get it. This is probably the only good point in their rebuttal: that NYT "had to feed the tool portions of the very articles they sought to elicit verbatim passages".
That being said, this is probably the first case that will determine the future of AI so staking a strong, extreme rebuttal is how the system should work, and the courts will decide the truth. It's going to be very interesting to see how this case progresses.
Preemptive Edit: "Truth" might not be the best word, but rather how this should be seen in context of copyright law. It's very unclear how AI will legally be allowed to operate since corpuses commonly include copyrighted material - this is a question, due to how US law is structured, that HAS to be litigated in court for a clear answer. Just because you don't like OpenAIs argument, and I don't, this is how the legal system is designed to work.
Edit 2: I feel many people are missing the point here (which I buried in the second paragraph, my bad) - This isn't some catty PR fight between NYT and OpenAI - this is the first major case about how LLMs/deep learning/etc fits in relation to copyright. This isn't about what is morally right, this is about do LLMs violate copyright law. NYT has made a strong case that it does violate copyright. OpenAI has some good and bad arguments (IMO 1 very good argument and the rest weak or bad) but they should be throwing their best arguments at the wall since that is how law works in the US. This won't be the last, but this is probably the first important case in the future of LLMs.
- amelius 3y agoHacking can be defined as "using a system in a way it was not intended to be used". In that sense, it was hacking.
- falcor84 3y agoVery good point! Essentially it's hacking in the same sense as used here on Hacker News.
- etiam 3y ago"hack us" isn't idiomatic with that sense of hack at all though, is it.
- klik99 3y agoYeah, I'd agree it's hacking in that definition of the word, but obviously OpenAI is using it in the boogieman scary evil guy hacking definition of the word. It is semantics, but to clarify definitions OpenAI, despite sama obviously knowing and using our definition of the word, is not using it in that sense. Otherwise the court document would read "NYT paid someone to creatively create a solution to generating copyrighted text" and would actually be praising NYT for that, instead it reads like they are damning them.
- BadHumans 3y agoThat is not what people see that word and think. In fact, we are so far away from that definition of the word that most people here wouldn't see that word and think that and OpenAI knows this.
- throwaway55671 3y agoOn the contrary, I think that's exactly what people think (except maybe people on this site). See the loads of GPT "prompt hacking" and "jailbreaking" guides...
- BadHumans 3y agoYou may be living in a bubble if you think the people who are talking about "prompt hacking" are representative of the average person who reads headlines like these.
- throwaway55671 3y agoWell, I'm talking about the 180 million ChatGPT users. A bubble, but a large one.
- BadHumans 3y agoI can promise you most ChatGPT users don't even know what a GPT is. That's like saying if you own an Apple Silicon Macbook then you must know what ARM architecture is.
- signatoremo 3y agoAre those guides featured in mainstream media that most of these people consume? Most ChatGPT users don’t know what LLM is, let alone “prompt engineering”.
- klik99 3y agoExploiting a bug, for example SQL injection, is completely a hack. But can it be considered a bug if it's inherent and unsolvable in the system? Can AI truly ever be resistant against prompt injection or will it always be a series of patches to cover up unwanted behaviour? (Genuine question - I do not know) If something is inherent in the design of the system I'd argue that it is not a bug and not hacking, just as it's not "hacking" to use a hunters rifle to shoot a human, despite it being "not the way it was intended to be used". It can still be illegal of course, but that's a separate issue.
- arp242 3y agoIn context it's clearly not intended to mean that. The quote is: "paid someone to hack OpenAI's products". This is clearly "hack" in the "malicious actor" type of "hack" rather than "Hacker News" type of "hack".
- me_me_me 3y agoStill the headline is deceptive. They 'tried to hack chatGTP model', not gain access to project or something. More colloquially when someone says: 'someone tried to hack us', it means the hacker tried to gain access to the companies backend servers.
- ShamelessC 3y agoMental gymnastics is useless in _actual _ communication.
- MrNeon 3y ago> Even if it takes a thousand tries, that means one out of thousand users would get it. If they use that particular prompt that primes the model for regurgitation. It's not one in a thousand of regular queries.
- klik99 3y agoYeah, I call out priming with text from the original model as the only good point in the article. Making a thousand tries is not the problem, but priming it with text from the original article could be. It'll be very interesting to see how the court decides that, since I can see an argument for priming it with a sentence or two from the original article either for or against that breaking copyright. And again, this is a question of law, not morals, so the question is does that break the law or not.