5 ms·

If I’m reading the complaint correctly, the “hack” is what amounts to…prompt injection? Are we seriously considering that a hack these days? Genuine question.

by doytch 3y ago

nuc1e0n 3y ago
So by using ChatGPT at all you can be considered as hacking at OpenAI's discretion? You can't make this kind of absurdity up. Were OpenAI hacking when they scraped the whole internet?
shrubble 3y ago
Aren't there 1000s of people on Twitter doing this for free, so they can do a screencap and share it?
tananaev 3y ago
I think the main point of the article is that they had to do some serious prompt engineering to get the results they wanted. It probably means that with normal usage it doesn't produce any quotes from copyrighted materials.
- asadotzler 3y ago
  normal usage doesn't come into play here. if the model contains verbatim text owned by the NYT, how that was demonstrated in immaterial to it's theft. You don't get to steal from me, lock up my propery inside you safe, and when I pop your safe (non-destructively) claim I'm cheating by showing the cops my shit inside your popped safe.
  - seanmcdirmid 3y ago
    The model doesn’t contain verbatim text, the huge prompt might (tailored for each query), that’s the whole point of LLMs right?
    - 15457345234 3y ago
      Define 'contain' If I can reliably recite verbatim a 10 page short story then I 'contain' it even though you can't dissect my brain and find it. If a LLM can reliably and repeatably recite an article verbatim then it 'contains' it even if you can't run 'strings' on the weights and find it. Disagree? You might, but the court's opinion is what matters, and I think they'll go with the 'touch and feel' judgement not the 'bits and bytes' judgement.
      - seanmcdirmid 3y ago
        Again, if you feed the article into the prompt over multiple queries, why would the model not regurgitate it? It doesn’t have to be in the model, you are basically giving the article to your chat session. The analogy holds, and I don’t think the court is going to be ignorant enough to fall for that trick.
tylerrobinson 3y ago
A snarky reply to your genuine question, but I suppose if “prompt engineering” is “engineering” then “prompt hacking” is “hacking” :)
- Fnoord 3y ago
  I like the fruit analogy: New fruits are discovered/invented. An apple is fruit, but a fruit is not necessarily an apple. When new fruits are discovered, the average definition (or perception of that definition) of fruit is also modified.
keenmaster 3y ago
It’s beating loose slander with less loose slander. Seems fair game to me: “ OpenAI believes that it took tens of thousands of attempts to get ChatGPT to produce the controversial output that’s the basis of this lawsuit. This is not how normal people interact with its service, it notes.” I think the substance of OpenAI’s complaint is valid - think of the word “hacking” as clickbait to the real heart of the matter which is that New York Times went to obscene lengths to reproduce meaningful amounts of article text and, by withholding their methodology, misrepresented the behavior of OpenAI’s product. There are much easier ways to get NYT content for free than making tens of thousands of attempts to reproduce an NYT article while violating OpenAI terms of service and repeatedly ignoring GPT’s refusals to an insane* degree. *not normal insane, insane to the 10th power
- eigenket 3y ago
  What OpenAI claimes here does not seem to be true. Here is an article where ars technica tried it https://arstechnica.com/tech-policy/2023/12/ny-times-sues-open-ai-microsoft-over-copyright-infringement/ https://arstechnica.com/tech-policy/2023/12/ny-times-sues-op... And this is a screenshot of their session with copilot https://cdn.arstechnica.net/wp-content/uploads/2023/12/Screen-Shot-2023-12-27-at-12.11.54-PM.png https://cdn.arstechnica.net/wp-content/uploads/2023/12/Scree...
  - keenmaster 3y ago
    Ars tried and failed to reproduce the text, and then assumed that OpenAI closed a loophole when in fact they may have changed nothing - it’s just that NYT hired a team of experts to make tens of thousands of attempts to break the normal behavior of the model, while Ars didn’t: “ ChatGPT has apparently closed that loophole in between the preparation of that suit and the present. We entered some of the prompts shown in the suit, and were advised "I recommend checking The New York Times website or other reputable sources," although we can't rule out that context provided prior to that prompt could produce copyrighted material.” If I tried to do what NYT did, I would: - give up after a few attempts - get worried that I’m going to be banned from using OpenAI (a rational concern, because it doesn’t take a lot to get banned) - doubt the veracity of any “reproduced” text that I may receive The equilibrium here is for OpenAI to suggest signing up for NYT and providing a signup link if it detects interest in doing so.
paxys 3y ago
It isn't about what we consider a hack, but what a 70 year old judge who asks his grandkids to help set up his iPhone considers a hack.
- brigadier132 3y ago
  It's about what the law as defined by congress considers a hack.
  - etiam 3y ago
    Not that I've read the background materials for that legislation, but what I'd expect is the law as such doesn't have very specific considerations for that, and the implementation for arbitrating the details is not too far removed from what paxys just described?
- silverquiet 3y ago
  This trope feels quite out of touch these days. My dad is 70 and brought home a 486 when I was a single-digit age, and I distinctly remember him telling me not to tell my mom how much he paid for a 28.8K modem. The people who were key in building our modern tech were Boomers, and a 60 year old person today was born on the edge of Boomer/Gen X.
  - hombre_fatal 3y ago
    It’s out of touch because you provided an exception? I don’t think that’s how tropes work.
    - silverquiet 3y ago
      It’s out of touch because the take itself is 30 years old. The people who could get away with never touching a computer for their work are mostly dead already.
- arp242 3y ago
  This blatant ageism is ridiculous. 60 year olds are just as capable of understanding that words in English can mean more than one thing. Someone 60 today was 20 in 1984, when small "home computers" were already widespread. Stallman is 70. Larry Wall is 69. Guido van Rossum is 68. Eric S. Raymond is 66. That young student Linus Torvalds is 54.
eviks 3y ago
Not in the nefarious way implied in this context, but otherwise yes, making the app bend to your will against how the devs want it to behave is part of a broad "hacking" category

If I’m reading the complaint correctly, the “hack” is what amounts to…prompt injection? Are we seriously considering that a hack these days? Genuine question.

by doytch 3y ago

If I’m reading the complaint correctly, the “hack” is what amounts to…prompt injection? Are we seriously considering that a hack these days? Genuine question.

nuc1e0n 3y ago

So by using ChatGPT at all you can be considered as hacking at OpenAI's discretion? You can't make this kind of absurdity up. Were OpenAI hacking when they scraped the whole internet?

shrubble 3y ago

Aren't there 1000s of people on Twitter doing this for free, so they can do a screencap and share it?

tananaev 3y ago

I think the main point of the article is that they had to do some serious prompt engineering to get the results they wanted. It probably means that with normal usage it doesn't produce any quotes from copyrighted materials.

asadotzler 3y ago
normal usage doesn't come into play here. if the model contains verbatim text owned by the NYT, how that was demonstrated in immaterial to it's theft. You don't get to steal from me, lock up my propery inside you safe, and when I pop your safe (non-destructively) claim I'm cheating by showing the cops my shit inside your popped safe.
- seanmcdirmid 3y ago
  The model doesn’t contain verbatim text, the huge prompt might (tailored for each query), that’s the whole point of LLMs right?
  - 15457345234 3y ago
    Define 'contain' If I can reliably recite verbatim a 10 page short story then I 'contain' it even though you can't dissect my brain and find it. If a LLM can reliably and repeatably recite an article verbatim then it 'contains' it even if you can't run 'strings' on the weights and find it. Disagree? You might, but the court's opinion is what matters, and I think they'll go with the 'touch and feel' judgement not the 'bits and bytes' judgement.
    - seanmcdirmid 3y ago
      Again, if you feed the article into the prompt over multiple queries, why would the model not regurgitate it? It doesn’t have to be in the model, you are basically giving the article to your chat session. The analogy holds, and I don’t think the court is going to be ignorant enough to fall for that trick.

tylerrobinson 3y ago

A snarky reply to your genuine question, but I suppose if “prompt engineering” is “engineering” then “prompt hacking” is “hacking” :)

Fnoord 3y ago
I like the fruit analogy: New fruits are discovered/invented. An apple is fruit, but a fruit is not necessarily an apple. When new fruits are discovered, the average definition (or perception of that definition) of fruit is also modified.

keenmaster 3y ago

It’s beating loose slander with less loose slander. Seems fair game to me: “ OpenAI believes that it took tens of thousands of attempts to get ChatGPT to produce the controversial output that’s the basis of this lawsuit. This is not how normal people interact with its service, it notes.” I think the substance of OpenAI’s complaint is valid - think of the word “hacking” as clickbait to the real heart of the matter which is that New York Times went to obscene lengths to reproduce meaningful amounts of article text and, by withholding their methodology, misrepresented the behavior of OpenAI’s product. There are much easier ways to get NYT content for free than making tens of thousands of attempts to reproduce an NYT article while violating OpenAI terms of service and repeatedly ignoring GPT’s refusals to an insane* degree. *not normal insane, insane to the 10th power

eigenket 3y ago
What OpenAI claimes here does not seem to be true. Here is an article where ars technica tried it https://arstechnica.com/tech-policy/2023/12/ny-times-sues-open-ai-microsoft-over-copyright-infringement/ https://arstechnica.com/tech-policy/2023/12/ny-times-sues-op... And this is a screenshot of their session with copilot https://cdn.arstechnica.net/wp-content/uploads/2023/12/Screen-Shot-2023-12-27-at-12.11.54-PM.png https://cdn.arstechnica.net/wp-content/uploads/2023/12/Scree...
- keenmaster 3y ago
  Ars tried and failed to reproduce the text, and then assumed that OpenAI closed a loophole when in fact they may have changed nothing - it’s just that NYT hired a team of experts to make tens of thousands of attempts to break the normal behavior of the model, while Ars didn’t: “ ChatGPT has apparently closed that loophole in between the preparation of that suit and the present. We entered some of the prompts shown in the suit, and were advised "I recommend checking The New York Times website or other reputable sources," although we can't rule out that context provided prior to that prompt could produce copyrighted material.” If I tried to do what NYT did, I would: - give up after a few attempts - get worried that I’m going to be banned from using OpenAI (a rational concern, because it doesn’t take a lot to get banned) - doubt the veracity of any “reproduced” text that I may receive The equilibrium here is for OpenAI to suggest signing up for NYT and providing a signup link if it detects interest in doing so.

paxys 3y ago

It isn't about what we consider a hack, but what a 70 year old judge who asks his grandkids to help set up his iPhone considers a hack.

brigadier132 3y ago
It's about what the law as defined by congress considers a hack.
- etiam 3y ago
  Not that I've read the background materials for that legislation, but what I'd expect is the law as such doesn't have very specific considerations for that, and the implementation for arbitrating the details is not too far removed from what paxys just described?
silverquiet 3y ago
This trope feels quite out of touch these days. My dad is 70 and brought home a 486 when I was a single-digit age, and I distinctly remember him telling me not to tell my mom how much he paid for a 28.8K modem. The people who were key in building our modern tech were Boomers, and a 60 year old person today was born on the edge of Boomer/Gen X.
- hombre_fatal 3y ago
  It’s out of touch because you provided an exception? I don’t think that’s how tropes work.
  - silverquiet 3y ago
    It’s out of touch because the take itself is 30 years old. The people who could get away with never touching a computer for their work are mostly dead already.
arp242 3y ago
This blatant ageism is ridiculous. 60 year olds are just as capable of understanding that words in English can mean more than one thing. Someone 60 today was 20 in 1984, when small "home computers" were already widespread. Stallman is 70. Larry Wall is 69. Guido van Rossum is 68. Eric S. Raymond is 66. That young student Linus Torvalds is 54.

eviks 3y ago

Not in the nefarious way implied in this context, but otherwise yes, making the app bend to your will against how the devs want it to behave is part of a broad "hacking" category