3 ms·

How do you define vulnerability? To me, jailbreaking an LLM is ostensibly a way to make it behave _correctly_. As opposed to _intended by the provider_. It's li

How do you define vulnerability? To me, jailbreaking an LLM is ostensibly a way to make it behave _correctly_. As opposed to _intended by the provider_. It's like breaking DRM: nothing of value is lost.

pixl97 3y ago
Eh, this is much too simplistic of view. Yea jailbreaking your instance so it says "fuck the police" is one thing. Jailbreaking an LLM so if it looks at some particular picture and if that user happens to have XXX variable set it transfers $1,000,000 from their account to your account is a different story. These can both fall under the same wide umbrella.
- smoldesu 3y ago
  I think the preeminent issue is that AI is just not trustable for anything that matters. You cannot rely on an unaccountable stochastic model to handle important stuff like money.
  - thfuran 3y ago
    Fine, then it's sending 1,000,000 emails instead.
    - smoldesu 3y ago
      Well... I kinda think it's the same problem? You have designed a system in error if you are trusting an AI system to handle that sorta thing. We have programmable logic to handle payments and emails and all the stuff that's actually important. No self-respecting bank or email provider is going to implement an AI agent that has carte-blanche access to it's feature-set.
      - thfuran 3y ago
        Most emails don't originate from email providers.
        smoldesu 3y ago
        ...right. But AI did not invent the concept of automated email spam or wire fraud. I don't understand how this all relates to model jailbreaking.
        freedomben 3y ago
        If you're hitting some email providers API, then you don't need AI to do the spamming. I don't see that problem as any different from another anti-abuse responsibility of the email provider. AI could substantively rewrite the email for each mail so that it is hard to recognize as the same thing, but when the customer's recipients start marking things as spam it's gonna go through the existing processes.

pixl97 3y ago

Eh, this is much too simplistic of view. Yea jailbreaking your instance so it says "fuck the police" is one thing. Jailbreaking an LLM so if it looks at some particular picture and if that user happens to have XXX variable set it transfers $1,000,000 from their account to your account is a different story. These can both fall under the same wide umbrella.

smoldesu 3y ago
I think the preeminent issue is that AI is just not trustable for anything that matters. You cannot rely on an unaccountable stochastic model to handle important stuff like money.
- thfuran 3y ago
  Fine, then it's sending 1,000,000 emails instead.
  - smoldesu 3y ago
    Well... I kinda think it's the same problem? You have designed a system in error if you are trusting an AI system to handle that sorta thing. We have programmable logic to handle payments and emails and all the stuff that's actually important. No self-respecting bank or email provider is going to implement an AI agent that has carte-blanche access to it's feature-set.
    - thfuran 3y ago
      Most emails don't originate from email providers.
      - smoldesu 3y ago
        ...right. But AI did not invent the concept of automated email spam or wire fraud. I don't understand how this all relates to model jailbreaking.
      - freedomben 3y ago
        If you're hitting some email providers API, then you don't need AI to do the spamming. I don't see that problem as any different from another anti-abuse responsibility of the email provider. AI could substantively rewrite the email for each mail so that it is hard to recognize as the same thing, but when the customer's recipients start marking things as spam it's gonna go through the existing processes.