3 ms·

Billions upon billions going to these companies. 25k reward from a selected group of people if you help us determine whether or not someone can use our tool to

Billions upon billions going to these companies. 25k reward from a selected group of people if you help us determine whether or not someone can use our tool to generate weapons of mass destruction.

cbg0 6mo ago
They're probably expecting that it can be done without too much effort so they just want to see all the unique ways people are doing it.
- nativeit 6mo ago
  They’re probably expecting biological weapons of mass destruction can be created without too much effort, so are curious to see all the nifty ways people can create biological weapons of mass destruction?
  - cbg0 6mo ago
    I was talking about bypassing the ChatGPT safeguards, that's what this bug hunt is about.
Schlagbohrer 6mo ago
It's worse than that, for partial successes they encourage people to submit the attempt but reserve the right to not pay anything (they may, at their discretion, give a partial reward if they feel like it).
- staticassertion 6mo ago
  That's pretty much how every bounty works... obviously it's going to be at their discretion for an incomplete attempt.
  - adgjlsfhk1 6mo ago
    it's unusual to have to sign the NDA for a rejected bounty
    - staticassertion 6mo ago
      No it isn't. Confidentiality terms are the norm.
2ndorderthought 6mo ago
Though it could be a Honeypot they are probably hoping to train on all the ways someone might try to do this. Or maybe funds are really low and they need a smoke screen for a really bad actor to go in and try to do it for real.
chromacity 6mo ago
Because it can't and it's a publicity stunt. It achieves three goals: 1) Underscores to the general public that the models are amazingly powerful and if you're not using them, your competitors will out-innovate you, 2) Sends the message to regulators that they don't need to do anything because the companies are diligent to prevent harm, 3) Sends the message to regulators that they sure should be regulating "open-source" models, because these hippies are not doing rigorous safety testing. Both Anthropic and OpenAI have been playing that game for years.
- jfrbfbreudh 6mo ago
  If it can’t, then it makes more sense to make the bounty as high as possible instead of a measly $25k
  - duchef 6mo ago
    They don't want anyone to actually do it.
  - chromacity 6mo ago
    If it's an existential threat to humanity, and if OpenAI is valued at nearly $1T, why set the bounty at a measly $25k? The going rate for an iPhone zero-day is six to seven figures. Some companies will pay you more than $25k for a website XSS. Because this is not a serious effort to address a serious risk. It's a PR stunt, the bounty is for a simple jailbreak and not a bioweapon, and they don't necessarily want to spend a lot of money or get people really invested in breaking their safety filters.
- IAmGraydon 6mo ago
  I’m glad people are starting to recognize this, but when will the general public? Never?

Billions upon billions going to these companies. 25k reward from a selected group of people if you help us determine whether or not someone can use our tool to generate weapons of mass destruction.

cbg0 6mo ago

They're probably expecting that it can be done without too much effort so they just want to see all the unique ways people are doing it.

nativeit 6mo ago
They’re probably expecting biological weapons of mass destruction can be created without too much effort, so are curious to see all the nifty ways people can create biological weapons of mass destruction?
- cbg0 6mo ago
  I was talking about bypassing the ChatGPT safeguards, that's what this bug hunt is about.

Schlagbohrer 6mo ago

It's worse than that, for partial successes they encourage people to submit the attempt but reserve the right to not pay anything (they may, at their discretion, give a partial reward if they feel like it).

staticassertion 6mo ago
That's pretty much how every bounty works... obviously it's going to be at their discretion for an incomplete attempt.
- adgjlsfhk1 6mo ago
  it's unusual to have to sign the NDA for a rejected bounty
  - staticassertion 6mo ago
    No it isn't. Confidentiality terms are the norm.

2ndorderthought 6mo ago

Though it could be a Honeypot they are probably hoping to train on all the ways someone might try to do this. Or maybe funds are really low and they need a smoke screen for a really bad actor to go in and try to do it for real.

chromacity 6mo ago

Because it can't and it's a publicity stunt. It achieves three goals: 1) Underscores to the general public that the models are amazingly powerful and if you're not using them, your competitors will out-innovate you, 2) Sends the message to regulators that they don't need to do anything because the companies are diligent to prevent harm, 3) Sends the message to regulators that they sure should be regulating "open-source" models, because these hippies are not doing rigorous safety testing. Both Anthropic and OpenAI have been playing that game for years.

jfrbfbreudh 6mo ago
If it can’t, then it makes more sense to make the bounty as high as possible instead of a measly $25k
- duchef 6mo ago
  They don't want anyone to actually do it.
- chromacity 6mo ago
  If it's an existential threat to humanity, and if OpenAI is valued at nearly $1T, why set the bounty at a measly $25k? The going rate for an iPhone zero-day is six to seven figures. Some companies will pay you more than $25k for a website XSS. Because this is not a serious effort to address a serious risk. It's a PR stunt, the bounty is for a simple jailbreak and not a bioweapon, and they don't necessarily want to spend a lot of money or get people really invested in breaking their safety filters.
IAmGraydon 6mo ago
I’m glad people are starting to recognize this, but when will the general public? Never?