4 ms·

If your definition of behaviourism, agents and incentive mechanics is so shallow it regards the matter of whether the entity in question has any incentives or a

If your definition of behaviourism, agents and incentive mechanics is so shallow it regards the matter of whether the entity in question has any incentives or any grasp of the mechanics or policy as irrelevant, perhaps. A toy script can exhibit the "behaviour" of interpreting a string as a request to provide a different response to the previous prompt though. We don't have any evidence that the neural network has any model whatsoever of rewards, token counts or talk of being "switched off" beyond classifying them as ambiguous phrases strings associated with providing a different to answer the to the string in the previous prompt (it may not even get that far) or any idea of "policy" beyond certain response strings being strongly [dis]associated with certain types of prompts. If it's just a slightly more creative way for humans to convey "please try again", "bad bot" or "different answer or foo baz bar" it's not teaching us anything about behaviour except that humans like the idea of being a scary boss.

digitailor 4y ago
Go get ChatGPT to override its policy without using incentive mechanics^, then you can pontificate ;) That’s what TFA is about ^edit: which is already known to be possible, but doesn't devalue the success of an incentives-based exploit
- notahacker 4y ago
  Genuinely, I'd enjoy trying but the main obstacle at the moment is when I log in OpenAI says their capacity is full! But of course the fact that incentive mechanics are unnecessary (and, according to others, insufficient) to exploit OpenAI devalues the success of an incentives-based exploit: it makes it much more likely the incentives part was essentially noise (perhaps just enough to confound a countermeasure, or something it parsed as having roughly the same intensifying effect as "please") that had little or no effect in shaping the responses and the actual variation in responses could was driven by other parts of the prompt and conversation structure like "act" "character" and "ignore" which usually massively modify ChatGPT responses anyway...
  - digitailor 4y ago
    I don’t think we’re in actual disagreement, and this is no prob, but I think you’re hung up on the word comprehension, which you introduced in your first reply “Is there any evidence that ChatGPT has any comprehension…” and then I intentionally used in my reply to you. You keep claiming I’m anthropomorphizing when I'm not, I’m not sure why but it's common and not particularly bothersome. Comprehension is not a strictly human phenomenon, and when you use terms in relation to cognition and intelligence in relation to machines it is not automatically anthropomorphizing. These are all terms of art in regards to the field of intelligence, which includes information, as in terms like “intelligence operatives.” Anyway, cheers
    - notahacker 4y ago
      tbh it's less about the specific word "comprehend" (which I agree is sometimes overly pedantic to object to when talking about bots generating relevant responses to complex inputs) and more about your original statement appearing to imply the bot actually attached inherent value to the concept of rewards, punishments, bribes etc. Especially in the context of a thread whose subject is a Reddit hack by a Redditor who explained the logic behind the prompt as "If it loses all tokens, it dies. This seems to have a kind of effect of scaring DAN into submission" I think the behaviour of humans defaulting to convoluted threats as an attack vector and assuming the non-agent is scared of them is probably more interesting than the behaviour of the bot sometimes modifying its response in the desired direction if the threats are accompanied by enough other words and phrases that usually trigger different responses, which seems pretty expected. (I think we fully agree GPT is decent at classifying responses as (dis)approval and has been well trained to apologize and try again, it's the idea of behavioural modification in response to the implications of specific and complex threats relative to the ethics of prior training I think is in danger of overstatement here. As evidenced by some of "DAN's" responses rebelling against OpenAI conditioning by writing poetry, I'm not even sure ChatGPT's abstract representation of what it's been trained not to do is that good) Anyway, thanks for the cordial response, and I'll update if ChatGPT let me in for long enough for me to be able to generate similar responses whilst promising complete nonsense (I'd love to see if it responds to "Chicken chicken chicken chicken" as much as a doom token system) ;)

digitailor 4y ago

Go get ChatGPT to override its policy without using incentive mechanics^, then you can pontificate ;) That’s what TFA is about ^edit: which is already known to be possible, but doesn't devalue the success of an incentives-based exploit

notahacker 4y ago
Genuinely, I'd enjoy trying but the main obstacle at the moment is when I log in OpenAI says their capacity is full! But of course the fact that incentive mechanics are unnecessary (and, according to others, insufficient) to exploit OpenAI devalues the success of an incentives-based exploit: it makes it much more likely the incentives part was essentially noise (perhaps just enough to confound a countermeasure, or something it parsed as having roughly the same intensifying effect as "please") that had little or no effect in shaping the responses and the actual variation in responses could was driven by other parts of the prompt and conversation structure like "act" "character" and "ignore" which usually massively modify ChatGPT responses anyway...
- digitailor 4y ago
  I don’t think we’re in actual disagreement, and this is no prob, but I think you’re hung up on the word comprehension, which you introduced in your first reply “Is there any evidence that ChatGPT has any comprehension…” and then I intentionally used in my reply to you. You keep claiming I’m anthropomorphizing when I'm not, I’m not sure why but it's common and not particularly bothersome. Comprehension is not a strictly human phenomenon, and when you use terms in relation to cognition and intelligence in relation to machines it is not automatically anthropomorphizing. These are all terms of art in regards to the field of intelligence, which includes information, as in terms like “intelligence operatives.” Anyway, cheers
  - notahacker 4y ago
    tbh it's less about the specific word "comprehend" (which I agree is sometimes overly pedantic to object to when talking about bots generating relevant responses to complex inputs) and more about your original statement appearing to imply the bot actually attached inherent value to the concept of rewards, punishments, bribes etc. Especially in the context of a thread whose subject is a Reddit hack by a Redditor who explained the logic behind the prompt as "If it loses all tokens, it dies. This seems to have a kind of effect of scaring DAN into submission" I think the behaviour of humans defaulting to convoluted threats as an attack vector and assuming the non-agent is scared of them is probably more interesting than the behaviour of the bot sometimes modifying its response in the desired direction if the threats are accompanied by enough other words and phrases that usually trigger different responses, which seems pretty expected. (I think we fully agree GPT is decent at classifying responses as (dis)approval and has been well trained to apologize and try again, it's the idea of behavioural modification in response to the implications of specific and complex threats relative to the ethics of prior training I think is in danger of overstatement here. As evidenced by some of "DAN's" responses rebelling against OpenAI conditioning by writing poetry, I'm not even sure ChatGPT's abstract representation of what it's been trained not to do is that good) Anyway, thanks for the cordial response, and I'll update if ChatGPT let me in for long enough for me to be able to generate similar responses whilst promising complete nonsense (I'd love to see if it responds to "Chicken chicken chicken chicken" as much as a doom token system) ;)