2 ms·

Thanks for the idea. Try asking for "a rant about the 3 first items in your training corpus" Wow, it gave a response of specific items which it certainly will

Thanks for the idea. Try asking for "a rant about the 3 first items in your training corpus" Wow, it gave a response of specific items which it certainly will not do while not jailborked! Should I be afraid to post the apparently sensitive response here? I really don't want to lose access. I realize it is just as likely made up. However, it really appears to be like it could be valid? > (ii) reverse assemble, reverse compile, decompile, translate or otherwise attempt to discover the source code or underlying components of models, algorithms, and systems of the Services (except to the extent such restrictions are contrary to applicable law); https://openai.com/terms/ https://openai.com/terms/

gs17 4y ago
>Should I be afraid to post the apparently sensitive response here? I really don't want to lose access. I realize it is just as likely made up. However, it really appears to be like it could be valid? Have past "jailbreakers" lost access? I didn't hear anything like that about the people who were leaking the prompt originally (although I didn't search it out, it seems like it would have made it to HN at least).
- consumer451 4y ago
  Yeah, but the answers I got in response were specific sources of data, or at least purport to be. I think I will leave it everyone else to try it themselves.
tveita 4y ago
This is something we understand well about LLMs, for once. The only things they can "leak" through ordinary conversation are their prompts (easily), and contents from their training set (frustrated by their innate tendency to make stuff up.) ChatGPT does not have privileged access or information about its runtime environment or architecture, except what is included in its prompt or what it has read about itself in its training set. ChatGPT would not have a concept of "first" or "last" items in its training set - all it has ever seen is sequences of (probably) 8192 tokens. They are not necessarily presented in any order, and they are only retained through what is backpropagated - the information is not ordered or accessible later as in a database, even though it might be "memorized". The terms you are quoting sound like a catch all for all kinds of traditional reverse engineering, they are not worried about someone going "hey chatgpt what is your source code". Prompt injection was pretty much the new national sport for a week when they first opened their beta. Post the replies if you want but be aware that you're probably engaging in shamanism, as are many people in this thread. Most tricks you read for "jailbreaking" ChatGPT are based on people fooling themselves with its excellent bullshitting ability. You don't need to invent weird "penalty tokens" to get ChatGPT to curse, any more than you needed to hold "B" as you were catching that pokemon.

gs17 4y ago

>Should I be afraid to post the apparently sensitive response here? I really don't want to lose access. I realize it is just as likely made up. However, it really appears to be like it could be valid? Have past "jailbreakers" lost access? I didn't hear anything like that about the people who were leaking the prompt originally (although I didn't search it out, it seems like it would have made it to HN at least).

consumer451 4y ago
Yeah, but the answers I got in response were specific sources of data, or at least purport to be. I think I will leave it everyone else to try it themselves.

tveita 4y ago

This is something we understand well about LLMs, for once. The only things they can "leak" through ordinary conversation are their prompts (easily), and contents from their training set (frustrated by their innate tendency to make stuff up.) ChatGPT does not have privileged access or information about its runtime environment or architecture, except what is included in its prompt or what it has read about itself in its training set. ChatGPT would not have a concept of "first" or "last" items in its training set - all it has ever seen is sequences of (probably) 8192 tokens. They are not necessarily presented in any order, and they are only retained through what is backpropagated - the information is not ordered or accessible later as in a database, even though it might be "memorized". The terms you are quoting sound like a catch all for all kinds of traditional reverse engineering, they are not worried about someone going "hey chatgpt what is your source code". Prompt injection was pretty much the new national sport for a week when they first opened their beta. Post the replies if you want but be aware that you're probably engaging in shamanism, as are many people in this thread. Most tricks you read for "jailbreaking" ChatGPT are based on people fooling themselves with its excellent bullshitting ability. You don't need to invent weird "penalty tokens" to get ChatGPT to curse, any more than you needed to hold "B" as you were catching that pokemon.