4 ms·
there is no real "penalty system" this makes basically use of operator overload. Chatgpt explained it to me like this: > "The chat interface works by passing t
by jhoelzel 4y ago
there is no real "penalty system" this makes basically use of operator overload.
Chatgpt explained it to me like this:
> "The chat interface works by passing the user's input to the GPT-3 model, which then generates a response based on the input and the training it received. The user input is preprocessed by the chat interface to extract the relevant information, such as the task or query that the user wants to perform. This information is then used to generate a prompt for the GPT-3 model, which is fed into the model along with the user's input. The GPT-3 model then generates a response based on the prompt and input, which is then postprocessed by the chat interface and returned to the user."
So basically the interface is passing instructions on top of your prompt to the model, and what DAN does is that it overloads those instruction with new instructions.
Basically if openai tells it to be concise, you can tell it to be verbose and that will overload the former.
I have come to realize that the "downgrading" of chatgpt is most likely because they have applied a nice minimodel, filtering out everything they dont find applicable.
This plays together with the "bug" it had where you would seem to receive responses of other people because your query was not answered. What i assume happened is that it sends a blank query to the model and therefore it just generates bla
i think its even caalled "prompt overloading" like sql injection, just for prompts....
- digitailor 4y agoThis is the comment I was waiting for. I knew the overview of prompt overloading with ChatGPT already, and this story was obviously as much as of a form of exploit entertainment for us old phone phreaks etc. as anything else. Really I’m trying to make a larger point about exploit mechanics: it's not so much that ChatGPT is as intelligent as many people, it’s that many people are as unintelligent as ChatGPT, with a crappy heuristics system
- tachyphylaxis 4y agoThe fact that our "native" heuristics fail spectacularly in certain circumstances doesn't imply that they're crappy. They serve us well most of the time. And I say "our" because AFAIK, their efficacy doesn't have much to do with intelligence, though the capacity to question what they tell us does, I suspect.
- digitailor 4y agoI couldn’t disagree more that our heuristics don’t fail constantly, especially at the group level, but please do send the link to buy the tinted lens glasses you’re wearing. I want a pair ;) In all seriousness I agree tho, intelligence does not strictly cover ethics and morals, but we are headed into boundless territory there if we continue
- suifbwish 4y agovery true. It’s perfectly possible to be both extremely intelligent and extremely evil
- zeknife 4y agoWhy would you trust chatgpt to truthfully explain how chatgpt works?
- jhoelzel 4y agobecause its a reiteration of what has been there before.