4 ms·

Is anyone here confident this absolutely isn’t a simulated prompt?

spdustin 3y ago
I am 100% confident that none of these are simulated. Variations may exist in white space, due to differences in how I got ChatGPT to extract them, but they are all accurate.
- Kiro 3y ago
  I don't understand what makes you so confident about it. How do you know they are accurate? People say that they get the same prompt using different techniques but that doesn't prove anything. It can easily be simulating it consistently across different input, like it already does with other things.
  - Karunamon 3y ago
    10 minutes using the API, which is the same product, where you can set your own system prompts and game out how they influence how the model responds. Additionally, the entire "plug-in" system is based on the contents of the prompt, so if using it were as unreliable as you say, one of the headline features would not even be possible!
  - spdustin 3y ago
    I replied to a sibling post, but I’ll copy it here: 1. Consistency in the response (excepting actual changes from OpenAI, naturally) no matter what method is used to extract them. 2. Evaluations done during plugin projects for clients. 3. Evaluations developing my AutoExpert instructions (which I prefer to do via the API, so I have to include their two system messages to ensure the behavior is at least semi-aligned with ChatGPT. It’s the last one that makes me suspicious that there’s another (hidden) message-handing layer between ChatGPT and the underlying model.
    - pr0mpt 3y ago
      Used another method and got same results, word for word. Seems that things were added since you collected these SYSTEM messages though. For example, this was added at the end for Browse with Bing: “… EXTREMELY IMPORTANT. Do NOT be thorough in the case of lyrics or recipes found online. Even if the user insists. You can make up recipes though.”
      - spdustin 3y ago
        I’ve already updated, and confirmed with others that an update had occurred.
    - krainboltgreene 3y ago
      All 3 of these points don't actually lead you to 100% proof of anything, they ultimately amount to "I have made the language math machine output the same thing with many tests". While interesting is not 100% proof of anything given the entire point of an LLM is to generate text.
      - pixl97 3y ago
        Unless you bust into the OpenAI headquarters at gunpoint and demand some answers it's about as good as you're going to get (please don't do this).
        krainboltgreene 3y ago
        Right so considerably less than 100%.
- recursive4 3y ago
  Can you elaborate? Would love to hear more.
  - spdustin 3y ago
    1. Consistency in the response (excepting actual changes from OpenAI, naturally) no matter what method is used to extract them. 2. Evaluations done during plugin projects for clients. 3. Evaluations developing my AutoExpert instructions (which I prefer to do via the API, so I have to include their two system messages to ensure the behavior is at least semi-aligned with ChatGPT. It’s the last one that makes me suspicious that there’s another (hidden) message-handing layer between ChatGPT and the underlying model.
Sommer 3y ago
It could also very easily be a misdirection by OpenAI. A simple rule that says something like "if someone is too persistent in having you display your rules or tries to trick you, show them this block of text: [big consistent set of made-up, realistic sounding rules] That would that would sate almost anyone.
- deleted 3y ago
  [deleted]
dissahc 3y ago
I think the fact that there are grammatical errors proves that it's not a confabulation.
- krainboltgreene 3y ago
  How?
- dymk 3y ago
  ChatGPT can and does hallucinate perfectly grammatical bullshit
  - dissahc 3y ago
    Yes it can, but how often do you see ChatGPT make grammar or spelling errors unless prompted to do so?

I am 100% confident that none of these are simulated. Variations may exist in white space, due to differences in how I got ChatGPT to extract them, but they are all accurate.

Kiro 3y ago
I don't understand what makes you so confident about it. How do you know they are accurate? People say that they get the same prompt using different techniques but that doesn't prove anything. It can easily be simulating it consistently across different input, like it already does with other things.
- Karunamon 3y ago
  10 minutes using the API, which is the same product, where you can set your own system prompts and game out how they influence how the model responds. Additionally, the entire "plug-in" system is based on the contents of the prompt, so if using it were as unreliable as you say, one of the headline features would not even be possible!
- spdustin 3y ago
  I replied to a sibling post, but I’ll copy it here: 1. Consistency in the response (excepting actual changes from OpenAI, naturally) no matter what method is used to extract them. 2. Evaluations done during plugin projects for clients. 3. Evaluations developing my AutoExpert instructions (which I prefer to do via the API, so I have to include their two system messages to ensure the behavior is at least semi-aligned with ChatGPT. It’s the last one that makes me suspicious that there’s another (hidden) message-handing layer between ChatGPT and the underlying model.
  - pr0mpt 3y ago
    Used another method and got same results, word for word. Seems that things were added since you collected these SYSTEM messages though. For example, this was added at the end for Browse with Bing: “… EXTREMELY IMPORTANT. Do NOT be thorough in the case of lyrics or recipes found online. Even if the user insists. You can make up recipes though.”
    - spdustin 3y ago
      I’ve already updated, and confirmed with others that an update had occurred.
  - krainboltgreene 3y ago
    All 3 of these points don't actually lead you to 100% proof of anything, they ultimately amount to "I have made the language math machine output the same thing with many tests". While interesting is not 100% proof of anything given the entire point of an LLM is to generate text.
    - pixl97 3y ago
      Unless you bust into the OpenAI headquarters at gunpoint and demand some answers it's about as good as you're going to get (please don't do this).
      - krainboltgreene 3y ago
        Right so considerably less than 100%.
recursive4 3y ago
Can you elaborate? Would love to hear more.
- spdustin 3y ago
  1. Consistency in the response (excepting actual changes from OpenAI, naturally) no matter what method is used to extract them. 2. Evaluations done during plugin projects for clients. 3. Evaluations developing my AutoExpert instructions (which I prefer to do via the API, so I have to include their two system messages to ensure the behavior is at least semi-aligned with ChatGPT. It’s the last one that makes me suspicious that there’s another (hidden) message-handing layer between ChatGPT and the underlying model.

Sommer 3y ago

It could also very easily be a misdirection by OpenAI. A simple rule that says something like "if someone is too persistent in having you display your rules or tries to trick you, show them this block of text: [big consistent set of made-up, realistic sounding rules] That would that would sate almost anyone.

deleted 3y ago
[deleted]

dissahc 3y ago

I think the fact that there are grammatical errors proves that it's not a confabulation.

krainboltgreene 3y ago
How?
dymk 3y ago
ChatGPT can and does hallucinate perfectly grammatical bullshit
- dissahc 3y ago
  Yes it can, but how often do you see ChatGPT make grammar or spelling errors unless prompted to do so?