3 ms·

i don't. just thought someone else might be interested in

by mkaramuk 5mo ago

jubilanti 5mo ago
But you have zero evidence this is actually the real system prompt.
- mkaramuk 5mo ago
  yeah, just assumed it is.
  - kibwen 5mo ago
    The chatbot craze in microcosm.
    - rgoulter 5mo ago
      I think it's worth elaborating. Loosely, LLMs give plausible responses. And LLMs are really good at writing confident-sounding responses. LLM output is as if someone is replying with the sole purpose of appearing helpful and knowledgeable. I wouldn't trust opinions on LLMs from people who are entirely positive or entirely negative: the technology is just too mixed for that. I'd say it's useful for someone to have had a bad experience with LLMs (e.g. LLMs being confidently wrong), as well as making use of LLMs for things they're powerful at. (e.g. "small" programming tasks).
- simonw 5mo ago
  LLMs are really good at repeating text that they've just seen. Very occasionally they'll mix up a word or two, but it's not at all challenging for them to regurgitate text from a previous section of input. I have yet to see a documented example of a system prompt leak that was NOT the real system prompt. Have you seen one?
  - sspiff 5mo ago
    > I have yet to see a documented example of a system prompt leak that was NOT the real system prompt. Have you seen one? I have yet to see a convincing proof that there is a teapot in an orbit around the sun between earth and mars. The people who can debunk these system prompts claims have zero incentive to do so. The fact that a lot of these "leaks" happen yet non of the prompts are the same or even seem to be different iterations of the same family of system prompts, should tell you all you need to know.
    - simonw 5mo ago
      Show me same examples of a leaked system prompt that came out different ever time and I'll concede that system prompt leaks are likely junk. My experience has been the opposite: the way you confirm a system prompts leaks leak is to try it several times and check that you get the same result, and I've seen that done many times over the past few years.

i don't. just thought someone else might be interested in

jubilanti 5mo ago

But you have zero evidence this is actually the real system prompt.

mkaramuk 5mo ago
yeah, just assumed it is.
- kibwen 5mo ago
  The chatbot craze in microcosm.
  - rgoulter 5mo ago
    I think it's worth elaborating. Loosely, LLMs give plausible responses. And LLMs are really good at writing confident-sounding responses. LLM output is as if someone is replying with the sole purpose of appearing helpful and knowledgeable. I wouldn't trust opinions on LLMs from people who are entirely positive or entirely negative: the technology is just too mixed for that. I'd say it's useful for someone to have had a bad experience with LLMs (e.g. LLMs being confidently wrong), as well as making use of LLMs for things they're powerful at. (e.g. "small" programming tasks).
simonw 5mo ago
LLMs are really good at repeating text that they've just seen. Very occasionally they'll mix up a word or two, but it's not at all challenging for them to regurgitate text from a previous section of input. I have yet to see a documented example of a system prompt leak that was NOT the real system prompt. Have you seen one?
- sspiff 5mo ago
  > I have yet to see a documented example of a system prompt leak that was NOT the real system prompt. Have you seen one? I have yet to see a convincing proof that there is a teapot in an orbit around the sun between earth and mars. The people who can debunk these system prompts claims have zero incentive to do so. The fact that a lot of these "leaks" happen yet non of the prompts are the same or even seem to be different iterations of the same family of system prompts, should tell you all you need to know.
  - simonw 5mo ago
    Show me same examples of a leaked system prompt that came out different ever time and I'll concede that system prompt leaks are likely junk. My experience has been the opposite: the way you confirm a system prompts leaks leak is to try it several times and check that you get the same result, and I've seen that done many times over the past few years.