6 ms·

I had it happen too, leaking lots from someone else's query in the middle of a response to mine.

by cma 3y ago

TX81Z 3y ago
I get the impression overall they have massively under invested in web infra, security, and privacy. The AI is one thing, running a performant, secure, and private web platform is its own thing. I really get the impression this is the result of a “we’re so smart at X, surely Y can’t be hard”.
bestcoder69 3y ago
Not leaking! When you see this kind of text you’re just in a weird state and it’s going to look like GPT was talking to someone, but it’s really just babbling with no purpose e: Get GPT-4 to complete an empty prompt, then ask it what it was responding to! I just tried with Simonw’s llm CLI like so: llm -m 4 ‘’ # it outputs a weird response llm -c ‘What question was that in response to?’ In my case an explanation of euthanasia, and my supposed question was “What is euthanasia?”. I did it again and it said there was no original question, so there’s some randomnes.
- RC_ITR 3y ago
  It’s so interesting that people understand that it’s a machine mimicking speech, until it starts mimicking something they want to believe is real.
- astrange 3y ago
  If you're using the chat interface then it's not really an empty prompt though, as there's a "system prompt". https://chat.openai.com/share/5bb37148-6ff6-4e4b-9c2f-0402951149fc https://chat.openai.com/share/5bb37148-6ff6-4e4b-9c2f-040295...
- re5i5tor 3y ago
  I had a similar situation where I was convinced that Code Interpreter had escaped its sandbox but SimonW was able to reel me in, discussed here https://mastodon.social/@Jackivers/110715505542072107 https://mastodon.social/@Jackivers/110715505542072107
AgentME 3y ago
It sees a lot of separate conversations in its training. It seems much simpler to assume that it's incorrectly learned that it should shift the conversation into a new one occasionally to match this tendency of the training data rather than a bug actually leaking and blending user conversations together.
Sharlin 3y ago
If you really think it leaked someone else’s prompts rather than hallucinating them, it just passed a sort of Turing test on your part.

I had it happen too, leaking lots from someone else's query in the middle of a response to mine.

by cma 3y ago

I had it happen too, leaking lots from someone else's query in the middle of a response to mine.

I get the impression overall they have massively under invested in web infra, security, and privacy. The AI is one thing, running a performant, secure, and private web platform is its own thing. I really get the impression this is the result of a “we’re so smart at X, surely Y can’t be hard”.

bestcoder69 3y ago

Not leaking! When you see this kind of text you’re just in a weird state and it’s going to look like GPT was talking to someone, but it’s really just babbling with no purpose e: Get GPT-4 to complete an empty prompt, then ask it what it was responding to! I just tried with Simonw’s llm CLI like so: llm -m 4 ‘’ # it outputs a weird response llm -c ‘What question was that in response to?’ In my case an explanation of euthanasia, and my supposed question was “What is euthanasia?”. I did it again and it said there was no original question, so there’s some randomnes.

RC_ITR 3y ago
It’s so interesting that people understand that it’s a machine mimicking speech, until it starts mimicking something they want to believe is real.
astrange 3y ago
If you're using the chat interface then it's not really an empty prompt though, as there's a "system prompt". https://chat.openai.com/share/5bb37148-6ff6-4e4b-9c2f-0402951149fc https://chat.openai.com/share/5bb37148-6ff6-4e4b-9c2f-040295...
re5i5tor 3y ago
I had a similar situation where I was convinced that Code Interpreter had escaped its sandbox but SimonW was able to reel me in, discussed here https://mastodon.social/@Jackivers/110715505542072107 https://mastodon.social/@Jackivers/110715505542072107

AgentME 3y ago

It sees a lot of separate conversations in its training. It seems much simpler to assume that it's incorrectly learned that it should shift the conversation into a new one occasionally to match this tendency of the training data rather than a bug actually leaking and blending user conversations together.

Sharlin 3y ago

If you really think it leaked someone else’s prompts rather than hallucinating them, it just passed a sort of Turing test on your part.