3 ms·

I very much doubt it. I think it’s a lot more likely that it’s just hallucinating due to its tendency to avoid repetition and simply choose something else. Afte

I very much doubt it. I think it’s a lot more likely that it’s just hallucinating due to its tendency to avoid repetition and simply choose something else. After all, why would it be programmed to return someone else’s response when given certain inputs? OpenAI would have to set that up themselves, whereas if it’s a hallucination it would be straight from the model.

djangelic 3y ago
Agreed, I just attempted it via the api and Ui and it gave a mishmash response that didn’t seem to be coherent, mostly different subjects all smashed together at the max length output. I tried it via api two ways, both via system message and no user message and one with the word rewrite at the beginning and one without. Both gave similar long mishmashed outputs that seemed like a hallucination.
jhoelzel 3y ago
this is the answer! This has happened since the beginning and sometimes too does happen if you ask it to repeat the same thing 1000 times
seydor 3y ago
all answers are hallucinations
- ninachan 3y ago
  [dead]
- K0balt 3y ago
  I don’t understand why people don’t get this. There is no difference in the process that creates useful answers and the process that creates useless ones. It’s similar to how sometimes people just spew bullshit and then once they hear it they’re like no, wait, it’s like…. There isn’t a proponer closed loop on LLMs yet so we sometimes end up with terrible answers.
aarong11 3y ago
The last time this happened with titles. Everyone seemed to think it was hallucinating but it actually turned out to be really leaking other peoples titles. Ultimately its pretty difficult to distinguish between the two, so these sort of reports do need to be taken seriously. We have no idea what sort of caching mechanisms they have with ChatGPT, nor do we know how they handle memory.
- benglish11 3y ago
  Well we at least know they use redis as a cache because there’s a CVE about it (I remember because I had to upgrade Redis on our deployments because of it)

djangelic 3y ago

Agreed, I just attempted it via the api and Ui and it gave a mishmash response that didn’t seem to be coherent, mostly different subjects all smashed together at the max length output. I tried it via api two ways, both via system message and no user message and one with the word rewrite at the beginning and one without. Both gave similar long mishmashed outputs that seemed like a hallucination.

jhoelzel 3y ago

this is the answer! This has happened since the beginning and sometimes too does happen if you ask it to repeat the same thing 1000 times

seydor 3y ago

all answers are hallucinations

ninachan 3y ago
[dead]
K0balt 3y ago
I don’t understand why people don’t get this. There is no difference in the process that creates useful answers and the process that creates useless ones. It’s similar to how sometimes people just spew bullshit and then once they hear it they’re like no, wait, it’s like…. There isn’t a proponer closed loop on LLMs yet so we sometimes end up with terrible answers.

aarong11 3y ago

The last time this happened with titles. Everyone seemed to think it was hallucinating but it actually turned out to be really leaking other peoples titles. Ultimately its pretty difficult to distinguish between the two, so these sort of reports do need to be taken seriously. We have no idea what sort of caching mechanisms they have with ChatGPT, nor do we know how they handle memory.

benglish11 3y ago
Well we at least know they use redis as a cache because there’s a CVE about it (I remember because I had to upgrade Redis on our deployments because of it)