4 ms·

> Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? "Plausible" means "that which the maj

> Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? "Plausible" means "that which the majority of people is likely to say". So, yes, a foundational model is likely to say the plausible thing. On the other hand, it has to have a way to output a truthful answer too, to not fail on texts produced by experts. So, it's not impossible that the model could be trained to prefer to output truthful answers (as well as it can do it, it's not an AGI with perfect factual memory and logical inference after all).

hgsgm 4y ago
> "that which the majority of people is likely to say" .. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.
- red75prime 4y ago
  I guess it's not that straightforward. It's probably a combination of much less prevalent use of "don't know" online, low scores of "don't know" in RLHF, system prompt instructing GPT to give helpful responses, and, yeah, maybe token sampling algorithm is tuned to disfavor explicitly uncertain responses.
- olalonde 4y ago
  On the subject of not knowing thigs... Your clain is incorrect. Prompt: Tell me what you know about the Portland metro bombing terrorist attack of 2005. GPT4: I'm sorry, but I cannot provide information on the Portland metro bombing terrorist attack of 2005, as there is no historical record of such an event occurring. It's possible you may have confused this with another event or have incorrect information. Please provide more context or clarify the event you are referring to, and I'll be happy to help with any information I can.
  - shjake 4y ago
    There is a big difference between what they are talking about (I believe) and asking it factual questions withal straightforward answers. Try discussing something a bit more complex and it’s very easy to get ChatGPT to start contradicting itself all the time (which is more or less the same thing). From my experience ChatGPT is prone to come up with lists of “probable” answer when it’s not feeling very confident. Then it just starts throwing them out in a confident tone. When challenged it switches to a complete different answer and then “pretends” that I misunderstood it previously.
    - olalonde 4y ago
      Yes I know it does, just not all of the time and it wasn't programmed to never say "I don't know".

hgsgm 4y ago

> "that which the majority of people is likely to say" .. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.

red75prime 4y ago
I guess it's not that straightforward. It's probably a combination of much less prevalent use of "don't know" online, low scores of "don't know" in RLHF, system prompt instructing GPT to give helpful responses, and, yeah, maybe token sampling algorithm is tuned to disfavor explicitly uncertain responses.
olalonde 4y ago
On the subject of not knowing thigs... Your clain is incorrect. Prompt: Tell me what you know about the Portland metro bombing terrorist attack of 2005. GPT4: I'm sorry, but I cannot provide information on the Portland metro bombing terrorist attack of 2005, as there is no historical record of such an event occurring. It's possible you may have confused this with another event or have incorrect information. Please provide more context or clarify the event you are referring to, and I'll be happy to help with any information I can.
- shjake 4y ago
  There is a big difference between what they are talking about (I believe) and asking it factual questions withal straightforward answers. Try discussing something a bit more complex and it’s very easy to get ChatGPT to start contradicting itself all the time (which is more or less the same thing). From my experience ChatGPT is prone to come up with lists of “probable” answer when it’s not feeling very confident. Then it just starts throwing them out in a confident tone. When challenged it switches to a complete different answer and then “pretends” that I misunderstood it previously.
  - olalonde 4y ago
    Yes I know it does, just not all of the time and it wasn't programmed to never say "I don't know".