I guess it's not that straightforward. It's probably a combination of much less prevalent use of "don't know" online, low scores of "don't know" in RLHF, system

I guess it's not that straightforward. It's probably a combination of much less prevalent use of "don't know" online, low scores of "don't know" in RLHF, system prompt instructing GPT to give helpful responses, and, yeah, maybe token sampling algorithm is tuned to disfavor explicitly uncertain responses.