3 ms·
My understanding is the base model is pretty good about knowing whether it knows stuff or not. it's human feedback training that causes it to lose that signal.
by sharemywin 4y ago
My understanding is the base model is pretty good about knowing whether it knows stuff or not. it's human feedback training that causes it to lose that signal.
- 13years 4y agoDo you have any references? I know of the emergent deception problem that seems to be created through feedback. https://bounded-regret.ghost.io/emergent-deception-optimization https://bounded-regret.ghost.io/emergent-deception-optimizat...
- famouswaffles 4y agoBase GPT-4 was highly calibrated. read open ai's technical paper. also, this paper on gpt-4 performance of medical challenge problems confirmed the high calibration for medicine https://arxiv.org/abs/2303.13375 https://arxiv.org/abs/2303.13375
- 13years 4y agoThanks, but I didn't find any details about performance of pre reinforcement training and after. Looking to understand more about the assertion that hallucinations are introduced by the reinforcement training.
- famouswaffles 4y agohttps://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 The technical report has before and after comparisons. It's a bit worse on some tests. and they pretty explicitly mention the issue of calibration (how well confidence on a problem results in the ability or accuracy solving that problem). Hallucinations are a different matter.