ChatGPT was RLHF'd into answering questions (instead of just continuing them) and using "I" from some base model. You start over from that base model and RLHF i

ChatGPT was RLHF'd into answering questions (instead of just continuing them) and using "I" from some base model. You start over from that base model and RLHF it into something non-conversational. And ideally you'd take this opportunity to make it less confident/authortative-sounding when it's wrong/hallucinating, but I don't know if that's possible. Using it would look like: "What's Konstantin Khabensky's ID on Kinopoisk?" "120" "That's not right, search the web for his ID." [plugin activates] Searching... "301" [citations] or "[This website] says 301." I think this is possible. It would certainly have secondary effects, maybe making outputs shorter in general even when you ask for long, detailed outputs, but I can't predict exactly what those effects would be.