2 ms·

Yes, they benefit from people not knowing what ChatGPT can do and being curious enough to try it out. No, they don’t benefit from giving users bad answers, and

Yes, they benefit from people not knowing what ChatGPT can do and being curious enough to try it out. No, they don’t benefit from giving users bad answers, and it’s already the case that ChatGPT will sometimes refuse with an error message or warn when it can’t do something. There’s no reason in principle they can’t do more RLHF training to report errors more often.

kenjackson 3y ago
> There’s no reason in principle they can’t do more RLHF training to report errors more often. I think it's just time and cost -- as I think they're doing a lot of this already. And the fact that the input/output schema is pretty much any text that fits within the context of a window size. So you can train on this set of prompts, but there can be a whole new class of prompts that you didn't think of before that are now problematic. It's like the self-driving car on a closed track, versus driving in real life when there's a huge pothole blocking your freeway exit while in a construction zone and a school bus for some pulls over to drop off kids to help clean liter.