> There’s no reason in principle they can’t do more RLHF training to report errors more often. I think it's just time and cost -- as I think they're doing a lo

> There’s no reason in principle they can’t do more RLHF training to report errors more often. I think it's just time and cost -- as I think they're doing a lot of this already. And the fact that the input/output schema is pretty much any text that fits within the context of a window size. So you can train on this set of prompts, but there can be a whole new class of prompts that you didn't think of before that are now problematic. It's like the self-driving car on a closed track, versus driving in real life when there's a huge pothole blocking your freeway exit while in a construction zone and a school bus for some pulls over to drop off kids to help clean liter.