The chain of thought that is apparent in the conversation is what's really interesting and what OpenAI exploits. OpenAI does not have to evaluate the quality be

The chain of thought that is apparent in the conversation is what's really interesting and what OpenAI exploits. OpenAI does not have to evaluate the quality because the conversation itself is already valuable. Whether the conversation was good or bad can easily be inferred from the back-and-forth. That's the key: they just have to take all those conversations and fine-tune their models with them.