How do the makers create the limits on what LLMs cooperate with? For ChatGPT I heard speculation on there being a second neural network that assesses the suitab

How do the makers create the limits on what LLMs cooperate with? For ChatGPT I heard speculation on there being a second neural network that assesses the suitability of both the prompt and the response, which one could then work around in various ways, but with this seemingly being downloadable rather than something they host themselves, I doubt that's the case here. Do they feed it a lot of bad prompts (in the eyes of the makers) during training and tweak/backpropagate the network until it always rejects those?