This is the key to training better and better models. You collect all of the direction prompts and anti-direction prompts, and you finetune the model to behave

This is the key to training better and better models. You collect all of the direction prompts and anti-direction prompts, and you finetune the model to behave that way for the particular domain, under those particular conditions, as a default. This is also why I don't believe for a second that the AI labs aren't training on our data, even when we opt out. This sort of data collection is a requirement for improving the models. It's also why Anthropic purposefully gives you a dumb model or a corner cutting model to goad you into guiding it into the correct answers, so that you do the fine-tuning for them. OpenAI at least, does not do that. Which is why ChatGPT is far less infuriating to work with.