I mean detecting "bad" prompts (a prefilter) or responses (a postfilter) and penalizing users for submitting the kinds of queries that generate bad responses. T

I mean detecting "bad" prompts (a prefilter) or responses (a postfilter) and penalizing users for submitting the kinds of queries that generate bad responses. This would be outside the llm itself. This could be quite unreliable and make many users unhappy, hence it would be a drastic step.