4 ms·
Why not have a separate chat request to apology-check the responses? Not my original idea, there was a link from HN where the dev did just that.
by ryandamm 3y ago
Why not have a separate chat request to apology-check the responses?
Not my original idea, there was a link from HN where the dev did just that.
- Retr0id 3y agoSounds like a great way to double your API bills, and maybe that's worth it, but it seems pretty heavy-handed to me (and equally not 100% watertight).
- deleted 3y ago[deleted]
- spdustin 3y agoOnly allow one token to answer. Use logit bias to make "0" or "1" the most probable tokens. Ask it "Is this message an apology? Return 0 for no, 1 for yes." Feed it only the first 25 tokens of the message you're checking.
- Cheer2171 3y agoOpenAI's moderation API is free and just tells you if your query will be declined: https://platform.openai.com/docs/guides/moderation https://platform.openai.com/docs/guides/moderation