4 ms·
Calibration of the response occurs through prompting. That is, if you ask GPT a question directly, it will give the most glib, unfiltered answer possible like
by syntheweave 4y ago
Calibration of the response occurs through prompting.
That is, if you ask GPT a question directly, it will give the most glib, unfiltered answer possible like an average Reddit user. But if you tell it to role-play a persona it will apply elements of that persona, causing it to filter itself. Or if you tell it "describe your reasoning step by step" it will proceed down a path with greater causality. Or if you tell it "here are the rules of a language, give a response to the prompt in this language" it will attempt to produce a response that looks like those rules, using the weighting associated with logical words like "must," "cannot," "if," etc.
Measuring calibration is the problem now: we know some prompts do much better than others, but not how to optimize that in a general sense, to make an LLM always adopt the persona needed for the job.