3 ms·
Not prompt engineer, but can't you ask it to answer yes or no to "Are these two comments vaguely about the same thing"? I'm not sure where the confabulation can
by gruez 3y ago
Not prompt engineer, but can't you ask it to answer yes or no to "Are these two comments vaguely about the same thing"? I'm not sure where the confabulation can happen, aside from misclassification which can be solved by getting a human and/or more LLMs to review the decision.
- greenbit 3y agoInteresting, did not know these things could be used to make assessments, judgments, categorizations, etc. Had been under the impression they just riff out text based on suggestions.
- gruez 3y ago>Interesting, did not know these things could be used to make assessments, judgments, categorizations, etc. Yep. For instance in this paper[1] they asked an LLM a bunch of political questions to figure out what its political alignment is. [1] "Discovering Language Model Behaviors with Model-Written Evaluations" https://www-cdn.anthropic.com/e4f69aacd8c0905030172bc6eb480c252ea7d6ad/model-written-evals.pdf https://www-cdn.anthropic.com/e4f69aacd8c0905030172bc6eb480c...