3 ms·
I like the term prompt performance; I am definitely going to use it: > prompt performance (n.) > the behaviour of a language model in which it conspicuously s
by sheepscreek 1y ago
I like the term prompt performance; I am definitely going to use it:
> prompt performance (n.)
> the behaviour of a language model in which it conspicuously showcases or exaggerates how well it is following a given instruction or persona, drawing attention to its own effort rather than simply producing the requested output.
:)
- jjcob 1y agoMight be a result of using LLMs to evaluate the output of other LLMs. LLMs probably get higher scores if they explicitly state that they are following instructions...
- _23sd 11mo agoIt's like writing an essay for a standardized test, as opposed to one for a college course or for a general audience. When taking a test, you only care about the evaluation of a single grader hurrying to get through a pile of essays, so you should usually attempt to structure your essay to match the format of the scoring rubric. Doing this on an essay for a general audience would make it boring, and doing it in your college course might annoy your professor. Hopefully instruction-following evaluations don't look too much like test grading, but this kind of behavior would make some sense if they do.
- siva7 1y agoThat's the equivalent of a performative male, so better call it performative model behaviour.