First time I have heard the term (and I have done quite a bit of llm benchmarking) but I love it! Similar to the xkcd "wtfs per minute" measure of code quality

First time I have heard the term (and I have done quite a bit of llm benchmarking) but I love it! Similar to the xkcd "wtfs per minute" measure of code quality measure I guess. Everyone I know who plays with LLMs says things like - it didn't answer my special secret benchmark question better than previous models - it surprised me by being able to give a satisfying answer for x that I wasn't expecting - it always seems to give answers that are [Some negative quality] Obviously these things are super subjective but still important and useful in a way that "answers 5th grade maths questions more accurately than previous models" sometimes isn't