> Metrics may not be perfect, but they are all we have. You could train a model, send it to a bunch of human crowdworkers, and ask them to rate it on how good i

> Metrics may not be perfect, but they are all we have. You could train a model, send it to a bunch of human crowdworkers, and ask them to rate it on how good it is (Anthropic does this!), but the result of that is... another metric. A metric that is an aggregate of human intuition is not the same as the usual metrics though. It's just a semi-formalized way to capture those intuitive observations, rather than trying to replace them with much-simpler piecemeal mechanical evaluations. > but I think it's bad science/engineering to rely on some indescribable quality of "goodness" when it comes to developing new models. IMO, bad science is what is currently happening across the entire field. Astonishing high-level behavior is being observed from models, but the tools to analyze it don't exist, so instead, people are pushing out papers at a record pace that analyze every low-level property imaginable, as if such analysis would eventually yield high-level insights. It's okay to not know. I wish every paper dealing with LLMs would start and end with the sentence "Overall, we have no idea what is happening." Instead, we get papers that add a few numbers and then wax philosophical about how LLMs supposedly do things (I'm exaggerating here, but the gist is accurate). Not a day goes by without a new article claiming that LLMs have reached their limit or similar, while we have no clue how they even work! This is really bad science, and yes, I know that much of it is coming from non-experts, but I've seen lots and lots of experts contribute to this nonsense by making completely unjustified claims of a similar nature.