While this comment is a good start, we should remember that for some scores, SOTA is only loosely correlated with improvements in downstream performance. This i

While this comment is a good start, we should remember that for some scores, SOTA is only loosely correlated with improvements in downstream performance. This is true in things like summarization with ROUGE scores (which suck and everyone hates them)