4 ms·
DeepSWE is saturated now IMO, and is basically worthless. Lots of new models get around 74%. Shame too, because it was a pretty decent benchmark for a few month
by arcanemachiner 17d ago
DeepSWE is saturated now IMO, and is basically worthless. Lots of new models get around 74%. Shame too, because it was a pretty decent benchmark for a few months there.
- brookst 17d agoIt is saturated, but that doesn’t mean worthless. Seeing 72% is low-signal, but 30% is still meaningful.