3 ms·
It is very unfortunate they upweighted SciCode from 8% to 10%. SciCode is a broken benchmark: see https://arxiv.org/abs/2608.04975 https://arxiv.org/abs/2608.04
by sanxiyn 3mo ago
It is very unfortunate they upweighted SciCode from 8% to 10%. SciCode is a broken benchmark: see https://arxiv.org/abs/2608.04975 https://arxiv.org/abs/2608.04975.