I'm a bit skeptical of this until it's proven that they're getting the right answers in the right ways. It could be that base models are just more random and wh

I'm a bit skeptical of this until it's proven that they're getting the right answers in the right ways. It could be that base models are just more random and when given 200 guesses out of 1000 possible answers tend to distribute them more evenly, bringing up the pass@k number.