8 ms·
Why do you take it personally? Just because you are genuinely interested doesn't mean there are army of people who is sitting their with the sole purpose of gam
by zooq_ai 3y ago
Why do you take it personally? Just because you are genuinely interested doesn't mean there are army of people who is sitting their with the sole purpose of gaming leaderboards.
If you think that doesn't happen, I have a bridge to sell you
- refulgentis 3y agoI'm not taking it personally: frankly, I'm not sure what that would mean in this context. You asked me to try it, I explained that I did, and provided context. Thank you for clarifying: to confirm, yes, I do understand your claim is a vast cabal of OSS zealots rigs LMSys's blind A/B testing. I don't think there's anything more of value I can contribute to a discussion on that topic. Have a good weekend!
- zooq_ai 3y agoI'm a big fan of Elon/Tesla, but I also know Elon/Tesla (a vast majority of them) is a cult. See? it's easy to have those seemingly 'conflicting' thoughts
- jantissler 3y agoThen explain to us how they game it? The test is blind. You don't know which models give the answer. If you trick them into disclosing their name, it is discarded right away and doesn't count towards the score. Genuine question, because as far as I know there is no feasible way to game this score. But if there is, I want to know.
- deleted 3y ago[deleted]
- extraduder_ire 3y ago> If you trick them into disclosing their name, it is discarded right away and doesn't count towards the score. However that process works would be the thing to circumvent, like by getting it to disclose some letters of its name, or any value that is different between models but the same or similar for each one. I assume it doesn't provide the numerical token values in the output or it would be trivial.