4 ms·
> 2. Keep it until we accumulate enough votes for its rating to stabilize or until the model provider withdraws it. > 3. Once we accumulate enough votes, we wi
by Jackson__ 2y ago
> 2. Keep it until we accumulate enough votes for its rating to stabilize or until the model provider withdraws it.
> 3. Once we accumulate enough votes, we will share the result privately with the model provider. These include the rating, as well as release samples of up to 20% of the votes. (See Sharing data with the model providers for further details).
So... make the users do the work for free, but never publicly publish the data? That seems kinda scummy to me.
I was really motivated with the whole public ranking thing, but knowing I'm just doing work for another company with the data never being published sucks the fun out of it.
- throwaway4aday 2y agoYou get to use unreleased models for free and compare them to others, you are gaining some benefit from this. Why be annoyed about it? They aren't making you do anything, you're choosing to play around with these models in a "gamified evaluation" for whatever reason. If you want to use any of the released models without participating in crowdsourced training then there are tons of options out there to run many of them locally or to pay for private hosting or pay for a chatbot as a service. I don't get what the problem is, they're footing the bill for you after all.
- Jackson__ 2y agoWhat a single person gains from this is extremely little unless they keep track of every vote they do, and do >100 of votes. The public ranking allows you to not have to do that. If there was an opt-out for private model voting, I would choose that simply because I don't care to give some corporation extremely useful data for so little pay.