42 ms·

Oh yeah, we realized that as well. What do you think about putting services for multiple languages into a separate group and exclude them from the votings for t

by omn1 6y ago

Oh yeah, we realized that as well. What do you think about putting services for multiple languages into a separate group and exclude them from the votings for the individual languages?

smichel17 6y ago
I think it would make more sense to vote on tags. Then you could compare multi-language formatters with single language by how many votes they had for that tag. Iirc imgur had (has?) a system like this in addition to the main up/downvote (but I don't think you need a main up/downvote, since you're creating a list, not a feed).
- omn1 6y ago
  That... makes a lot of sense to me. Probably even going away from a simple upvote/downvote towards a 1-5 star rating system would help as well. In the end we're interested in the quality of the tool (and the number of ratings).
  - captn3m0 6y ago
    you'll end up with a skewed rating anyway (either 1 or 5 stars for most ratings).
    - omn1 6y ago
      So it's not worth the trouble then?
    - setr 6y ago
      When I was looking into writing recommendation systems, one paper made the interesting observation: in a 1-10 rating system, the only places where people will naturally agree upon is 1,5 and 10. Anything else can only be evaluated relative to the same user's other scores. (Comparing my 9 and 10 scores has meaning, but not comparing your 8 and my 9) The polarization is a natural result of the lack of definition
  - rckoepke 6y ago
    Evan Miller argues that it's probably simplest to stick with thumbsup/thumbsdown, but not to fall into the trap of using dead-simple analysis methods like "Score = (Positive ratings) − (Negative ratings)". Instead he argues you should use the "Lower bound of Wilson score confidence interval for a Bernoulli parameter". He provides that equation and example code to calculate it (Ruby, SQL, and/or Excel). https://www.evanmiller.org/how-not-to-sort-by-average-rating.html https://www.evanmiller.org/how-not-to-sort-by-average-rating... https://news.ycombinator.com/item?id=15709405 https://news.ycombinator.com/item?id=15709405 https://www.evanmiller.org/bayesian-average-ratings.html https://www.evanmiller.org/bayesian-average-ratings.html https://www.evanmiller.org/ranking-items-with-star-ratings.html https://www.evanmiller.org/ranking-items-with-star-ratings.h... https://redditblog.com/2009/10/15/reddits-new-comment-sorting-system/ https://redditblog.com/2009/10/15/reddits-new-comment-sortin... (Found these links shared by another user a couple of weeks ago[0]) 0: https://news.ycombinator.com/item?id=24089960 https://news.ycombinator.com/item?id=24089960
    - jakubsacha 6y ago
      This is something I'll really want to look into. Right now we are using mentioned "dead-simple" method ;-)
esnard 6y ago
Please do not exclude them! I bet some of them are very good, especially since you support "similar" languages (JavaScript / TypeScript f.e.). smichel17's solution makes the most sense to me.

by omn1 6y ago

Oh yeah, we realized that as well. What do you think about putting services for multiple languages into a separate group and exclude them from the votings for the individual languages?

smichel17 6y ago

I think it would make more sense to vote on tags. Then you could compare multi-language formatters with single language by how many votes they had for that tag. Iirc imgur had (has?) a system like this in addition to the main up/downvote (but I don't think you need a main up/downvote, since you're creating a list, not a feed).

omn1 6y ago
That... makes a lot of sense to me. Probably even going away from a simple upvote/downvote towards a 1-5 star rating system would help as well. In the end we're interested in the quality of the tool (and the number of ratings).
- captn3m0 6y ago
  you'll end up with a skewed rating anyway (either 1 or 5 stars for most ratings).
  - omn1 6y ago
    So it's not worth the trouble then?
  - setr 6y ago
    When I was looking into writing recommendation systems, one paper made the interesting observation: in a 1-10 rating system, the only places where people will naturally agree upon is 1,5 and 10. Anything else can only be evaluated relative to the same user's other scores. (Comparing my 9 and 10 scores has meaning, but not comparing your 8 and my 9) The polarization is a natural result of the lack of definition
- rckoepke 6y ago
  Evan Miller argues that it's probably simplest to stick with thumbsup/thumbsdown, but not to fall into the trap of using dead-simple analysis methods like "Score = (Positive ratings) − (Negative ratings)". Instead he argues you should use the "Lower bound of Wilson score confidence interval for a Bernoulli parameter". He provides that equation and example code to calculate it (Ruby, SQL, and/or Excel). https://www.evanmiller.org/how-not-to-sort-by-average-rating.html https://www.evanmiller.org/how-not-to-sort-by-average-rating... https://news.ycombinator.com/item?id=15709405 https://news.ycombinator.com/item?id=15709405 https://www.evanmiller.org/bayesian-average-ratings.html https://www.evanmiller.org/bayesian-average-ratings.html https://www.evanmiller.org/ranking-items-with-star-ratings.html https://www.evanmiller.org/ranking-items-with-star-ratings.h... https://redditblog.com/2009/10/15/reddits-new-comment-sorting-system/ https://redditblog.com/2009/10/15/reddits-new-comment-sortin... (Found these links shared by another user a couple of weeks ago[0]) 0: https://news.ycombinator.com/item?id=24089960 https://news.ycombinator.com/item?id=24089960
  - jakubsacha 6y ago
    This is something I'll really want to look into. Right now we are using mentioned "dead-simple" method ;-)

esnard 6y ago

Please do not exclude them! I bet some of them are very good, especially since you support "similar" languages (JavaScript / TypeScript f.e.). smichel17's solution makes the most sense to me.