4 ms·

Ha! Thanks for this. I knew that this must be a solved problem but wasn't searching for the right thing. I'll take a look at implementing a better default sort

Ha! Thanks for this. I knew that this must be a solved problem but wasn't searching for the right thing. I'll take a look at implementing a better default sort order.

MayeulC 5y ago
For another similar rating system, there's also this one: https://steamdb.info/blog/steamdb-rating/ https://steamdb.info/blog/steamdb-rating/ Not exactly sure how the two compare.
- driscoll42 5y ago
  He compares it at the bottom of the article: Compared to Wilson's formula, it's very short, and it's not nearly as difficult to understand the idea behind how it works. One could easily get the question "Why would this random formula give better results than what a very established mathematician came up with?" I can't really answer that, but it would seem a lot of you agree that it does indeed produce better results when rating Steam's games. I can, however, try to give some insight into this. For one, Wilson's formula isn't really meant to be used quite like this. It takes a rating and the sample size (the number of reviews), and outputs a confidence interval. And a confidence interval basically says that “We are some% sure that the score is between x and y”. If you increase the % of how sure you are, the distance between x and y also increases, and vice versa. But to get a single rating, it's not quite okay to just take the lower bound of that interval. Secondly, because of what was mentioned in the last paragraph, it always gives us a lower rating than the original. This is clearly the incorrect behaviour, as something that just came out and gets a single negative review will be marked as having a score of 0%. Meanwhile, an established terrible game can have 10 positive and 500 negative reviews, and it will rank higher. This is also the reason why one of the two rules I listed was that all ratings should be biased towards the average. Finally, while Wilson's formula probably gives us a more “precise” rating, so to say, it's not necessarily what we want to see. There's a lot of mathematics behind why what it does is correct, while the previously mentioned numbers of 2 and 10 that I picked for my formula were rather arbitrary. Still, I selected them so that the result would also account for the high number of reviews when assigning a good score. It's why you'll probably notice a lot less games with a low review count among the top games than before. I think that's important because a game that is very popular and very highly rated should be ranked higher than a game that isn't as popular and is also very highly rated. Not because we can be more certain that this rating is indeed correct, but because you, as a random person who has yet to try that game, will more probably like it if a lot of other people have liked it as well — if it's not a niche game. And I think this aspect is definitely important and should be accounted for when trying to represent an entire game with just a single number.
- bitsoda 5y ago
  Fascinating. Thanks for sharing this. I could grok this one much more easily than the other algorithm.
hinkley 5y ago
You need multiple rating systems for recipes and the lack of that model is why recipe sites are bullshit, and will continue to be bullshit. A recipe that tastes bad should never be shown to anyone. But if there are two flavorful recipes and one of them has better instructions, then that's the one that should be sorted to the top. My go-to analogy for people being bad at documentation is to compare them to the variation in quality in cooking recipes. That's both a commentary on developer docs and on chef's docs. I have been reworking a highly rated pie recipe. They have not covered browning the crust. They have made no mention of order and grouping of ingredients. If you follow the recipe literally, you're going to end up with a fluffy mix that won't fit in the pie tin. You're also going to get lumpy spices. That has a rustic appeal, but as the picture does not have lumps, the author is just bad at documentation. In my version, I split the spices so that you get a little texture but most are homogenized. I'm still experimenting with number of eggs. I've had 2 eggs (as in the recipe) taste eggy, and 3 eggs (minus a little egg white for the crust) taste fine, depending on how good the emulsion is, and that depends on order. Most recipes still seem to be built on a model of fear and social barter. Fear that if Aunt Susie figures out Grandma Beth's brownie recipe, then nobody will 'have' to invite Grandma Beth to holidays or at least be excited to see her. If you want brownies then you need Grandma Beth.
- sumtechguy 5y ago
  You are so correct on the order and timing of items. Everyone in my family was trying to make my grandmothers cookies. I was the only one who went over and documented exactly how she made them. Now that she is gone. I have been designated as the only one who 'knows how to make them'. Even though I proclaim loud and clearly and have documented 'you must make them exactly like this or they will not taste the same'. They all seem to want to take short cuts. There are no shortcuts with this one and it will take 2 days. I even gladly show them exactly how to do it. Yet they still fail, because they want to skip steps. If you do not follow along exactly you are doomed. Now I want cookies...
  - bluGill 5y ago
    And now I want the recipe. Two days ago please, I want the cookies now.

Ha! Thanks for this. I knew that this must be a solved problem but wasn't searching for the right thing. I'll take a look at implementing a better default sort order.

MayeulC 5y ago

For another similar rating system, there's also this one: https://steamdb.info/blog/steamdb-rating/ https://steamdb.info/blog/steamdb-rating/ Not exactly sure how the two compare.

driscoll42 5y ago
He compares it at the bottom of the article: Compared to Wilson's formula, it's very short, and it's not nearly as difficult to understand the idea behind how it works. One could easily get the question "Why would this random formula give better results than what a very established mathematician came up with?" I can't really answer that, but it would seem a lot of you agree that it does indeed produce better results when rating Steam's games. I can, however, try to give some insight into this. For one, Wilson's formula isn't really meant to be used quite like this. It takes a rating and the sample size (the number of reviews), and outputs a confidence interval. And a confidence interval basically says that “We are some% sure that the score is between x and y”. If you increase the % of how sure you are, the distance between x and y also increases, and vice versa. But to get a single rating, it's not quite okay to just take the lower bound of that interval. Secondly, because of what was mentioned in the last paragraph, it always gives us a lower rating than the original. This is clearly the incorrect behaviour, as something that just came out and gets a single negative review will be marked as having a score of 0%. Meanwhile, an established terrible game can have 10 positive and 500 negative reviews, and it will rank higher. This is also the reason why one of the two rules I listed was that all ratings should be biased towards the average. Finally, while Wilson's formula probably gives us a more “precise” rating, so to say, it's not necessarily what we want to see. There's a lot of mathematics behind why what it does is correct, while the previously mentioned numbers of 2 and 10 that I picked for my formula were rather arbitrary. Still, I selected them so that the result would also account for the high number of reviews when assigning a good score. It's why you'll probably notice a lot less games with a low review count among the top games than before. I think that's important because a game that is very popular and very highly rated should be ranked higher than a game that isn't as popular and is also very highly rated. Not because we can be more certain that this rating is indeed correct, but because you, as a random person who has yet to try that game, will more probably like it if a lot of other people have liked it as well — if it's not a niche game. And I think this aspect is definitely important and should be accounted for when trying to represent an entire game with just a single number.
bitsoda 5y ago
Fascinating. Thanks for sharing this. I could grok this one much more easily than the other algorithm.

hinkley 5y ago

You need multiple rating systems for recipes and the lack of that model is why recipe sites are bullshit, and will continue to be bullshit. A recipe that tastes bad should never be shown to anyone. But if there are two flavorful recipes and one of them has better instructions, then that's the one that should be sorted to the top. My go-to analogy for people being bad at documentation is to compare them to the variation in quality in cooking recipes. That's both a commentary on developer docs and on chef's docs. I have been reworking a highly rated pie recipe. They have not covered browning the crust. They have made no mention of order and grouping of ingredients. If you follow the recipe literally, you're going to end up with a fluffy mix that won't fit in the pie tin. You're also going to get lumpy spices. That has a rustic appeal, but as the picture does not have lumps, the author is just bad at documentation. In my version, I split the spices so that you get a little texture but most are homogenized. I'm still experimenting with number of eggs. I've had 2 eggs (as in the recipe) taste eggy, and 3 eggs (minus a little egg white for the crust) taste fine, depending on how good the emulsion is, and that depends on order. Most recipes still seem to be built on a model of fear and social barter. Fear that if Aunt Susie figures out Grandma Beth's brownie recipe, then nobody will 'have' to invite Grandma Beth to holidays or at least be excited to see her. If you want brownies then you need Grandma Beth.

sumtechguy 5y ago
You are so correct on the order and timing of items. Everyone in my family was trying to make my grandmothers cookies. I was the only one who went over and documented exactly how she made them. Now that she is gone. I have been designated as the only one who 'knows how to make them'. Even though I proclaim loud and clearly and have documented 'you must make them exactly like this or they will not taste the same'. They all seem to want to take short cuts. There are no shortcuts with this one and it will take 2 days. I even gladly show them exactly how to do it. Yet they still fail, because they want to skip steps. If you do not follow along exactly you are doomed. Now I want cookies...
- bluGill 5y ago
  And now I want the recipe. Two days ago please, I want the cookies now.