3 ms·
Honestly, all review aggregates are kind of trash. It's crazy that we have stuck with either binary or 5 star ratings this whole time across the internet. I ha
by legitster 3y ago
Honestly, all review aggregates are kind of trash. It's crazy that we have stuck with either binary or 5 star ratings this whole time across the internet.
I had a product idea I have yet to make where you replace ratings with rankings. Instead of giving something a 1-5 review, you just answer a few quick questions whether something is better or worse than a listed alternative. You aggregate enough rankings and you can give everything a percentile score. The number is actually meaningful - a 70% means people on average think that it's 70% better than all ranked alternatives.
And you can't lie or influence a ranking as easily. "You think Rings of Power is a good show? Okay, but are you are actually going to rank it above The Sopranos?"
- passwordoops 3y agoI love this idea and put it in practice (sort of) when working on prioritization. For example, say we want 15 features done, I compare feature A to the other fourteen asking "If I can only do A or X, which would I do?" with the winner earning a point, kind of like a Round Robin tournament. Then do the same for the rest on down and tally up the scores. Hasn't failed me yet and if you do it as an org it helps with arguments down the line
- hi_im_vijay 3y agothat's a sweet product idea, you should prototype it :)
- speedgoose 3y agoThe French senscritique.com has a lot of community made ranked lists. They are a nice alternative to the classic ratings.
- mfenniak 3y agoI like that idea quite a bit. I wonder if there's an algorithm that doesn't get completely screwed up by circles... since it is subjective, A > B > C > A is a valid input and shouldn't cause a complete algorithmic failure.
- FartyMcFarter 3y agoThat is a solved problem. Elo ratings in chess and other games have to deal with such situations by design.
- legitster 3y agoYou only present users with the bare number of options to establish the ranking. "Was your most recent purchase better or worse than purchase B? Was it better or worse than purchase F?" If the software keeps an up to date ranking, you don't have to ask more than a handful of questions to accurately assess where the new entry goes. If an entry causes a logical fallacy, that is an opportunity to represent the data in a different order and see if the user changes their ranking. This will actually help to keep the data fresh. And you can retain "fuzzy" rankings in certain areas without threat to the accuracy of the overall database. If you want to have a multivariable structure, users could rank more than facet at once. So for a car, you could compare if a Honda Civic is better or worse than a Toyota Corolla on handling, comfort, features, etc. Combine this with non-subjective data (price, 0-60, etc) and users can choose if they want an aggregate ranking or weighted based on their criteria.
- dameyawn 3y agoLove this idea.
- terminous 3y ago> Honestly, all review aggregates are kind of trash. It's crazy that we have stuck with either binary or 5 star ratings this whole time across the internet. Are you aware of metacritic? They take all kinds of ratings, scales, stars, grades, etc, from all kinds of critics and reviewers and turn them into nice 1-100 percent ratings to average.
- legitster 3y agoYes. But this is just a weighted aggregate of subjective reviews. A 7/10 from Gamespot is useless data because they give 7/10 to everything.
- terminous 3y agoSure, I agree with you on that. But that's a different argument than "It's crazy that we have stuck with either binary or 5 star ratings this whole time across the internet." We haven't.
- ramesh31 3y agoThis is why I love Ranker so much. Puts any kind of review aggregation to shame, especially with movies and video games.
- gffrd 3y agohahaha: https://www.ranker.com/list/actors-that-love-history-but-not-accuracy/will-morgan-1?ref=hp_ranking&l=1 https://www.ranker.com/list/actors-that-love-history-but-not... The _Actors Who Love History But Not Accuracy_ list. Mel Gibson holding #1, but look out: Costner is on the rise!
- johnchristopher 3y ago> And you can't lie or influence a ranking as easily. "You think Rings of Power is a good show? Okay, but are you are actually going to rank it above The Sopranos?" Why not ? It depends what you are looking for at that moment.
- gffrd 3y agoIsn't the advantage of ranking in support of what you're saying? If, in the future, your tastes change, a few things get ranked "above" what formerly held your top slot. The top slot was never "200 absolute points," it was just "the highest single ranking" Although, I do see the coarseness of a new #1 bumping everything down … and forcing a reconsideration of whole blocks of rankings … arriving at "groups" … and basically a star system.
- johnchristopher 3y agoMy tastes don't change (bear with me) but the mood I am in does [0]. So one saturday Rings of Power is a good fit but next saturday the Sopranos is a better fit even if all in all I ´d systematically rank one above the other. So suggestions solely based on how I'd rank things on a single axis are not relevant enough. - [0] my mood changes more often than my taste do
- gffrd 3y agoFair! If I've spent all day on calls, then proceed to watch, for example, anything Aaron Sorkin, I'm likely to treat it less charitably (because I'm tired of flapping gums) than if I watched it after a week in the desert (and human contact is wonderful). My mood would color ratings as well … How would one flatten the effect of mood on a either-or ranking system? Is it possible?
- johnchristopher 3y ago> How would one flatten the effect of mood on a either-or ranking system? Is it possible? I suppose the system should ask for every movie watched how they'd rank given a particular mood ? So it's "is A better than B when you want something with deep thinking to watch, is A better than B when you want something easy to follow ?" but it has its own can of worms: sometimes I want to watch something with deep thinking even though I am in the mood to unwind... All in all, I think it's a waste of time to catalogue our own tastes and try to build a personal recommendation system. I hope/think/want to believe thank knowing ourselves gives better reward.
- deleted 3y ago[deleted]
- EatingWithForks 3y agoThis might run into conflict with nuanced tastes, e.g. I would be a miserable gamer if this is how games are recommended to me because I actually really don't like very shiny new triple a shit. Spiritfarer changed my life and made me weep ugly tears multiple times in a way God of War never did, but I don't think most people would enjoy the gentle and tender approach it has towards its subject matter. [In Spiritfarer, you are a boat captain who picks up and hosts dead spirits until they ask you to take them to the gate where they are gone forever; in the meantime they share their lives with you and mull over death with you. Your only task is to care for these people until they are ready to go.]
- legitster 3y agoI think this would help people find more games like Spiritfarer if you can cut through the obligatory noise generated for the big AAA titles. Imagine if you had the dataset to say "remove every reviewer who ranked God of War above Spiritfarer" you would probably be left with an amazing set of recommendations.
- EatingWithForks 3y agoAt that point, is it different from a general recommendation engine of users who like what you liked also liked X? [A thing which I also struggle with because I can never search based off of specific qualities no matter how many tags I search and exclude in Steam...]
- legitster 3y agoYes. It would be an order of magnitude more accurate. Current recommendation engines currently have account for lots and lots of false positives and are generally constrained to shared product tags.
- freeopinion 3y agoGreat idea. Add categories: is A a better Comedy than B? Romance? Mystery? Action? Is Schindler's List a better Comedy than Gone With the Wind? Is Schindler's List a better Romance than Gone With the Wind? Is Schindler's List a better Documentary than Gone With the Wind?
- legitster 3y agoBingo. You could also make this much bigger than a single product category. If you have enough people willing to mindlessly to swipe on random comparisons during the day so they can see their own report, and you could properly sort and tag all categories, you could have a truly bonkers data set. Like whether dollar for dollar consumers prefer the Barbie movie to owning a Porsche Cayenne.
- rvba 3y agoThe problem is that the people giving the rank probably never saw Sopranos in the first place, or are idiots whose taste changes on a whim.