5 ms·
Ask HN: Why do recommendation algorithms suck?
I see plenty of discussions about recommendation algorithms sucking (ex: YouTube).
When I hear about recommendation algorithms someone always brings up Machine Learning. I've been thinking about how to make a better recommendation algorithm, here's my idea:
First, we ask the user to select topics he likes from a given list.
Second, we ask the user to select topics he dislikes from the same given list.
Then we present recommendations of 4 types :
(Familiar) : Content from topics the user clearly likes.
(Fresh): Content from topics that the user likes that intersects with topics that user does not dislike.
(Novel): Content from topics that the user does not dislike.
(Hit-or-miss): Content from liked topics that intersects with disliked topics or content from liked topics that intersects with not-disliked topics that intersects with disliked topics. (more brefly: [like and disliked] or [liked and not-disliked and disliked])
We present those 4 types with the following ratio :
(Familiar) 50%
(Fresh) 30%
(Novel) 18%
(Hit-or-miss) 2%
Each time the user watches a video from type (Fresh),(Novel) and (Hit-or-miss) we give the opportunity to the user to add the topics of the video that are in doesn't disliked or disliked to the liked category or not-disliked category. If the user does not do anything it stays in their respective categories.
At any time the user can change his preferences in the settings.
I wonder if others think that this is a good idea for handling recommendations? Do you think that it is better to do it that way as opposed to relying on machine learning or the tiktok 5 sec rule.
Was inspired by this discussion specifically : https://news.ycombinator.com/item?id=24578603
- Shared404 6y agoOn a surface level I would like this better. It's not a black box, and seems like a decent spread. My question would be how do you get all of your content classified in a reasonable way?
- nassimBreeze 6y agoIf I look at websites like Quora or Medium they usually ask you to select broad categories. I haven't thought deeply about it but tags that are already used on YouTube and Medium seems to be the most obvious to classify the content. Do you have an idea in mind?
- Shared404 6y agoI don't have an idea other than tags, I was curious if you did. Tags seem like a good idea, the only thing that worries me is that they would be easy to abuse. That being said, I'd be willing to bet it would come out better than what we currently have.
- nassimBreeze 6y agoIf you look at YouTube there are tags but also the category that you can select when you upload a video (Lifestyle/How-to and so on) so maybe we could simply use that along with tags to get accurate classification. However, if I made a new video platform (Daunting task) I would penalize content creator who purposefully mis-categories their content.
- Shared404 6y agoSeems like a decent proposition. If you wanted to hack around with the idea invidious might be a good place to start. That should cover your frontend, so you could focus on the recommendation scheme. Edit: If you do decide to play around with it and decide to open source it, could you send me a link to your repo? I'd be interested to watch, and possibly contribute here and there. My email is in my bio.
- madamelic 6y agoDefinitely not an ML person perspective: One type of recommendation system is user-user recommendation, so it is taking stuff you've (A) looked at, comparing it to what other's have seen (B), then trying to find what from A that B might like from weighting features of each possible item to find the item that person might like most (and therefore, do the action you want them to: view, buy, etc) So if you view Video 1 (action), Video 3 (action) and Video 4 (comedy). And someone else views Video 4 (comedy), Video 2 (comedy) and Video 3 (action). If we are only using genre as a feature, the system would want to recommend you Video 2 (a comedy you haven't seen) and it would recommend the other user Video 1 (an action video they haven't seen). Your system is introducing new features that could be useful. In my opinion, the reason why recommendation systems suck is either people learning how to game it and therefore have their stuff recommended when their item shouldn't be, the system not having enough data or the system just not being great at figuring out what you like due to bad weights. EDIT: Fully re-reading this post, it seems my post is a bit off-topic. With that said, ML is not a dirty word or has to be some magical thing. It is applying stats to a large dataset. It's not anything fancy nor does it have to use the new shiny thing. ML can be as simple as: go through this list of users and videos, given these weights, what videos would each user likely like most?
- nassimBreeze 6y agoI wonder why big platforms don't give more control over their recommendations compared to what I propose. Maybe it's stakeholders or they want to promote certain things regardless of user liking. Also thank you for making tube and reddit.qnzl.co. There is a whole community at reddit.com/r/nosurf who really need those.
- deleted 6y ago[deleted]
- woofcat 6y agoIt's more complicated to the point that Netflix offered a $1,000,000 prize for someone who could build a better one. https://en.wikipedia.org/wiki/Netflix_Prize https://en.wikipedia.org/wiki/Netflix_Prize
- nassimBreeze 6y agoLooking at the link. It seems that it was beaten and someone received the money. Now, this is my reason for posting my idea here : it's to get feedback on it. It seems to be rather unlikely that I would come up with something better than teams who are payed to come up with those recommendation algorithms. However, what do you think of the proposed idea. Maybe in theory it looks better but until I A/B test it I might never know if it's really better than what's currently out there.
- jfengel 6y agoI believe that the Netflix Prize was fundamentally flawed. They have access to much better information than user rankings, especially for the online service, things like "started but not finished" or "watched all of it the day it came out" and "watched more than once". That's much more honest feedback than user's star rankings, which can be skewed by a lot of factors: users failing to rate things, rating based on what they feel they should like rather than what they actually enjoy, the psychological anchor of the suggested ranking. It's not surprising that their existing work got most of the value out of that data. They have much better data now, and I'm sure they're not letting anybody get so much as a glimpse.
- youtubetoss 6y agoAnother reason you didn't list is that it is surprisingly difficult to generate recommendation algorithms that don't empower hate speech. As an example we had to update the filters downstream of the main recommendation model dozens of times over the course of a week to stop Epstein and Pizzagate "researchers" from connecting with each other during the last election
- ethn 6y agoI’ve given up on content-based recommendation. The issue is that people’s interests in their topics and subtopics is primarily dictated by their individual circumstance and environmental real-world context. That is, if you are able to perfectly know the interests of an individual today, perfectly label the topic of all content, you’ll still give them incorrect content after a few days—-and with news article recommendations, by the next session of platform engagement (since they’ve sufficiently extracted the maximum value from the present topics by the nature of the session).
- nassimBreeze 6y agoMy idea allows for change of categories pretty much anytime. This user control maybe be able to circumvent the problem you're describing.
- ethn 6y agoThey’ll still have to sift through tons of no longer context-relevant content. You’re also expecting the user to rationally know his context in terms of your chosen categories, and worse, to put in the work to formalize them for your engine at every session. There is no information in any content or a previous interest matrix which guarantees any real-world relevance in the next session. Instead, it’s likely the user exhausted the utility of their previous topic matrix. The effect is that the moment you determine content relevant to their context, they’ve already been sufficiently informed. I’ve tried algorithms far more advanced with ML/Lebesgue measure theory, which were technically optimal, but with practically laughable results.
- jacobobryant 6y agoI'd be interested to hear your thoughts on this: https://findka.com/blog/essays/ https://findka.com/blog/essays/ (a project idea I'm thinking about doing). I wonder if preference for essays/evergreen content (as opposed to news articles) would be less affected by day-to-day context.
- sovok_x 6y agoI think there can be some additions: 1) for user to be able to manage those percentages directly or using various pre-made profiles and maybe allow some randomness mixed in; and 2) "exploraton mode" where topics with content that is "disliked but not conflicting with the core values", "disliked but they may try to convince me here" and "initially neutral, unknown or not entirely irrelevant to them" are suggested. I also think that attempting to extract the core values as hidden variables and using them to predict, or guess from oher users and external news, the future direction of their evolution can also be useful.
- st1x7 6y agoIt's difficult to talk about a good or better algorithm without quantifying what that means. Youtube aren't trying to build a good algorithm. They're optimizing for a metric like time spent on the website. If your algorithm could beat their current solution in an A/B test, I'm sure that they would be up for it.
- l0b0 6y agoYouTube recommendations could be made enormously more useful with two trivial (really!) fixes: SELECT * FROM videos WHERE NOT EXISTS (SELECT 1 FROM votes WHERE video_id = videos.id AND user_id = $user_id) AND NOT EXISTS (SELECT 1 FROM not_interested WHERE video_id = videos.id AND user_id = $user_id); Aka don't recommend anything I've already voted on or already explicitly told you I'm not interested in. That would remove at least 98% of my recommendations. Even without that improvement, though, YouTube is an amazing service, and the reliability is first class. I can't remember the last time it was actually down. Personally I suspect the YouTube team is just tasked with keeping things ticking over, and that they're too busy putting out fires and coping with tech debt to actually develop useful features.
- sovok_x 6y agoYeah, they probably just plain economize on compute there.
- l0b0 6y agoConsidering the computation necessary to downscale every video I'd suggest this join would be a small drop in a big ocean.
- sovok_x 6y agoMaybe there are some differences between offline and online compute like page response times, cluster specifics, database latency etc. There are no hard timing or effectiveness requirements for processing videos after all.
- cameldrv 6y agoThe recommender is not there to be useful to you. If it is, that is just a side effect. The recommender is there to make you watch more YouTube. Even if you've explicitly said you're not interested in something, that doesn't necessarily mean you won't watch it. Believe me, the YouTube recommender has plenty of resources allocated to it, because it's what makes YouTube such a huge business. If YouTube cared about being useful, they wouldn't constantly reenable autoplay after you turn it off. Autoplay turns back on because it makes you watch more YouTube.
- mcphage 6y agoI remember when the Netflix prize was underway, there were certain movies—Napoleon Dynamite was one of them specifically—where it was incredibly difficult to determine if someone would enjoy it based on their other ratings. And yet it was also enormously popular, so lots of people ended up reviewing it. Honestly, read up on the articles discussing the Netflix prize, they tended to mention a lot of unexpected gotchas. For instance, it turns out, ratings aren’t really independent. If you rate something good, it will affect how you rate the next thing. Stuff like that.
- zaurbekstark 6y agoTikTok as an interesting take on recommendations compared to other platforms. Instead of presenting a list of recommendations (like YouTube, Netflix or Instagram does), TikTok infers what it thinks you will like and directly shows it to you. Then based on how you react: if you like the video, follow the creator, watch it until the end, or swipe away, they know with much more accuracy the type of content you enjoy. This allows for more testing on their part too. On the other hand, on platforms like YouTube, there are a lot of factors beyond the video itself that influences if a person even clicks on it (like the thumbnail, the title, the number of views, who the creator is), so if someone doesn't click on a video, that doesn't tell you if they would have liked it or not. Because of that their data to make recommendations isn't as accurate.