For a long time, I have been curious why overfitting their recommendation model generates more revenue. My guess is that this is stark for kid-friendly content.

For a long time, I have been curious why overfitting their recommendation model generates more revenue. My guess is that this is stark for kid-friendly content. If parents have noticed that a certain set of videos is safe for their children, they are likely to not deviate from that set. Today, when I found this thread, I started looking for past HN threads discussing this topic. I came across a 2019 thread (https://news.ycombinator.com/item?id=18999326 https://news.ycombinator.com/item?id=18999326) where a similar discussion was held. In that post, a user mentioned the following: > "Read the YouTube recommendation paper (https://ai.google/research/pubs/pub45530 https://ai.google/research/pubs/pub45530) and it will become clear why it recommends a ton of high-engagement, clickbait-y content based on a minimal set of recent watches." I skimmed through the paper, and they have some interesting stuff in there! Maybe I should read it more carefully later. Here's some I found: > We consistently observe that users prefer fresh content, though not at the expense of relevance. In addition to the first-order effect of simply recommending new videos that users want to watch, there is a critical secondary phenomenon of bootstrapping and propagating viral content. > Consider as an example a case in which the user has just issued a search query for “taylor swift”. Since our problem is posed as predicting the next watched video, a classifier given this information will predict that the most likely videos to be watched are those which appear on the corresponding search results page for “taylor swift”. Unsurpisingly, reproducing the user’s last search page as homepage recommendations performs very poorly. This is probably the most interesting one! > We observe that the most important signals are those that describe a user’s previous interaction with the item itself and other similar items ... As an example, consider the user’s past history with the channel that uploaded the video being scored - how many videos has the user watched from this channel? When was the last time the user watched a video on this topic? These continuous features describing past user actions on related items are particularly powerful because they generalize well across disparate items...