10 ms·
The secret sauce of TikTok’s recommendations
- dakial1 4y agoTo be fair tiktok helped me find a lot content (series, movies, etc) and artists (specially standup comedians) that it would take a much longer time to cross-paths with. So I would see a short of that, and then looked around to watch the whole thing. But in the 1 year I've interacted with the app I've already seen the effects of the content creators learning how to game the algorithm and get presented with some contents that were made in a format only to get engagement, similar to a clickbait article. The result? It has spoiled the whole experience for me and I uninstalled the app. And this is how those platforms slowly crumble to their own weight as the crowdsourcing becomes a business for creators and, little by little, content becomes more and more superficial, fake, commercial and uninteresting for the audience, who migrates to another app and everything starts again...
- foxhill 4y agothis is a wall of text citing a single paper which itself is entirely unverifiable. regardless, if this described how drug dealers engineer the distribution of their product to optimise addiction, we would not be celebrating it as an engineering marvel.
- zenogantner 4y agoThis blog post is unlikely to contain TikTok's secret sauce, or even anything that is actually used by TikTok. ByteDance runs TikTok, but they also sell recommendations as a service. The paper that this blog post is based on explicitly talks about that service in the abstract, and never even mentions TikTok. Now, if I were to advertise a service that contains some of the ingredients of my other, very popular product, I would surely mention that fact ... but this is not the case here.
- PaulHoule 4y agoI like those charts showing they get an AUC of 0.8 or so, my own content-based recommender gets 0.72 or so on a typical day with a very simple model, I think it could get 0.75 with mainly data-oriented tweaks. I was thinking a better model could get me into the high 70's but the fuzzy nature of the recommender problem means I am never going to get into the 90's. I would like to see more about how they are formulating the problem, I know there is work lately in "sequential recommendation" that is focused on generating a sequence rather than scoring content items, I'd like to learn more about that.
- O__________O 4y agoAUC being “area under curve” or something else?
- acomms 4y agoCorrect, it is a performance measure for ML algos (https://www.sciencedirect.com/science/article/abs/pii/S0031320396001422 https://www.sciencedirect.com/science/article/abs/pii/S00313...)
- PaulHoule 4y agoYep. See https://en.wikipedia.org/wiki/Receiver_operating_characteristic https://en.wikipedia.org/wiki/Receiver_operating_characteris... If you had to pick one number for judging classifiers that is the one. "Accuracy" is a really bad one because it is not meaningful w/o careful thought about the problem. For instance if you have a diagnostic system for a disease that has a .01 prevalence, you get 99% accuracy if you say nobody has a disease. Practically a person might want to operate a classifier with different thresholds depending on how bothered they are by false positives vs false negatives, the ROC shows what choices are have available with a particular and the AUC of the ROC gives you one number that characterizes the quality of the classifier overall. A system with an AUC under say 0.55 hasn't really learned anything at all, at 0.6 it is showing signs of life, looking at those curves I'd say Tik Tok is beating me solidly. In this case it is limited by the problem being fuzzy (something I thumbs up today might get a thumbs down tomorrow), something up in the 0.95 level could be attained for something like "is this article about the outcome of a sports game?"
- mochomocha 4y ago> the fuzzy nature of the recommender problem means I am never going to get into the 90's Right. If your AUC is in the 90's you usually either have a bug, or a predictive task that isn't very challenging.
- PaulHoule 4y agoSome tasks are hard, others are easy. I get in the low 60’s trying to predict if a headline will get a lot of votes on HN. Of course this is going to be bad because the same article can get submitted several times, hit it big once and not hit it big the other times. The content of the article matters too but the model can only make a wild-assed guess. I get in the high 70’s trying to predict if a post with a lot of hits gets a lot of comments, it is a cleaner problem. I get about 90% trying to predict if a post winds up ‘dead’, I found up HN is cleaning up a lot of spam posts when they go in but I know it is terrible overfit because it says an arXiv paper which has no chance of being flagged has a 30% chance of flagging. For some other like “is this an astrophysics or materials science paper?”, getting 0.95 or better was straightforward in 2005.
- Jamesmoorez 4y ago[dead]
- droopyEyelids 4y agoIn these articles I never see reference to the 'objective algorithm' aka what the business processes at TikTok are optimizing for. I think that's the real secret sauce. TikTok is optimizing for user generated content that connects deeply with their users. Thats what they reward money which is largely derived from TikTok's "gift economy", and recommendations and therefore views are largely based on the gifts. This is in stark contrast with something like YouTube, which is optimizing for 'time spent on YouTube' which drives their creators to make longer clickbait type videos, regardless of how dissatisfying they are. The recommendation algorithm described here could be tuned to serve any 'objective algorithm', like time spent, profit to Bytedance, or whatever- but because of TikTok's choices it is making a really fun an enjoyable experience for users.
- deleted 4y ago[deleted]
- hgsgm 4y ago> regardless of how dissatisfying they are. Why would people spend time watching dissatisfying videos? > content that connects deeply I'd say "parasocially". But how does the app measure that? It makes decisions based on a few seconds of watch time. It could optimize for "show videos that trigger donations", but that's kind of trivial.
- thomasahle 4y agoThere might not be a simple topline objective everything is optimized for. More likely it's a linear combination of all kinds of dynamic business needs.
- rightbyte 4y agoI believe their secret sauce is mainly that they don't manipulate the feed for other factors than if it fits. The Facebook feed used to work like that before 2013 something. You could clearly see the difference when they started optimizing for ads or what ever metrics they used. Posts could not go "viral" anymore in the way it used to. Zuckerberg capped the vitality so that companies had to pay I guess.
- nikolay 4y agoCan't agree more! Facebook is useless to me today and replaces meaningful to me stuff with ads and other things I don't care about and never engage with, i.e. there's for sure other weights and the feed is not organic!
- InitialLastName 4y agoI've tested in the past and counted streaks of 30+ consecutive posts in my Facebook feed without any posts from friends, groups I'm in, or pages I've followed. Just "recommendations" and ads for miles.
- nikolay 4y agoI did a test as well. Unlike most people, I've set up relatives very well. Yet not a single post by a relative and some of those have high engagement! All kinds of Instagram Reels and ads and a bunch post from people with high a high absolute engagement, but historically a very low one from me. So, this obviously is a poor filter, which does not keep me engaged and which proves that Facebook is NOT a social network as to be social, it needs to act as such, and I never hear from my closest people over there anymore! How hard it is to see that a post is from my mother-in-law and multiply its weight by 100, or one from my sister and multiply by 1,000? Another thing that Facebook never puts any effort into is photos of children under 13! They had a feature called "scrapbooks" where you can create profiles of your children with the promise of those turning into profiles after kids turn 13. Well, they've abandoned this feature and people kept tagging children as their parents! Not sure how this served their facial recognition, but it inconvenienced us parents!
- randomdata 4y ago> Why is TikTok's feed so addicting? What's addicting about it? Admittedly it kept me entertained the first day or two that I used it, but from that point forward it never wavered from the content that entertained me that first day and soon it became really boring to see the same type of thing over and over, leaving no remaining appeal. Because everyone talks about how great the recommendation engine is, and me wanting to find great content, I went back to it after a while to see if they fixed the problem. But no, still the same content that bored me away from the app the first time.
- paulpauper 4y agoSame here. It's too short. Nothing there. like watching a bunch of gifs but with audio.
- willio58 4y agoI believe you but you should know you are in the minority here. Part of why TikTok has taken over the social media space is because of how addicting it is compared to the older players.
- jdminhbg 4y agoTikTok is massively popular but it's not yet the case that not using it puts you in the minority, like Facebook does.
- fragmede 4y agoWho's your cohort? If you're in high school, yes it does.
- gardenhedge 4y agoDo you have the stats for that? I recall reading something earlier this year than Insta is still king for teens/young people. I can't find the source though.
- fatcat500 4y agoAll that brilliance and knowledge, used to get vulnerable, suggestible teenagers addicted to mind-numbing, anxiety-inducing media. The shame of software engineers is that we have been used to create 1000s of these inhuman products.
- luckylion 4y agoNobody has "been used" to create those things. They do it knowingly and willingly, because they're paid huge sums of money in return. Don't turn it into "we just followed commands" or "we were tricked", own whatever you're working on.
- eimrine 4y ago> The shame of software engineers What decisions are on these guys? How to name some variable? How to receive mole salary? Or what manager's/marketologists' decision not to code?
- jbaczuk 4y agoThe shame of those software engineers
- JohnFen 4y agoI do think it brings shame on the whole field. It's not right, but it seems to be the case.
- blakebreeder 4y agoidk, my TikTok feed is full of cats and dogs doing funny things.
- falcolas 4y agoAnd birds and sneks. Good, wholesome stuff a vast majority of the time.
- tester457 4y agoMy question is why teenagers gravitate to horrendous content en masse. Young girls have feeds full of eating-disorder content, and young boys have feeds mixed with misogyny like Tate. Not all teenagers watch either, but these recommendations are a disease plaguing the platform.
- londons_explore 4y agoThe main reason TikTok has awesome recommendations is that all the other big players (Facebook, Youtube, etc) realised back in 2017 that training machine learning models to persuade people to spend more time on their platform was unreasonably effective. Ie. tell ML to trick someone to spend 14 hours watching youtube every day, and for some small percentage of users, it will actually succeed! For those people, it's as addictive as drugs. They spent all day on youtube rather than going to work, going to school, caring for their kids, eating or even sleeping! Can you imagine the size of lawsuits that would be heading youtubes way when those people realise they've effectively been enslaved by an algorithm?? Leadership of the big companies put an end to that, instead trying to focus on other metrics, and trying to get more users to each spend some time on the platform. Well it seems TikTok didn't get the memo...
- dragonelite 4y agoPeople will pick their own poison to fill their free time, in the past it was reading books then it became watching TV all day im sure most millennials are using apps to fill their free time. I used to fill my free time with world of warcraft spending more time in Azeroth than i did with friends or at school or at work. But i came to the realisation it was just escapism from a boring country/region and a waste of my time. But this is also made me realise how lucky i was to be born in such a boring country/region.
- londons_explore 4y agoI know someone, mid 30's, jobless, who spends 12+ hours a day on TikTok. They don't even post to it - just watch hours and hours of mindless videos about mental health issues.
- justinator 4y agoMaybe not 12+ hours a day, but a good chunk of time and this is my house mate. They watch more Tik Toks about ADHD than they do actually trying to live with ADHD.
- 4y ago
- fdgsdfogijq 4y agoThe secret sauce: Let grass roots content creators actually distribute their content. Let the best content go to the top. Give everyone a chance. Facebooks algorithm: Support influencers, corporations, and whatever pumps the ad machine.
- ssnistfajen 4y agoIt's way easier to find "smaller" content creators on TikTok than Meta's apps. Youtube has been improving a bit in that regard but my recommendations still have too many videos I've already watched. More than half of the accounts I follow on TikTok have <25000 followers, many with average views per video under 10000, with a diverse topic range including: woodworking, software dev, comedy sketches, book reviews, history trivia, exercise tips, urban design, etc. Those who complain about seeing only unintelligent garbage are unwittingly revealing themselves to be the target audience for such content.
- fragmede 4y agoI hate to break it to you, but if you think TikTok has "grass roots content creators" vs Facebook having influencers, I have a bridge, as well as a TikTok content house to sell you. https://www.kapwing.com/resources/tiktok-houses-list/ https://www.kapwing.com/resources/tiktok-houses-list/
- nerdponx 4y agoBut they do seem to be optimizing more explicitly for "user enjoyment" than other platforms, perhaps realizing that this is what keeps and attracts users and their precipitation eyeball-minutes in the long term. Youtube also (not always) does a pretty good job at this, while Instagram can be rough.
- smoldesu 4y agoAnother possibility is that TikTok's short-form content feed allows for a greater volume of feed to fit into the same timeframe. Twitter was popular because it condensed blogging into microblogging; TikTok's success is pretty easy to delineate from that logic. Even if you don't have an algorithm promoting enjoyable/affirming content, you're just more likely to encounter that content when you watch more videos. Whether that behavior is healthy or not... I'll leave that up to people more educated than I. My guess is that trading the user's attention span for short-term engagement isn't worth it, though.
- MonkeyMalarky 4y agoClearing out old and stale IDs from the model is interesting. It makes sense for a platform serving a high volume of viral / short lived content. I can't imagine the same trick working at a streaming service.
- ludoro 4y agoI also think that paper is super interesting, I talked about it here: https://www.machinelearningatscale.com/how-tiktok-recommendation-algorithm-scales-to-billion/ https://www.machinelearningatscale.com/how-tiktok-recommenda...
- O__________O 4y agoRelate HN comments: - https://news.ycombinator.com/item?id=33494796 https://news.ycombinator.com/item?id=33494796 Related paper: - https://arxiv.org/abs/2209.07663 https://arxiv.org/abs/2209.07663 Related GitHub: - https://github.com/bytedance/monolith https://github.com/bytedance/monolith
- paradoxyl 4y agoI was behind a woman today on an escalator. I wasn't shoulder-surfing, she held her phone up so high I couldn't help but see it. She was on Tik-Tok. She watched like twenty videos in one minute. One second, click. One second, click. What good this does anybody at all, anywhere, is beyond me. It was all trash.
- ssnistfajen 4y agoHow is that any different from other social media feeds? Users familiar with a content platform will skim through content faster. That has been the case since the TV era.
- reidjs 4y agoSomeone might say the same thing about you if they were looking over your shoulder and you were reading HN comments.
- zamfi 4y agoWhat good does a cigarette do?
- deleted 4y ago[deleted]
- Aaronstotle 4y agoI smoke one a day, and the good it does for me is carve out 5 minutes of reflection on the day.
- mochomocha 4y agoPretty click-baity title. This article is a good "ELI5" summary for non-ML people of some of the techniques & infra described in the arxiv paper, but it has nothing to do with TikTok "secret sauce".
- t3estabc 4y ago[dead]
- boredemployee 4y agoAnother analysis I'd really like to see is how Amazon does their forecasting. A friend that works there told me it's crazy.
- danrocks 4y agoSecond half of this video [0] is a relatively deep view of some of the key phases of order fulfillment. [0] - https://www.youtube.com/watch?v=nceTzlnZ_gk https://www.youtube.com/watch?v=nceTzlnZ_gk
- steele 4y agohttps://www.npr.org/sections/codeswitch/2022/02/14/1080577195/tiktok-algorithm https://www.npr.org/sections/codeswitch/2022/02/14/108057719...
- barbariangrunge 4y agoCurious: every terms of service says you are committed to not reverse engineering their tech. Does this count, and if so, what does that mean?
- datruth29 4y agoAs a person who occasionally consumes TikTok videos, I feel like it taps into the same thing that the original StumbleUpon did; show the user fairly quickly a number of related things and give them the option to either consume more of that same thing or find something else that's related but also new. Same goes for Pinterest. They all share that core element of quick consumption of related things in common.
- elkos 4y agoI feel that StumbleUpon provided more relevant content to me when I used to use it than TikTok.
- andirk 4y agoPro-tip: On Instagram, interact with ads that you don't mind. That way the algo picks those or similar ads which makes a more pleasant experience. For example, now I mostly get ads of nice pictures of watches.
- Jamesmoorez 4y ago[dead]