7 ms·
I gave a talk at PyData Berlin on how to build your own TikTok recommendation algorithm. The TikTok personalized recommendation engine is the world's most valua
by jamesblonde 8mo ago
I gave a talk at PyData Berlin on how to build your own TikTok recommendation algorithm. The TikTok personalized recommendation engine is the world's most valuable AI. It's TikTok's differentiation. It updates recommendations within 1 second of you clicking - at human perceivable latency. If your AI recommender has poor feature freshness, it will be perceived as slow, not intelligent - no matter how good the recommendations are.
TikTok's recommender is partly built on European Technology (Apache Flink for real-time feature computation), along with Kafka, and distributed model training infrastructure. The Monolith paper is misleading that the 'online training' is key. It is not. It is that your clicks are made available as features for predicitons in less than 1 second. You need a per-event stream processing architecture for this (like Flink - Feldera would be my modern choice as an incremental streaming engine).
* https://www.youtube.com/watch?v=skZ1HcF7AsM https://www.youtube.com/watch?v=skZ1HcF7AsM
* Monolith paper - https://arxiv.org/pdf/2209.07663 https://arxiv.org/pdf/2209.07663
- computerthings 8mo ago[dead]
- deleted 8mo ago[deleted]
- dmix 8mo agoI noticed Youtube shorts also seems to update the feed based on how long the last video you watched. If you're scrolling quickly then stop to watch a dog video long enough the next one is likely to be another dog video.
- pandemic_region 8mo agoI've been insta-skipping tennis video's for months now. Still getting Federer on a daily basis.
- randysalami 8mo agoI’ve noticed the same thing and this creates such a negative user experience. Every short is a reaction test and if I fail, I get slop. Makes the whole experience very jarring (for better or for worse).
- datsci_est_2015 8mo agoFor better or worse with regards to my addiction, my subscriptions are all either science channels or high effort / high production comedy skits (e.g. DropoutTV). I still get slop, but I never subscribe and it mostly remains background noise
- AreShoesFeet000 8mo agoThat’s the point though. It may seem as if you’re not in control when scrolling, but you can adjust your behavior to get the content you’re looking for almost intuitively. That’s actually something good in my honest opinion.
- Jensson 8mo agoWhy is it good that you need self control to not get slop? Its much better if you can just turn that off and relax rather than having to stay alert to avoid certain content that it tries to trick you to serve you more slop. Distancing yourself from temptations is an effective and proven way to get rid of addictions, the programs constantly trying to get you to relapse is not a good feature. Like imagine a fridge that constantly puts in beer, that would be very bad for alcoholics and people would just say "just don't drink the beer?" even though this is a real problem with an easy fix.
- AreShoesFeet000 8mo agoIt’s because content curation is inherently impossible to reach the same level of relevance as direct feedback from user behavior. You mix in all kinds of biases, commercial interests, ideology of the curator, etc, and you inevitably get irrelevant slop. The algorithm puts you in control a little bit more.
- BoxOfRain 8mo agoOne of my gripes with youtube at the moment is that they break my adblock filters to remove shorts more often than they break the filters stopping the actual ads.
- sidharthv 8mo agoI naively searched in the mobile app settings for a way to turn off shorts, before realising there will not be one.
- ffsm8 8mo agoYou can't turn it off entirely, but if you keep using "show less shorts" from the 3 dot menu it eventually goes away, mostly...
- direwolf20 8mo agoIt comes back. It acts like it executed shortiness+=1 every day, and "show fewer shorts" does shortiness-=10 or thereabouts. The shorts position on the home screen is based on this hidden shortiness variable. It always bubbles back to the top unless you keep pressing "show fewer shorts" whenever you see it.
- ffsm8 8mo agoStrange, for me it only comes back up if I get baited into clicking on any short (or have one linked to me)
- mghackerlady 8mo agoIf you're on android it's better to just use revanced or something
- notpushkin 8mo agoYou can use NewPipe or YouTube ReVanced, or set up a “child” account and disable Shorts in the parental restrictions settings.
- beAbU 8mo agoFacebook does the same. The longer I dwell on an image post, the more likely the next batch of posts would be similar
- coliveira 8mo agoThe right way to look at these networks is that people are being trained by the algorithm, not the other way around. The ultimate goal is to elicit behaviors in humans, normally to spend more time and spend more money in the platform, but also for other goals that may be designed by the owners of the network.
- touristtam 8mo agoIs amazon using the same thing??? I can't count the number of times I am getting recommended the EXACT same type of product I just purchased.
- beAbU 8mo agoOn amazon.ie I'm convinced they are running only two ads, because all I'm ever seeing are ads for grime brushes and window squeegees. Literally nothing else.
- kgeist 8mo agoIt creates a weird feedback loop: after I watch video A, it recommends a similar video B, and if I make the mistake of watching that too, it then recommends video C on the same topic. Suddenly my feed is nothing but Stranger Things shorts for two whole days (literally not a single video about anything else). Skipping or disliking didn't help, then somehow it went back to normal after two days.
- rjh29 8mo agoyoutube's algorithm seems to be "oh you watched this video? now here's every other video by this creator, pretty much without a break, until you downvote it" It never reliably gives me videos similar but not exactly the same, i.e. things I might be interested in.
- sensanaty 8mo agoFor me it's the same exact 5 videos on repeat, over and over and over again. I've gotten in a loop a lot of times, where it'll autoplay the same video I just watched, it's absolute madness
- ryanjshaw 8mo agoGreat insight. Any thoughts on RisingWave?
- jamesblonde 8mo agoThat, too, and materialize. Feldera is my favourite, though.
- Jamesbeam 8mo ago[flagged]
- deleted 8mo ago[deleted]
- vjerancrnjak 8mo agoFlink is too slow for this. If by features you mean tracking state per user, that stuff can be tracked without Flink insanely fast with Redis as well. If you re saying they dont have to load data to update the state, I dont see how massive these states are to require inmemory updates, and if so, you could just do inmemory updates without Flink. Similarly, any consumer will have to deal with batches of users and pipelining. Flink is just a bottleneck. If they actually use Flink for this, its not the moat.
- btown 8mo agoYea, the Monolith paper by Bytedance uses Flink but they only say it's in use for their B2B ecommerce optimization system. Maybe this is intentional ambiguity, but I'd believe that they wouldn't rely on something like Flink for their core TikTok infrastructure. My hunch is we start to learn a lot more about the core internals as Oracle tries to market to B2B customers, as Oracle is wont to do!
- vjerancrnjak 8mo agoFlink is not really a performance choice, it's bloat to throw software as fast as possible at problems. I don't think there's any benchmark demonstrating insane capabilities per machine. I definitely couldn't get it to any numbers I liked, given other stream processing / state processing engines that exist (if compute and inmemory state management is the goal). Pretty sure any pathway that touches RocksDB slows everything down to 1-10k events per second, if not less. The problem of finding out which video is next, by immediately taking into account the recent user context (and other user context) is completely unrelated to what Flink does -- exactly-once state consistency, distributed checkpoints, recovery, event-time semantics, large keyed state. I would even say you don't want a solution to any of the problems Flink solves, you want to avoid having these problems.
- miohtama 8mo agoTikTok's differention is the userbase of all teenagers in the world.
- AlienRobot 8mo agoIt also provides different opportunities for growth compared to other social media. A video that gets over half a million views on TikTok may not get 5 thousand on Youtube, or even 10 views on Instagram or Facebook.
- bigfishrunning 8mo agoIsn't the inverse true though? it's not as if nobody's watching youtube, it's just that different videos are popular there.
- dehue 8mo agoIt's not just different videos, Tiktok is much better at recommending videos by very small creators and people with no followers. On Instagram or Facebook if you don't already have a large following you most likely wont get any views at all no matter how well your video matches the platform. YouTube often pushes big creators that already made it big while Tiktok allows me to discover new and niche ones.
- notyourwork 8mo agoThat didn’t by accident though.
- wongarsu 8mo agoBut go just one layer deeper to 'why is every teenager using Tiktok' and the primary answer once again becomes 'Tiktok's recommendation engine'
- deleted 8mo ago[deleted]
- 8mo ago
- lsuresh 8mo agoThanks for the Feldera shoutout Jim. For anyone else, if you want to try out Feldera and IVM for feature-engineering (it gives you perfect offline-online parity), you can start here: https://docs.feldera.com/use_cases/fraud_detection/ https://docs.feldera.com/use_cases/fraud_detection/
- bobek 8mo agoIt is not only recommender though. These guys [1] seem to be able to react pretty quickly and not to create addicts on the way ;( [1] https://recombee.com https://recombee.com
- 3abiton 8mo agoIt's interesting to how they found out the "lifetime" of features is a feature by itself. Meta features is real.
- cactusplant7374 8mo agoI thought was secret information. How long as it been publicly known?
- permo-w 8mo agoThe secret is how it chooses videos to recommend, not the tech stack it uses and how fast it is
- cactusplant7374 8mo agoI would assume it's time spent on video and they start to build a profile of which users like what kind of content. X liked video 934934 so Y probably also likes that kind of video. Group people in buckets.
- permo-w 8mo agoI'm sure this is part of it, but I suspect it goes deeper than that. I'd guess they probably have some kind automated categorisation algorithm that can extract features from the videos
- eddd-ddde 8mo agoI have to say, it is _extremely_ impressive when a tiktok I watched reminds me of some other tiktok, so I go and search for a very loose description of the tiktok, and the first result is 95% of the time what I wanted to find. I don't think any single other platform has as good a search feature as TikTok does.
- tridentboy 8mo agooh wow, you're really lucky. around my friend groups who use tiktok, the main complaint is how bad the search is. unfortunately for us, getting a specific video is almost impossible =(
- nik_0_0 8mo agoThats super interesting (I deleted Tiktok because it was too addicting!), but this is a common complaint about Instagram is that it feels impossible to find a reel based on keywords.
- deleted 8mo ago[deleted]
- not_ai 8mo agoI’m happy to see that Flink is in this stack, I wish that Pulsar was as well instead of Kafka.
- SpaceManNabs 8mo agoapache flink is so good. i think netflix used it heavily in 2018. not sure about now.
- NedF 8mo ago[dead]
- permo-w 8mo agoI'm sorry to point out the obvious here, but who is going to perceive their recommended feed as slow or unfresh if it doesn't learn from exactly the last video you clicked on within 1 second? The bar simply is not that high. The special sauce of TikTok is how it chooses the videos, not the speed it does it at. I'm sure the speed helps to give it that "spookily intelligent" feeling, but that's a cherry on the recommendation cake, a cake which is already twice as good as the nearest competitor. I'm sure your talk goes deeper than this, but if this is the main focus, then you've missed the point.
- owenversteeg 8mo agoI partly agree and disagree. Speed completely changes the game in a few ways. The first is identifying interests. Imagine every possible interest in a tree structure. Let's say you're into kumiko. There are so many levels of the tree to traverse to find kumiko; perhaps Skilled crafts -> Woodworking -> Japanese -> Construction without use of fasteners -> Panels and decorative elements -> Kumiko. The more iterations you can get through, the better you can match people's interests. If someone has 10 interests and each one requires many questions to determine, it can take forever to find exact interests with a system that only narrows down your interests every X videos vs. after each video. The second is matching current moods. Let's say you just broke up with your girlfriend, or your pet fish died, or you're on vacation in Spain. A rapidly-updating system can capture those trends and get right to the heart of them in time for them to matter. A slow system might only get through a few iterations and capture a vague interest in Spain; a fast-updating one can get through countless iterations of guessing. Spain? What city? Tourist or moving there? What type of tourist? Foodie? What type of food? How fancy? Bam, you're watching the perfect video about an upscale seafood restaurant in Barcelona. The third is type and flavor of content. Even inside of a small niche you will find many flavors of content. Super-short or long form, fast paced or slow, funny or serious, intellectual, irreverent, political leanings, background music, et cetera. Maybe you like slow long-form woodworking content but like fast-paced travel guides. Maybe you hate background music except when it's in skateboarding videos. To determine this requires an incredible amount of "questioning" of the user. Now, of course, an algorithm that updates once daily can also make inferences about your interests and preferences. It can certainly learn, with enough time, what you are into and how you like to consume it. But the key thing is that these inferences only enable _predetermined_ changes. Imagine you are a human showing someone TikToks. Imagine that you can ask them any questions about their preferences right as they watch a video. You may not ask a question after every video, but you will ask countless questions over the hours of scrolling that day, and you will get good data. Now imagine a new restriction: you must decide your questions once a day in advance. You will manage far fewer questions; and to follow up on them you must wait yet another day. Now, why do I partly agree? Well, I don't think speed is everything; I think TikTok has another sort of je ne sais quoi to it. I think it has a unique culture and community. It has a better UI and better features than Instagram. It has a young and cool reputation, far from the Millennial taint of Instagram or Facebook. And I suspect that they are good at identifying _who_ you are and acting on that information. But in my eyes, the speed could very well be the most important part of the puzzle.