9 ms·

An empirical investigation of personalization factors on TikTok

65 points by asmy 5y ago

fsflover 5y ago
See also: TikTok shares your data more than any other app and it’s unclear where it goes (cnbc.com) 473 points by underscore_ku 5 days ago | 364 comments https://news.ycombinator.com/item?id=30272682 https://news.ycombinator.com/item?id=30272682
asmy 5y ago
I was searching some information on what are the principles behind the TikTok algorithm/recommendation engine and I found this very recent preprint. I wanted to have some opinions on its validity.
- PaulHoule 5y ago
  It's probably valid but on a quick reading I don't see a lot of depth or a clear understanding of "what the magic is".
  - asmy 5y ago
    Yeah, I pretty much understood some of it because it is very clear. I was researching some explanation or first-principles approach to the TikTok algorithm but going through ResearchGate/Arxiv but found few technical reports and the best one was this one (and very recent). And going through Google is a mine-field because of marketing companies SEO bombing the words "Tiktok" and "algorithm"
mrkramer 5y ago
I think TikTok essentially throws a lot of videos at you and then they observe what sticks and what not sticks. Basically lather, rinse and repeat Shampoo algorithm[0] strategy. [0] https://courses.cs.duke.edu/summer04/cps001/lectures/Lecture3.pdf https://courses.cs.duke.edu/summer04/cps001/lectures/Lecture...
wongarsu 5y ago
> Through this approach we were able to test and analyse the affect of the language and location used to access TikTok, follow- and like feature, as well as how the recommended content changes as a user watches certain posts longer than others. Our results revealed that all tested factors have an effect on the way TikTok’s RS recommends content to its users. We have also shown that the follow-feature influences the recommendation algorithm the strongest, followed by the video view rate and like feature; besides, we found that the location is a stronger influential factor than the language that is used to access TikTok. Of course, this analysis is not exhaustive and includes only the most explicit factors, while the algorithm without a doubt can be influenced by many other aspects such as, for instance, users’ commenting or sharing actions Subjectively that follows my experience on TikTok: everything you do feeds back into the recommendation algorithm, including how long you watch a video; but the explicit signals (like and follow) have enough weight to make it feel like you have control over where the current takes you. I assume being able to get at the very least an implicit like/dislike signal out of every view is key to TikTok's Exploration/Exploitation tradeoff, which is what makes the platform feel much less stale than e.g. YouTube (which seems to only recommend things it's very sure about).
- PaulHoule 5y ago
  I wonder if there is a multi-arm bandit hidden in there somewhere. I guess the short form video and the "skip past it" culture lowers the risk of exploration -- it's kinda the opposite of that notorious Netflix optimization challenge.
  - fragmede 5y ago
    It's hidden behind the 'live' button top left. Viewers buy emoji with real money and send them to creators, creators get money, and Bytedance takes a cut. Creators do all sorts of things, some show them working, at a farm, at a shipping warehouse, whatever. Others play bingo or scratch-off tickets. Still others sit and talk or play music with their viewers. Sending an emoji is highlighted and so are easier for creators to acknowledge the emoji/give a shout out to that particular username. Thing is, creators only sometimes shout out, encouraging multi-arm bandit behavior on the part of the viewer.
RomanAlexander 5y ago
I wonder if the recommendation engines are different per country. For example in the US, you usually can't be recommended videos from accounts outside the US. Even with a VPN, spoofed geo location, system language, added video language you will be served only mostly videos based on your IMEI.
- ggus 5y ago
  I think it's not the IMEI as on my imported phone I would see mostly local content. (and very bad content at that.. uninstalled after a while and never looked back)
- 5040 5y ago
  That's odd. I get recommended Japanese content despite living in Canada. Much of this content is welcome too.
- el-salvador 5y ago
  Have you tried without a SIM card? I think they use the SIM Card Country Code too.
- Dma54rhs 5y ago
  Set more languages from your profile or account settings. I put it to English, I live in a small euro country and 80% of the content is American to be honest. And at least football related content has various people from all over the world who speak in English in their videos. That comes from top of my head but I don't keep track of it obviously. I only wanted English content but you can add more languages.
dmingod666 5y ago
For the longest time I didn't download it cause of its reputation. It's pretty good, no cruft of YouTube. 'youre watching this, that.. like subscribe.. word from of sponsers' none of that nonsense - pure content. Both FB and YouTube are copying the features button for button. IMO, Not bad
- ggus 5y ago
  I haven't managed to find non-meme content on tiktok.. But I found that SponsorBlock and uBlock make youtube bearable
BiteCode_dev 5y ago
I don't understand how it works for people. I tried for what felt an eternity, and it kept showing me terrible content. So I removed it. Is there something that is not made to turn your brain into a sponge on tik tok ?
- bpodgursky 5y ago
  Your experience, while worth noting, is clearly atypical.
  - BiteCode_dev 5y ago
    I understand that I probably have a niche taste, been a nerd and all, but still. Maybe I'm not the target? Do they have a lot of millennial users or is it mostly for the younger generation?
    - Merem 5y ago
      I think it's just broken in some cases for whatever reason. Like 9 months ago I mentioned trying out Tik Tok for some weeks and despite lots of "not interested", likes etc, I got swamped by 99% garbage, including videos I specifically said I don't want to see.
    - wongarsu 5y ago
      TikTok has lots of content for nerds (though I think they prefer the term neurodivergent). But there is certainly a trend that TikTok content is comparatively shallow. For example Hank [1] and John Green [2] have quite nerdy content, but it's much shorter less developed thoughts than their youtube channel [3]. Or NileRed: his most recent TikTok[4] (eating capsaicin) is in no way comparable to his most recent YouTube[5] (synthesizing cotton candy from cotton balls). In terms of nerdy content it probably shines more with life advice than with science facts, simply because of the short video format. Not sure about the age group, most of TikTok is probably targeted at ages 12-40. 1: https://www.tiktok.com/@hankgreen1?lang=en https://www.tiktok.com/@hankgreen1?lang=en 2: https://www.tiktok.com/@literallyjohngreen?lang=en https://www.tiktok.com/@literallyjohngreen?lang=en 3: https://www.youtube.com/c/vlogbrothers https://www.youtube.com/c/vlogbrothers 4: https://www.tiktok.com/@nilered/video/7063145462794145030 https://www.tiktok.com/@nilered/video/7063145462794145030 5: https://www.youtube.com/watch?v=vHuFizITMdA https://www.youtube.com/watch?v=vHuFizITMdA
mrfusion 5y ago
I’m disappointed with the personalization. I was hoping it would teach me interesting things in short videos and expose me to new ideas. Instead after a few weeks it seems to mostly show me political stuff and pranks.
dakial1 5y ago
I started using TikTok and wanted to check how it would suggest me videos if I didn't like anything, as it would make it easier for it and I wouldn't be able to see how sophisticated it really is. So it obviously would start in a very broad content but already target at my demographics (male 30-40 y/o, which BTW gave a worrying amount of very young girls dancing in a sexualized way) but with time it would certainly adapt and be really spot on on the videos I liked (mainly stand up/comedy stuff). Everything based in signals other than liking of following. They obviously track all interactions from the user (time watching, repeats, shares, OP profile checks, etc) to make this targeting. Also its ML for visual recognition used to identify key aspects on the video (people, location, etc...) is very powerful as it would guess my favorite comedians and start exploring with other videos. Also they certainly use bandit/lookalike model there as it would recommend some wildly different videos for me. And I can bet that that last video it shows when you click to leave the app (on Android at least) is probably the highest rated recommendation for you. To really lock you in there.
bitshiftfaced 5y ago
I'd like to see them address videos that game the system. I'll see videos structured so that the beginning is very slow, and so you must wait to get to the meat of the content. People do wait, and so the video is algorithmically rewarded, but that doesn't mean the video is good. Some videos even have a "fake glitch" effect where the first few frames are from later in the video. It's as if to say "here's what the video's about, but first wait through this boring part so that my video gets rewarded for engagement." I once watched a video over several times because I couldn't make out what was supposed to be "off" about it at the end. I finally realized that there was nothing strange or anything, but the video led you to believe there was. Therefore many people, like me, stupidly rewatched it in order to figure out that there was nothing interesting. And that's why it had a boosted signal!
cormorant 5y ago
In Figure 6 (described in section 4.4), the content seen by "the control user 50" and "the active user 49" are both in large part by content creator "miakhalifa" (32.4% for the control, 68.6% for the active). The "experimental scenario" is "scenario 28", described as "Follow a random content creator". Mia Khalifa is a porn actress. A quick search indicates that her TikTok posts are at least sexually suggestive. I wish I understood what "control" meant in this paper, and what exactly this scenario tested. Does it really mean that a fresh user profile is going to see 1/3 of its feed coming from a porn star? I've never used TikTok so I don't know!

65 points by asmy 5y ago

See also: TikTok shares your data more than any other app and it’s unclear where it goes (cnbc.com) 473 points by underscore_ku 5 days ago | 364 comments https://news.ycombinator.com/item?id=30272682 https://news.ycombinator.com/item?id=30272682

asmy 5y ago

I was searching some information on what are the principles behind the TikTok algorithm/recommendation engine and I found this very recent preprint. I wanted to have some opinions on its validity.

PaulHoule 5y ago
It's probably valid but on a quick reading I don't see a lot of depth or a clear understanding of "what the magic is".
- asmy 5y ago
  Yeah, I pretty much understood some of it because it is very clear. I was researching some explanation or first-principles approach to the TikTok algorithm but going through ResearchGate/Arxiv but found few technical reports and the best one was this one (and very recent). And going through Google is a mine-field because of marketing companies SEO bombing the words "Tiktok" and "algorithm"

mrkramer 5y ago

I think TikTok essentially throws a lot of videos at you and then they observe what sticks and what not sticks. Basically lather, rinse and repeat Shampoo algorithm[0] strategy. [0] https://courses.cs.duke.edu/summer04/cps001/lectures/Lecture3.pdf https://courses.cs.duke.edu/summer04/cps001/lectures/Lecture...

wongarsu 5y ago

> Through this approach we were able to test and analyse the affect of the language and location used to access TikTok, follow- and like feature, as well as how the recommended content changes as a user watches certain posts longer than others. Our results revealed that all tested factors have an effect on the way TikTok’s RS recommends content to its users. We have also shown that the follow-feature influences the recommendation algorithm the strongest, followed by the video view rate and like feature; besides, we found that the location is a stronger influential factor than the language that is used to access TikTok. Of course, this analysis is not exhaustive and includes only the most explicit factors, while the algorithm without a doubt can be influenced by many other aspects such as, for instance, users’ commenting or sharing actions Subjectively that follows my experience on TikTok: everything you do feeds back into the recommendation algorithm, including how long you watch a video; but the explicit signals (like and follow) have enough weight to make it feel like you have control over where the current takes you. I assume being able to get at the very least an implicit like/dislike signal out of every view is key to TikTok's Exploration/Exploitation tradeoff, which is what makes the platform feel much less stale than e.g. YouTube (which seems to only recommend things it's very sure about).

PaulHoule 5y ago
I wonder if there is a multi-arm bandit hidden in there somewhere. I guess the short form video and the "skip past it" culture lowers the risk of exploration -- it's kinda the opposite of that notorious Netflix optimization challenge.
- fragmede 5y ago
  It's hidden behind the 'live' button top left. Viewers buy emoji with real money and send them to creators, creators get money, and Bytedance takes a cut. Creators do all sorts of things, some show them working, at a farm, at a shipping warehouse, whatever. Others play bingo or scratch-off tickets. Still others sit and talk or play music with their viewers. Sending an emoji is highlighted and so are easier for creators to acknowledge the emoji/give a shout out to that particular username. Thing is, creators only sometimes shout out, encouraging multi-arm bandit behavior on the part of the viewer.

RomanAlexander 5y ago

I wonder if the recommendation engines are different per country. For example in the US, you usually can't be recommended videos from accounts outside the US. Even with a VPN, spoofed geo location, system language, added video language you will be served only mostly videos based on your IMEI.

ggus 5y ago
I think it's not the IMEI as on my imported phone I would see mostly local content. (and very bad content at that.. uninstalled after a while and never looked back)
5040 5y ago
That's odd. I get recommended Japanese content despite living in Canada. Much of this content is welcome too.
el-salvador 5y ago
Have you tried without a SIM card? I think they use the SIM Card Country Code too.
Dma54rhs 5y ago
Set more languages from your profile or account settings. I put it to English, I live in a small euro country and 80% of the content is American to be honest. And at least football related content has various people from all over the world who speak in English in their videos. That comes from top of my head but I don't keep track of it obviously. I only wanted English content but you can add more languages.

dmingod666 5y ago

For the longest time I didn't download it cause of its reputation. It's pretty good, no cruft of YouTube. 'youre watching this, that.. like subscribe.. word from of sponsers' none of that nonsense - pure content. Both FB and YouTube are copying the features button for button. IMO, Not bad

ggus 5y ago
I haven't managed to find non-meme content on tiktok.. But I found that SponsorBlock and uBlock make youtube bearable

BiteCode_dev 5y ago

I don't understand how it works for people. I tried for what felt an eternity, and it kept showing me terrible content. So I removed it. Is there something that is not made to turn your brain into a sponge on tik tok ?

bpodgursky 5y ago
Your experience, while worth noting, is clearly atypical.
- BiteCode_dev 5y ago
  I understand that I probably have a niche taste, been a nerd and all, but still. Maybe I'm not the target? Do they have a lot of millennial users or is it mostly for the younger generation?
  - Merem 5y ago
    I think it's just broken in some cases for whatever reason. Like 9 months ago I mentioned trying out Tik Tok for some weeks and despite lots of "not interested", likes etc, I got swamped by 99% garbage, including videos I specifically said I don't want to see.
  - wongarsu 5y ago
    TikTok has lots of content for nerds (though I think they prefer the term neurodivergent). But there is certainly a trend that TikTok content is comparatively shallow. For example Hank [1] and John Green [2] have quite nerdy content, but it's much shorter less developed thoughts than their youtube channel [3]. Or NileRed: his most recent TikTok[4] (eating capsaicin) is in no way comparable to his most recent YouTube[5] (synthesizing cotton candy from cotton balls). In terms of nerdy content it probably shines more with life advice than with science facts, simply because of the short video format. Not sure about the age group, most of TikTok is probably targeted at ages 12-40. 1: https://www.tiktok.com/@hankgreen1?lang=en https://www.tiktok.com/@hankgreen1?lang=en 2: https://www.tiktok.com/@literallyjohngreen?lang=en https://www.tiktok.com/@literallyjohngreen?lang=en 3: https://www.youtube.com/c/vlogbrothers https://www.youtube.com/c/vlogbrothers 4: https://www.tiktok.com/@nilered/video/7063145462794145030 https://www.tiktok.com/@nilered/video/7063145462794145030 5: https://www.youtube.com/watch?v=vHuFizITMdA https://www.youtube.com/watch?v=vHuFizITMdA

mrfusion 5y ago

I’m disappointed with the personalization. I was hoping it would teach me interesting things in short videos and expose me to new ideas. Instead after a few weeks it seems to mostly show me political stuff and pranks.

dakial1 5y ago

I started using TikTok and wanted to check how it would suggest me videos if I didn't like anything, as it would make it easier for it and I wouldn't be able to see how sophisticated it really is. So it obviously would start in a very broad content but already target at my demographics (male 30-40 y/o, which BTW gave a worrying amount of very young girls dancing in a sexualized way) but with time it would certainly adapt and be really spot on on the videos I liked (mainly stand up/comedy stuff). Everything based in signals other than liking of following. They obviously track all interactions from the user (time watching, repeats, shares, OP profile checks, etc) to make this targeting. Also its ML for visual recognition used to identify key aspects on the video (people, location, etc...) is very powerful as it would guess my favorite comedians and start exploring with other videos. Also they certainly use bandit/lookalike model there as it would recommend some wildly different videos for me. And I can bet that that last video it shows when you click to leave the app (on Android at least) is probably the highest rated recommendation for you. To really lock you in there.

bitshiftfaced 5y ago

I'd like to see them address videos that game the system. I'll see videos structured so that the beginning is very slow, and so you must wait to get to the meat of the content. People do wait, and so the video is algorithmically rewarded, but that doesn't mean the video is good. Some videos even have a "fake glitch" effect where the first few frames are from later in the video. It's as if to say "here's what the video's about, but first wait through this boring part so that my video gets rewarded for engagement." I once watched a video over several times because I couldn't make out what was supposed to be "off" about it at the end. I finally realized that there was nothing strange or anything, but the video led you to believe there was. Therefore many people, like me, stupidly rewatched it in order to figure out that there was nothing interesting. And that's why it had a boosted signal!

cormorant 5y ago

In Figure 6 (described in section 4.4), the content seen by "the control user 50" and "the active user 49" are both in large part by content creator "miakhalifa" (32.4% for the control, 68.6% for the active). The "experimental scenario" is "scenario 28", described as "Follow a random content creator". Mia Khalifa is a porn actress. A quick search indicates that her TikTok posts are at least sexually suggestive. I wish I understood what "control" meant in this paper, and what exactly this scenario tested. Does it really mean that a fresh user profile is going to see 1/3 of its feed coming from a porn star? I've never used TikTok so I don't know!