17 ms·

TikTok and the Sorting Hat

332 points by hardmaru 6y ago

bsoist 6y ago
Very interesting. We might make real progress as a species if we can leverage this more generally as a society. Also, I expect the "machines" to disrupt the film industry in a very similar way.
- lotsofpulp 6y ago
  How will showing people short video clips to create a drug like drip of enjoyable chemical reactions in the brain progress the species? > (and yes, one feed that contained the thirst trap photos of attractive Indian girls in rather suggestive outfits standing under things like waterfalls; some parts of culture are universal). Doesn’t sound like any version of progress that I would get excited about.
  - mistermann 6y ago
    > How will showing people short video clips to create a drug like drip of enjoyable chemical reactions in the brain progress the species? Possibly (pure speculation here): if you carefully observe, at massive scale, how a system [1] behaves, it may provide some insights into how it is implemented. And based on those learnings, iteratively improve how the platform feeds tests and analyzes responses to gain deeper understanding of the system over time. [1] In this case, "the system" consists of individual human minds, as well as the overall network of minds.
    - lotsofpulp 6y ago
      Is there any insight to gain than the already known fact that people will sacrifice long term interests for short term gratification?
      - mistermann 6y ago
        I'd bet fairly big $ that there are plenty of them.
  - zerowangtwo 6y ago
    What's the long term end goal for society? I think one feature of such a society would be unlimited entertainment available for anyone, which TikTok provides.
    - Ericson2314 6y ago
      I think with most work automated away humans should mostly entertain themselves in person. Art should be increasing interactive as everyone has time to level up beyond passive audience.
- skinnymuch 6y ago
  What progress as a species can be made? I am not seeing how any of this is progress.
codekansas 6y ago
Some really good points. Does anyone have a good explanation for why Chinese startups seem to have mastered build AI-centric apps while American startups (and large companies) lag behind? Edit: Want to rephrase this to avoid promoting nationalism. Why has ByteDance been so successful? TikTok / Douyin and Toutiao are both really big hits, Facebook hasn't had a comparable ground-up hit despite having top-notch ML engineers.
- dillonmckay 6y ago
  Labor costs?
  - logicchains 6y ago
    ByteDance, the company behind TikTok, pays more for AI developers than most western companies: https://crm.org/articles/3-million-pay-packages-how-a-chinese-startup-is-attracting-top-talent https://crm.org/articles/3-million-pay-packages-how-a-chines....
  - codekansas 6y ago
    Speaking entirely anecdotally, I suspect having a bunch of people working on an ML algorithm actually does more harm than good. It's more important that everyone in the product stack is on the same page about the point. The Facebook Newsfeed UI, for example, has to balance so many different signals. Compare that with TikTok's UI, which is very efficient at collecting a clear reinforcement signal and only has to deal with one homogenous kind of data. It's like the entire TikTok team, not just the data scientists, focused in on doing ML really well. I'm really impressed and kind of scared whenever I use TikTok. It really does feel like a whole different level of ML-powered addictiveness, designed from the ground up rather than having the ML part added on.
- dumb1224 6y ago
  Which other chinese startups with machine-learning as its core? Do you mind listing some examples?
  - 6y ago
chvid 6y ago
TikTok really is next-generation social media. Interesting that this should come out of China and not the west. Maybe we are seeing the effect of monopolies stifling innovation.
- rollinggoron 6y ago
  Isn't this very similar to Vine, the old video social network? Which Twitter bought and properly crash a burned?
  - actuator 6y ago
    I think Vine was follower based still. I haven't used Tiktok but based on the article it seems like Vine+YouTube feed.
  - mikepurvis 6y ago
    Similar in the "posting short videos" thing, but some of the social features are pretty different from anything that's come before, like the ability to easily make a new lipsync video with the audio from a previous video, and then make all videos sharing an audio trivially searchable. Am not myself a TikTok user, but my partner is, so I've seen a bunch of it second hand.
  - chvid 6y ago
    The point is not media format (short video clips) but the "sorting hat". That it breaks the traditional social media model with a news feed, discussions, a social graph of friends. I think it is completely meaningful to call it next generation (or second generation where the social graph model is the first generation). Also notice that this model can be applied to other media formats: Text, pictures, audio ...
    - gruez 6y ago
      >That it breaks the traditional social media model with a news feed, discussions, a social graph of friends. don't most social networks have the "sorting hat" in the form of algorithmic feeds?
taylorhou 6y ago
Fantastic read. Time for Ben of Stratechery to pass the baton :p - I'm assuming Eugene is now going to get a gazillion messages of employment or consulting requests at the aforementioned FAANG companies to help steer the direction of their tiktok ambitions.
nnao45 6y ago
hahaha
kangnkodos 6y ago
The article didn't have many details about the "For You Page feed algorithm". Any guesses as to how it analyzes a video, and which properties it looks at?
- cycrutchfield 6y ago
  Latent visual features are likely a strong component. I imagine if you created a new account and started watching content with only blond people in it, the ranking algorithm would pretty quickly start showing you only blond people.
- whlr 6y ago
  Don’t know much about this, but I would have thought TikTok relies on the watch patterns of its viewers, rather than direct video analysis.
- aabhay 6y ago
  Yeah I'm imagining a mix of facial features, audio/music metadata, and some kind of collaborative filtering between viewers and creators. The vast majority of feeds use at least geography based personalization to give you content in a familiar language, too.
- Firebrand 6y ago
  This gives a general idea of how the algorithm works: https://www.veed.io/grow/reverse-engineering-how-tiktok-algorithm-works/ https://www.veed.io/grow/reverse-engineering-how-tiktok-algo...
Firebrand 6y ago
>At the time Musical.ly got renamed TikTok, it was still dominated by teen girls doing lip synch videos. Many U.S. teens at the time described TikTok as “cringey,” usually a kiss of death for networks looking to expand among youths, fickle as they are about what’s cool. Scrolling the app at the time felt like eavesdropping on the theater kids clique from high school. Entertaining, but hardly a mainstream entertainment staple. TikTok devs mentioned PewDiePie reacting to these cringey videos as a large impact for the company. His massive gamer audience downloaded the app to make fun of the early TikTok users too and got sucked in. They started making memes instead of lip-sync videos and it took off from there.
- ipsum2 6y ago
  Interestingly, the subreddit r/TiktokCringe is the same - was made to make fun of TikTok, but now highlights Redditors favorite videos.
- tmpz22 6y ago
  From the outside looking in a lot of their success came from sexualizing (underage) teenage girls. Don't know if that would've flown as well if the app had come out of Facebook or a different American tech giant.
  - deleted 6y ago
    [deleted]
  - neixidbeksoxyd 6y ago
    Is there a way a social app targeted at young people in the US doesn't end up looking as if their success comes from sexualizing girls? I don't use any social media, but Facebook, Instagram, and Snapchat also seemed to me like that in their early stages. Maybe Whatsapp is an exception, and that might be why they largely failed here but took off elsewhere.
    - tmpz22 6y ago
      I think the difference maker is that new social media apps like TikTok are artificially promoting this content from day zero via algorithms as opposed to Facebook and instagram which started by showing that content only from within your own network or through your own intentional curation. Now the second they identify a male you get a discover page full of bikini pictures and women bending over in yoga pants whether you want it or not. And of course this brings all the teenagers to the yard, and older predatory males with deeper pockets... “we’d love to have you model for us!” comments all over the place the second a pretty girl puts her profile to public.
xivzgrev 6y ago
this doesn’t surprise me at all. I work at a consumer company and we use neural networks to personalize feeds. Thousands of features feed in and each member gets a different feed. It doesn’t take much / long to personalize. What I am surprised by is that YouTube or others dont use as effectively. Neural networks are very well known. or, is there something “next level” that TikTok is using that isn’t well known? Also worth calling out that Netflix also used to use personalized recommendation engines, but eventually found that “top in the US” won out, which I found fascinating. If one person loves action movies for example, wouldn’t they find a list of those more appealing than a generic list of top 10? I sometimes get curious about top 10 but rarely actually watch them myself
- ry_co 6y ago
  My gut instinct is that Netflix can't use the personalized algorithmic approach that TikTok succeeds with because Netflix has to pay dearly for its content, where TikTok does not. Because of the cost structure of licensing, Netflix cannot afford to have the diversity of content that TikTok's user-generated approach enables. A small content library makes it impossible to build a meaningfully personalized feed.
  - basch 6y ago
    It's also much harder to tell why somebody liked a 2h movie, vs a 15 second clip. The 15 second clip only has so many properties, where as there's millions of reasons I might like one movie but not a similar one. Also, as far as Netflix, id rather see more diversity, and not see the same movie made over and over again. I would wager TikTok viewers are more willing to watch repetitive things play out.
    - beaunative 6y ago
      This is not as difficult to understand as it might sound, Netflix can only run recommendation based on the series its audience has watched, and you can only watch so many series in a week. Whereas for 15 seconds clips, the engine get feed hundred if not thousands times more data it would on netflix data. so
082349872349872 6y ago
How much sorting needs to be done? I'd guess fewer than 20 questions worth.
gazelleeatslion 6y ago
Modern day tobacco. 100% unregulated addiction. FB, TWTR, GOOG, INSTA, everyone switched from timeline to personal recommendation engines and TikTok is just doing it so much better. Where will things be in 5, 10, 50 years? Terrifying and exciting. Do these tech companies sit on research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Will they all have their tobacco-causes-cancer moment? Old-Age Fear: YouTube algorithm sending people down the conspiracy rabbit hole New-Age Fear (Hyper-Efficient): TikTok secretly radicalizing individual in X? The big question strikes: "what is free will?" Amazing read
- catalogia 6y ago
  > Do these tech companies sit on research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Facebook was running mood-manipulation experiments years ago and thought it was worth bragging about publicly. https://slate.com/technology/2014/06/facebook-unethical-experiment-it-made-news-feeds-happier-or-sadder-to-manipulate-peoples-emotions.html https://slate.com/technology/2014/06/facebook-unethical-expe...
  - canjobear 6y ago
    They found extremely small effects.
- sandworm101 6y ago
  >> research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Cambridge Analytica certainly had that research. Media campaigns are no longer about "getting the word out", projecting an idea and hoping it sticks. Modern campaigns are about changing a narrative, moving individuals from one opinion to another through whatever means works best for that individual. I cannot see how Facebook wouldn't be studying this.
  - fsociety 6y ago
    Cambridge Analytica was both snake-oil and a wake up call. A sign of where things could go if we don’t smarten up soon.
smaddali 6y ago
Eugene mentions that the algorithm is the one that built this latent interest graph and this algorithm itself is worth a lot of money. Given that most personalization is using ML/Deep Learning and various video features like scene descriptions, object identification in each scene etc, Is it really that proprietary to warrant this level of premium. Once the awareness of interest graph could be built to this level of efficiency using usual techniques, Is the algorithm and approach exclusive to TikTok ? Awesome Insights from Eugene !!
melonkidney 6y ago
To pick up on one point from the (fascinating) article: > In the other direction, the U.S. hasn’t made a huge dent in China. Obviously, the Great Firewall played a huge role in keeping a lot of U.S. companies out of the Chinese market, but in the few cases where a U.S. company got a crack at the Chinese market, like Uber China, the results were mixed. I'm genuinely curious to know which non-Chinese apps/services have managed to establish themselves in China. I feel like an argument could be made for LinkedIn... Are there obvious ones I'm missing?
- bigpumpkin 6y ago
  Paul Graham's greatest hit: AirBnB
- est 6y ago
  microsoft, bing, azure. skype, linkedin, z.cn, kindle, aws china, zoom, android, wework.
angel_j 6y ago
The simpler answer is TikTok paid an ample Chinese labor force to watch and heart U.S. videos. This gives the users a nice dopamine rush and brings them back. It was obvious by the number of likes and comments very dumb videos would have.
- greeniron 6y ago
  that's just plain wrong. if you even used tiktok, you'll understand how truly frighteningly powerful their algorithm is at finding exactly the kind of content you want to watch. i was never the kind of person to scroll my phone right before sleeping, but somehow i found myself doing that these days on tiktok, mindlessly just swiping up over and over again, not able to stop because almost every video seems to appeal precisely to my taste, even in the weird humor stuff.
  - angel_j 6y ago
    Wow I get downvoted to make room for "it's magic".
- balola 6y ago
  Chinese who master both English language and US culture are quite rare in China.
  - angel_j 6y ago
    TikTok already admitted they filtered out "poor and ugly" people. It's not hard to for a laborer to like a video and pick a (suggested) response.
- zerowangtwo 6y ago
  Maybe you just don't understand the particular brands of irony/satire or the current trends going on at the time those videos you've seen were made?
yamrzou 6y ago
For all the naive and idealistic dreams of the so-called “marketplace of ideas,” the first generation of large social networks has proven mostly unprepared and ill-equipped to deal with the resulting culture wars. Until they have some real substantial ideas and incentives to take on the costly task of mediating between strangers who disagree with each other, they’re better off sorting those people apart. The only types of people who enjoy being thrown into a gladiatorial online arena together with those they disagree with seem to be trolls, who benefit asymmetrically from the resultant violence. This seems to contradict what has been said about Facebook exploiting users's attraction to divisiveness and polarization. See [1] and [2]. [1] https://news.ycombinator.com/item?id=23313007 https://news.ycombinator.com/item?id=23313007 [2] https://news.ycombinator.com/item?id=23314507 https://news.ycombinator.com/item?id=23314507
- Nacraile 6y ago
  I'm not sure it's as much of a contradiction as you think it is. Noting "It manifests itself in the declining visit and posting frequency on Facebook across many cohorts", I find it entirely plausible that feeding the trolls yielded short term lift in target metrics at the expense of (much harder to measure and correctly attribute) long-term attrition of non-troll users. That hypothesis certainly fits the popular perception that FB is dying/dead. It somewhat fits my personal experience: I fairly aggressively pruned my news feed of any political content, which seems to have kept away the trolling, but the personal content that I wanted has dried up and my feed is now mostly ads and generic "recommended" clickbait. Why bother visiting?
WhompingWindows 6y ago
Is anyone else here trying to reduce their use of technology outside of the workplace? My hobbies are music, painting, and woodworking, and it's all I can muster to focus on a screen from 9-5. I do comment here on HN and I peruse other websites, but I don't actively engage with any social media except HN. I just find myself questioning the merits of attention spent on these social media platforms, given the information quality and density is so much lower than HN. Maybe this is where TikTok has succeeded: short, snappy, quickly accessed videos; much more informational content than textual or image-based FB or Reddit, not to mention more skillfully executed, right? It's like a short-form, Twitter-ified entertainment version of YT.
- bdickason 6y ago
  I’ve been getting back into electronic music production and there’s a growing group of people who previously used ableton, logic, etc that are moving to pure hardware production (it’s called Dawless) to reduce screen time. Many are software engineers or work a day job at a tech company. Interesting trends to keep an eye on.
- zozin 6y ago
  All of these apps are just a gigantic waste of time, and proven to be highly manipulative and emotionally draining. I've managed to avoid all of them, including FB which I only keep to message family members and friends. The fact that normally astute, rational, intelligent individuals find themselves mindlessly clicking through games or video apps goes to show how addictive and manipulative these apps are. Some of my friends realize this, but they still waste time on them as if they cannot stop. It's sad. The top comment is correct, it's a modern/high-tech addictive substance.
  - Avtomatk 6y ago
    Those applications will be a great waste of time, but they are there to solve the problem behind all social media: loneliness and seeking care. You can distract your mind with other things for a while, but sooner or later you will fall back on social networks, unless you stop being alone or have enough attention. I think that living beings do not know how to survive alone, so being alone is something bad biologically and arouses something in us that makes us feel bad ... call it "loneliness" or whatever you want, but that thing kills you if you stay alone
praveen9920 6y ago
Nice read. I guess TikTok has taken the quote "users don't know what they want" to next level. FB, YT, Twitter, Instagram are trying to use the labels/tags/likes/network provided by user while TikTok is using user's interactions to categorize user and recommend the content which the user himself didn't even knew he/she liked.
creeble 6y ago
The article mentions that TikTok owes a lot of its early traction to its predecessor, musical.ly. but there's no mention of how/whether musical.ly got license for their lip sync music. Was the success of musical.ly (and by some large extension, TikTok) based on the difficult-to-stop (because China) pirating of music? I now unleash the piracy-is-not-theft dogs...
- bigpumpkin 6y ago
  They used licensed music
  - creeble 6y ago
    Musical.ly did? How did they negotiate that license, I wonder? Let me put it another way then: Is the success of musical.ly and its successor TikTok really about somehow getting licensed music on a platform out the gate? And the sorting hat is just a good refinement that wouldn't have a chance to exist without the licensing?
reedwolf 6y ago
"On my way in and out of this office, just one of several Bytedance spaces all across the city, I gawked at hundreds of workers sitting side by side in row after row in the open floorplan. It resembled what I’d seen at tech giants like Facebook in the U.S., but even denser." -- As much as US devs whine about open offices, it could be worse.
- draw_down 6y ago
  Oh, things can always be worse. But it's probably not a great idea to wait until they can't get any worse, before you speak up.
- Cactus2018 6y ago
  Found photos of Bytedance's open office: empty: https://officesnapshots.com/2016/07/13/bytedance-offices-beijing/ https://officesnapshots.com/2016/07/13/bytedance-offices-bei... full of staff: https://www.caixinglobal.com/2020-04-22/bytedance-drops-paywall-for-novels-on-its-toutiao-app-101545886.html https://www.caixinglobal.com/2020-04-22/bytedance-drops-payw... with a faux ceiling: https://www.livemint.com/companies/start-ups/tiktok-parent-eyeing-content-startup-deals-in-big-push-into-india-1563301051925.html https://www.livemint.com/companies/start-ups/tiktok-parent-e...
  - kakkun 6y ago
    What's the purpose of the faux ceiling? I would have hoped the large expansive space and glass ceiling would be used to help alleviate feelings of claustrophobia.
  - stevesearer 6y ago
    I run Office Snapshots (first link) and hadn't seen the faux ceiling you linked to -- definitely interesting. @kakkun My guess is that it would be limit direct sunlight, though if that were the case, a shade higher up might look better. I do remember someone telling me that if you go into a Costco food court people often gravitate to the tables with an umbrella on them even though they are indoors. Their theory was that people do like to have something over their heads in such a big warehouse-size space.
mgraczyk 6y ago
> How did an app designed by two guys in Shanghai managed to run circles around U.S. video apps Part of the explanation is that they weren't in Shanghai. The founders both worked full time at US tech companies in California when musical.ly was founded. musical.ly may have been founded while they were temporarily in Shanghai, but the founders were not some random people who didn't understand US culture. This is important to the article's thesis. At least part of the early insight into the US market comes from the founders living and working in the US.
- wenbin 6y ago
  The musicaly people were already veterans in Silicon Valley tech scene (not young under silicon valley standard & with years of working experience). They raised money to build education product with short videos in it. The education product was not doing well. So they cut everything and kept the short video part. Sounds familiar? YouTube was initially a dating website with videos. Then they cut the dating part and kept videos. Instagram was initially a checkin app with photos. Then they cut the checkin part and kept photos.
  - jehlakj 6y ago
    For a second I thought you were going to mention vine.
    - dilyevsky 6y ago
      With vine twitter went one step further and got rid of video part too
  - egypturnash 6y ago
    Flickr: initially an image management/sharing tool for a 2d online game. Cut the game, kept the image sharing. Slack: initially an internal communication tool for another incarnation of the same game. Cut the game, kept the chat. (An incarnation of this game did briefly exist between these two, but only lasted about a year.)
preommr 6y ago
Instead of being amazed that TikTok beat juggernauts, maybe it should be reframed that they beat companies that have terrible products. Sine we're talking about niches and customized feeds, let's look at reddit. My god is the reddit app absolutely horrendous. It's so bad I use the web app in desktop mode because other forms have terrible UX. They just don't care. Or let's talk about another site that was mentioned: youtube. It's like Google with all their money, doesn't really care to spend real money on trying to solve problems. If they dumped something like 100-200 million into better community outreach, and actual going in and tagging videos, and dealing with content, the service would be so much better. Instead, they made their money on search, and now it's spinning it's wheels trying to find clever algorithms to solve problems. We knew there was a market for an app that lets you easily create, edit and upload videos. Vine proved the formula worked. TikTok stepped in and filled the void.
tmabraham 6y ago
Very interesting read about why TikTok is succeeding. It makes me more curious about how the TikTok curation algorithm works! I appreciate the comparison of TikTok to other social networks like Facebook and Twitter, but I am surprised there is no discussion about Reddit? Reddit is interesting in that people have to follow topics/communities in the form of subreddits, so it only works if the user puts in the effort to find communities to subscribe to. Maybe Reddit could also benefit from a better curation algorithm.
peter303 6y ago
Interesting article in NYTimes a few minutes ago that several Trump advisers had to talk him into the MicroSoft compromise just before he was to sign the banning order. The advisors were horrified they'd lose an angry generation of new voters. In a sense this now a win-win for the administration: they punish China a little bit and dont overly alienate young voters.
titanomachy 6y ago
I installed TikTok out of curiosity after reading this article. I scrolled without stopping for a couple hours and then uninstalled the app. An endless stream of algorithmically curated 30-second videos is pure crack to my brain, apparently. I bet a lot of younger folks with still-developing brains are also unable to resist.
- ontifica 6y ago
  I'm on the younger side, it's very addictive. People love to come up with wild theories for why the app blew up, but I think it's just pure algorithmically curated fun.
- intended 6y ago
  Think of it this way - how long can this network exist without having to deal with political speech ? And the moment people figure out how to optimize for dispersion through the tik too algo, what would be the result ?
- Noumenon72 6y ago
  There's a classical conditioning loop where you see an attractive woman dancing to a song, which makes you subconsciously start to like the song, then you like videos with that song... it's the combined power of earworms and arousal. If they somehow make it taste like cheesecake we will have achieved full wireheading.
  - newen 6y ago
    Smell-o-vision. The next revolution in smartphone technology!
- Yhippa 6y ago
  I'm an adult and normally don't get got by these types of apps but I found myself doing the same. I had to uninstall.
corford 6y ago
Very insightful read. Loved the part describing TikTok's algorithm as essentially a zero friction, global 'market maker' for content producers and consumers. Really clicked with me.
intended 6y ago
Does this dodge the problem of political speech ?
m3kw9 6y ago
Am I the only one not hooked on TikTok? It’s just a lot of good looking people repeating similar jokes. Maybe their algorithm haven’t figured me out yet.
auganov 6y ago
> But in the reverse direction, America has been almost as impenetrable to Chinese companies because of what might be though of as America’s cultural firewall Why would they even try? You've got a nice big domestic tech ecosystem which would not have existed without protectionism. Most of these products aren't competitive at all. Nothing to do with culture, same reason Canada, UK etc aren't known for big tech successes. Musical.ly is unusual in that they did actually work on an original idea and decided to launch the product in America where it took off first. Ended up not playing the China game early on. > With it, a massive team made up mostly by people who’ve never left China, and many who never will, grabbed massive marketshare in cultures and markets they’d never experienced firsthand. To a cultural determinist like myself, that feels like black magic. Pretty much every big American tech company does that. Nothing notable about this. > I don’t think the Chinese product teams I’ve met in recent years in China are much further ahead than the ones I met in 2011 when it comes to understanding foreign cultures like America. But what the Bytedance algorithm did was it abstracted that problem away. Understanding in what way? You work on a product and you optimize it for your users. If you work in a 100% domestically focused ecosystem obviously you'll design your products a little differently. What happened here is a Chinese company was for a change actually working on a product with a global userbase and they had to make them happy. The whole thing is borderline insulting to the founders.
webninja 6y ago
The author of this article mentions that the TikTok algorithm alone is worth 30B. One TikTok user claims to have figured it out. 1 rewatch = 6 points 1 completion = 5 points Share = 3 points Comments = 2 points Like = 1 point Videos with the most points above a minimum threshold are shown to an increasingly wider audience. Full video: https://vm.tiktok.com/Jjv2cXC/ https://vm.tiktok.com/Jjv2cXC/

332 points by hardmaru 6y ago

Very interesting. We might make real progress as a species if we can leverage this more generally as a society. Also, I expect the "machines" to disrupt the film industry in a very similar way.

lotsofpulp 6y ago
How will showing people short video clips to create a drug like drip of enjoyable chemical reactions in the brain progress the species? > (and yes, one feed that contained the thirst trap photos of attractive Indian girls in rather suggestive outfits standing under things like waterfalls; some parts of culture are universal). Doesn’t sound like any version of progress that I would get excited about.
- mistermann 6y ago
  > How will showing people short video clips to create a drug like drip of enjoyable chemical reactions in the brain progress the species? Possibly (pure speculation here): if you carefully observe, at massive scale, how a system [1] behaves, it may provide some insights into how it is implemented. And based on those learnings, iteratively improve how the platform feeds tests and analyzes responses to gain deeper understanding of the system over time. [1] In this case, "the system" consists of individual human minds, as well as the overall network of minds.
  - lotsofpulp 6y ago
    Is there any insight to gain than the already known fact that people will sacrifice long term interests for short term gratification?
    - mistermann 6y ago
      I'd bet fairly big $ that there are plenty of them.
- zerowangtwo 6y ago
  What's the long term end goal for society? I think one feature of such a society would be unlimited entertainment available for anyone, which TikTok provides.
  - Ericson2314 6y ago
    I think with most work automated away humans should mostly entertain themselves in person. Art should be increasing interactive as everyone has time to level up beyond passive audience.
skinnymuch 6y ago
What progress as a species can be made? I am not seeing how any of this is progress.

codekansas 6y ago

Some really good points. Does anyone have a good explanation for why Chinese startups seem to have mastered build AI-centric apps while American startups (and large companies) lag behind? Edit: Want to rephrase this to avoid promoting nationalism. Why has ByteDance been so successful? TikTok / Douyin and Toutiao are both really big hits, Facebook hasn't had a comparable ground-up hit despite having top-notch ML engineers.

dillonmckay 6y ago
Labor costs?
- logicchains 6y ago
  ByteDance, the company behind TikTok, pays more for AI developers than most western companies: https://crm.org/articles/3-million-pay-packages-how-a-chinese-startup-is-attracting-top-talent https://crm.org/articles/3-million-pay-packages-how-a-chines....
- codekansas 6y ago
  Speaking entirely anecdotally, I suspect having a bunch of people working on an ML algorithm actually does more harm than good. It's more important that everyone in the product stack is on the same page about the point. The Facebook Newsfeed UI, for example, has to balance so many different signals. Compare that with TikTok's UI, which is very efficient at collecting a clear reinforcement signal and only has to deal with one homogenous kind of data. It's like the entire TikTok team, not just the data scientists, focused in on doing ML really well. I'm really impressed and kind of scared whenever I use TikTok. It really does feel like a whole different level of ML-powered addictiveness, designed from the ground up rather than having the ML part added on.
dumb1224 6y ago
Which other chinese startups with machine-learning as its core? Do you mind listing some examples?
- 6y ago

chvid 6y ago

TikTok really is next-generation social media. Interesting that this should come out of China and not the west. Maybe we are seeing the effect of monopolies stifling innovation.

rollinggoron 6y ago
Isn't this very similar to Vine, the old video social network? Which Twitter bought and properly crash a burned?
- actuator 6y ago
  I think Vine was follower based still. I haven't used Tiktok but based on the article it seems like Vine+YouTube feed.
- mikepurvis 6y ago
  Similar in the "posting short videos" thing, but some of the social features are pretty different from anything that's come before, like the ability to easily make a new lipsync video with the audio from a previous video, and then make all videos sharing an audio trivially searchable. Am not myself a TikTok user, but my partner is, so I've seen a bunch of it second hand.
- chvid 6y ago
  The point is not media format (short video clips) but the "sorting hat". That it breaks the traditional social media model with a news feed, discussions, a social graph of friends. I think it is completely meaningful to call it next generation (or second generation where the social graph model is the first generation). Also notice that this model can be applied to other media formats: Text, pictures, audio ...
  - gruez 6y ago
    >That it breaks the traditional social media model with a news feed, discussions, a social graph of friends. don't most social networks have the "sorting hat" in the form of algorithmic feeds?

taylorhou 6y ago

Fantastic read. Time for Ben of Stratechery to pass the baton :p - I'm assuming Eugene is now going to get a gazillion messages of employment or consulting requests at the aforementioned FAANG companies to help steer the direction of their tiktok ambitions.

nnao45 6y ago

hahaha

kangnkodos 6y ago

The article didn't have many details about the "For You Page feed algorithm". Any guesses as to how it analyzes a video, and which properties it looks at?

cycrutchfield 6y ago
Latent visual features are likely a strong component. I imagine if you created a new account and started watching content with only blond people in it, the ranking algorithm would pretty quickly start showing you only blond people.
whlr 6y ago
Don’t know much about this, but I would have thought TikTok relies on the watch patterns of its viewers, rather than direct video analysis.
aabhay 6y ago
Yeah I'm imagining a mix of facial features, audio/music metadata, and some kind of collaborative filtering between viewers and creators. The vast majority of feeds use at least geography based personalization to give you content in a familiar language, too.
Firebrand 6y ago
This gives a general idea of how the algorithm works: https://www.veed.io/grow/reverse-engineering-how-tiktok-algorithm-works/ https://www.veed.io/grow/reverse-engineering-how-tiktok-algo...

Firebrand 6y ago

>At the time Musical.ly got renamed TikTok, it was still dominated by teen girls doing lip synch videos. Many U.S. teens at the time described TikTok as “cringey,” usually a kiss of death for networks looking to expand among youths, fickle as they are about what’s cool. Scrolling the app at the time felt like eavesdropping on the theater kids clique from high school. Entertaining, but hardly a mainstream entertainment staple. TikTok devs mentioned PewDiePie reacting to these cringey videos as a large impact for the company. His massive gamer audience downloaded the app to make fun of the early TikTok users too and got sucked in. They started making memes instead of lip-sync videos and it took off from there.

ipsum2 6y ago
Interestingly, the subreddit r/TiktokCringe is the same - was made to make fun of TikTok, but now highlights Redditors favorite videos.
tmpz22 6y ago
From the outside looking in a lot of their success came from sexualizing (underage) teenage girls. Don't know if that would've flown as well if the app had come out of Facebook or a different American tech giant.
- deleted 6y ago
  [deleted]
- neixidbeksoxyd 6y ago
  Is there a way a social app targeted at young people in the US doesn't end up looking as if their success comes from sexualizing girls? I don't use any social media, but Facebook, Instagram, and Snapchat also seemed to me like that in their early stages. Maybe Whatsapp is an exception, and that might be why they largely failed here but took off elsewhere.
  - tmpz22 6y ago
    I think the difference maker is that new social media apps like TikTok are artificially promoting this content from day zero via algorithms as opposed to Facebook and instagram which started by showing that content only from within your own network or through your own intentional curation. Now the second they identify a male you get a discover page full of bikini pictures and women bending over in yoga pants whether you want it or not. And of course this brings all the teenagers to the yard, and older predatory males with deeper pockets... “we’d love to have you model for us!” comments all over the place the second a pretty girl puts her profile to public.

xivzgrev 6y ago

this doesn’t surprise me at all. I work at a consumer company and we use neural networks to personalize feeds. Thousands of features feed in and each member gets a different feed. It doesn’t take much / long to personalize. What I am surprised by is that YouTube or others dont use as effectively. Neural networks are very well known. or, is there something “next level” that TikTok is using that isn’t well known? Also worth calling out that Netflix also used to use personalized recommendation engines, but eventually found that “top in the US” won out, which I found fascinating. If one person loves action movies for example, wouldn’t they find a list of those more appealing than a generic list of top 10? I sometimes get curious about top 10 but rarely actually watch them myself

ry_co 6y ago
My gut instinct is that Netflix can't use the personalized algorithmic approach that TikTok succeeds with because Netflix has to pay dearly for its content, where TikTok does not. Because of the cost structure of licensing, Netflix cannot afford to have the diversity of content that TikTok's user-generated approach enables. A small content library makes it impossible to build a meaningfully personalized feed.
- basch 6y ago
  It's also much harder to tell why somebody liked a 2h movie, vs a 15 second clip. The 15 second clip only has so many properties, where as there's millions of reasons I might like one movie but not a similar one. Also, as far as Netflix, id rather see more diversity, and not see the same movie made over and over again. I would wager TikTok viewers are more willing to watch repetitive things play out.
  - beaunative 6y ago
    This is not as difficult to understand as it might sound, Netflix can only run recommendation based on the series its audience has watched, and you can only watch so many series in a week. Whereas for 15 seconds clips, the engine get feed hundred if not thousands times more data it would on netflix data. so

082349872349872 6y ago

How much sorting needs to be done? I'd guess fewer than 20 questions worth.

gazelleeatslion 6y ago

Modern day tobacco. 100% unregulated addiction. FB, TWTR, GOOG, INSTA, everyone switched from timeline to personal recommendation engines and TikTok is just doing it so much better. Where will things be in 5, 10, 50 years? Terrifying and exciting. Do these tech companies sit on research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Will they all have their tobacco-causes-cancer moment? Old-Age Fear: YouTube algorithm sending people down the conspiracy rabbit hole New-Age Fear (Hyper-Efficient): TikTok secretly radicalizing individual in X? The big question strikes: "what is free will?" Amazing read

catalogia 6y ago
> Do these tech companies sit on research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Facebook was running mood-manipulation experiments years ago and thought it was worth bragging about publicly. https://slate.com/technology/2014/06/facebook-unethical-experiment-it-made-news-feeds-happier-or-sadder-to-manipulate-peoples-emotions.html https://slate.com/technology/2014/06/facebook-unethical-expe...
- canjobear 6y ago
  They found extremely small effects.
sandworm101 6y ago
>> research to suggest that these algorithms can induce behavior-changing tactics through trial-and-error? Cambridge Analytica certainly had that research. Media campaigns are no longer about "getting the word out", projecting an idea and hoping it sticks. Modern campaigns are about changing a narrative, moving individuals from one opinion to another through whatever means works best for that individual. I cannot see how Facebook wouldn't be studying this.
- fsociety 6y ago
  Cambridge Analytica was both snake-oil and a wake up call. A sign of where things could go if we don’t smarten up soon.

smaddali 6y ago

Eugene mentions that the algorithm is the one that built this latent interest graph and this algorithm itself is worth a lot of money. Given that most personalization is using ML/Deep Learning and various video features like scene descriptions, object identification in each scene etc, Is it really that proprietary to warrant this level of premium. Once the awareness of interest graph could be built to this level of efficiency using usual techniques, Is the algorithm and approach exclusive to TikTok ? Awesome Insights from Eugene !!

melonkidney 6y ago

To pick up on one point from the (fascinating) article: > In the other direction, the U.S. hasn’t made a huge dent in China. Obviously, the Great Firewall played a huge role in keeping a lot of U.S. companies out of the Chinese market, but in the few cases where a U.S. company got a crack at the Chinese market, like Uber China, the results were mixed. I'm genuinely curious to know which non-Chinese apps/services have managed to establish themselves in China. I feel like an argument could be made for LinkedIn... Are there obvious ones I'm missing?

bigpumpkin 6y ago
Paul Graham's greatest hit: AirBnB
est 6y ago
microsoft, bing, azure. skype, linkedin, z.cn, kindle, aws china, zoom, android, wework.

angel_j 6y ago

The simpler answer is TikTok paid an ample Chinese labor force to watch and heart U.S. videos. This gives the users a nice dopamine rush and brings them back. It was obvious by the number of likes and comments very dumb videos would have.

greeniron 6y ago
that's just plain wrong. if you even used tiktok, you'll understand how truly frighteningly powerful their algorithm is at finding exactly the kind of content you want to watch. i was never the kind of person to scroll my phone right before sleeping, but somehow i found myself doing that these days on tiktok, mindlessly just swiping up over and over again, not able to stop because almost every video seems to appeal precisely to my taste, even in the weird humor stuff.
- angel_j 6y ago
  Wow I get downvoted to make room for "it's magic".
balola 6y ago
Chinese who master both English language and US culture are quite rare in China.
- angel_j 6y ago
  TikTok already admitted they filtered out "poor and ugly" people. It's not hard to for a laborer to like a video and pick a (suggested) response.
zerowangtwo 6y ago
Maybe you just don't understand the particular brands of irony/satire or the current trends going on at the time those videos you've seen were made?

yamrzou 6y ago

For all the naive and idealistic dreams of the so-called “marketplace of ideas,” the first generation of large social networks has proven mostly unprepared and ill-equipped to deal with the resulting culture wars. Until they have some real substantial ideas and incentives to take on the costly task of mediating between strangers who disagree with each other, they’re better off sorting those people apart. The only types of people who enjoy being thrown into a gladiatorial online arena together with those they disagree with seem to be trolls, who benefit asymmetrically from the resultant violence. This seems to contradict what has been said about Facebook exploiting users's attraction to divisiveness and polarization. See [1] and [2]. [1] https://news.ycombinator.com/item?id=23313007 https://news.ycombinator.com/item?id=23313007 [2] https://news.ycombinator.com/item?id=23314507 https://news.ycombinator.com/item?id=23314507

Nacraile 6y ago
I'm not sure it's as much of a contradiction as you think it is. Noting "It manifests itself in the declining visit and posting frequency on Facebook across many cohorts", I find it entirely plausible that feeding the trolls yielded short term lift in target metrics at the expense of (much harder to measure and correctly attribute) long-term attrition of non-troll users. That hypothesis certainly fits the popular perception that FB is dying/dead. It somewhat fits my personal experience: I fairly aggressively pruned my news feed of any political content, which seems to have kept away the trolling, but the personal content that I wanted has dried up and my feed is now mostly ads and generic "recommended" clickbait. Why bother visiting?

WhompingWindows 6y ago

Is anyone else here trying to reduce their use of technology outside of the workplace? My hobbies are music, painting, and woodworking, and it's all I can muster to focus on a screen from 9-5. I do comment here on HN and I peruse other websites, but I don't actively engage with any social media except HN. I just find myself questioning the merits of attention spent on these social media platforms, given the information quality and density is so much lower than HN. Maybe this is where TikTok has succeeded: short, snappy, quickly accessed videos; much more informational content than textual or image-based FB or Reddit, not to mention more skillfully executed, right? It's like a short-form, Twitter-ified entertainment version of YT.

bdickason 6y ago
I’ve been getting back into electronic music production and there’s a growing group of people who previously used ableton, logic, etc that are moving to pure hardware production (it’s called Dawless) to reduce screen time. Many are software engineers or work a day job at a tech company. Interesting trends to keep an eye on.
zozin 6y ago
All of these apps are just a gigantic waste of time, and proven to be highly manipulative and emotionally draining. I've managed to avoid all of them, including FB which I only keep to message family members and friends. The fact that normally astute, rational, intelligent individuals find themselves mindlessly clicking through games or video apps goes to show how addictive and manipulative these apps are. Some of my friends realize this, but they still waste time on them as if they cannot stop. It's sad. The top comment is correct, it's a modern/high-tech addictive substance.
- Avtomatk 6y ago
  Those applications will be a great waste of time, but they are there to solve the problem behind all social media: loneliness and seeking care. You can distract your mind with other things for a while, but sooner or later you will fall back on social networks, unless you stop being alone or have enough attention. I think that living beings do not know how to survive alone, so being alone is something bad biologically and arouses something in us that makes us feel bad ... call it "loneliness" or whatever you want, but that thing kills you if you stay alone

praveen9920 6y ago

Nice read. I guess TikTok has taken the quote "users don't know what they want" to next level. FB, YT, Twitter, Instagram are trying to use the labels/tags/likes/network provided by user while TikTok is using user's interactions to categorize user and recommend the content which the user himself didn't even knew he/she liked.

creeble 6y ago

The article mentions that TikTok owes a lot of its early traction to its predecessor, musical.ly. but there's no mention of how/whether musical.ly got license for their lip sync music. Was the success of musical.ly (and by some large extension, TikTok) based on the difficult-to-stop (because China) pirating of music? I now unleash the piracy-is-not-theft dogs...

bigpumpkin 6y ago
They used licensed music
- creeble 6y ago
  Musical.ly did? How did they negotiate that license, I wonder? Let me put it another way then: Is the success of musical.ly and its successor TikTok really about somehow getting licensed music on a platform out the gate? And the sorting hat is just a good refinement that wouldn't have a chance to exist without the licensing?

reedwolf 6y ago

"On my way in and out of this office, just one of several Bytedance spaces all across the city, I gawked at hundreds of workers sitting side by side in row after row in the open floorplan. It resembled what I’d seen at tech giants like Facebook in the U.S., but even denser." -- As much as US devs whine about open offices, it could be worse.

draw_down 6y ago
Oh, things can always be worse. But it's probably not a great idea to wait until they can't get any worse, before you speak up.
Cactus2018 6y ago
Found photos of Bytedance's open office: empty: https://officesnapshots.com/2016/07/13/bytedance-offices-beijing/ https://officesnapshots.com/2016/07/13/bytedance-offices-bei... full of staff: https://www.caixinglobal.com/2020-04-22/bytedance-drops-paywall-for-novels-on-its-toutiao-app-101545886.html https://www.caixinglobal.com/2020-04-22/bytedance-drops-payw... with a faux ceiling: https://www.livemint.com/companies/start-ups/tiktok-parent-eyeing-content-startup-deals-in-big-push-into-india-1563301051925.html https://www.livemint.com/companies/start-ups/tiktok-parent-e...
- kakkun 6y ago
  What's the purpose of the faux ceiling? I would have hoped the large expansive space and glass ceiling would be used to help alleviate feelings of claustrophobia.
- stevesearer 6y ago
  I run Office Snapshots (first link) and hadn't seen the faux ceiling you linked to -- definitely interesting. @kakkun My guess is that it would be limit direct sunlight, though if that were the case, a shade higher up might look better. I do remember someone telling me that if you go into a Costco food court people often gravitate to the tables with an umbrella on them even though they are indoors. Their theory was that people do like to have something over their heads in such a big warehouse-size space.

mgraczyk 6y ago

> How did an app designed by two guys in Shanghai managed to run circles around U.S. video apps Part of the explanation is that they weren't in Shanghai. The founders both worked full time at US tech companies in California when musical.ly was founded. musical.ly may have been founded while they were temporarily in Shanghai, but the founders were not some random people who didn't understand US culture. This is important to the article's thesis. At least part of the early insight into the US market comes from the founders living and working in the US.

wenbin 6y ago
The musicaly people were already veterans in Silicon Valley tech scene (not young under silicon valley standard & with years of working experience). They raised money to build education product with short videos in it. The education product was not doing well. So they cut everything and kept the short video part. Sounds familiar? YouTube was initially a dating website with videos. Then they cut the dating part and kept videos. Instagram was initially a checkin app with photos. Then they cut the checkin part and kept photos.
- jehlakj 6y ago
  For a second I thought you were going to mention vine.
  - dilyevsky 6y ago
    With vine twitter went one step further and got rid of video part too
- egypturnash 6y ago
  Flickr: initially an image management/sharing tool for a 2d online game. Cut the game, kept the image sharing. Slack: initially an internal communication tool for another incarnation of the same game. Cut the game, kept the chat. (An incarnation of this game did briefly exist between these two, but only lasted about a year.)

preommr 6y ago

Instead of being amazed that TikTok beat juggernauts, maybe it should be reframed that they beat companies that have terrible products. Sine we're talking about niches and customized feeds, let's look at reddit. My god is the reddit app absolutely horrendous. It's so bad I use the web app in desktop mode because other forms have terrible UX. They just don't care. Or let's talk about another site that was mentioned: youtube. It's like Google with all their money, doesn't really care to spend real money on trying to solve problems. If they dumped something like 100-200 million into better community outreach, and actual going in and tagging videos, and dealing with content, the service would be so much better. Instead, they made their money on search, and now it's spinning it's wheels trying to find clever algorithms to solve problems. We knew there was a market for an app that lets you easily create, edit and upload videos. Vine proved the formula worked. TikTok stepped in and filled the void.

tmabraham 6y ago

Very interesting read about why TikTok is succeeding. It makes me more curious about how the TikTok curation algorithm works! I appreciate the comparison of TikTok to other social networks like Facebook and Twitter, but I am surprised there is no discussion about Reddit? Reddit is interesting in that people have to follow topics/communities in the form of subreddits, so it only works if the user puts in the effort to find communities to subscribe to. Maybe Reddit could also benefit from a better curation algorithm.

peter303 6y ago

Interesting article in NYTimes a few minutes ago that several Trump advisers had to talk him into the MicroSoft compromise just before he was to sign the banning order. The advisors were horrified they'd lose an angry generation of new voters. In a sense this now a win-win for the administration: they punish China a little bit and dont overly alienate young voters.

titanomachy 6y ago

I installed TikTok out of curiosity after reading this article. I scrolled without stopping for a couple hours and then uninstalled the app. An endless stream of algorithmically curated 30-second videos is pure crack to my brain, apparently. I bet a lot of younger folks with still-developing brains are also unable to resist.

ontifica 6y ago
I'm on the younger side, it's very addictive. People love to come up with wild theories for why the app blew up, but I think it's just pure algorithmically curated fun.
intended 6y ago
Think of it this way - how long can this network exist without having to deal with political speech ? And the moment people figure out how to optimize for dispersion through the tik too algo, what would be the result ?
Noumenon72 6y ago
There's a classical conditioning loop where you see an attractive woman dancing to a song, which makes you subconsciously start to like the song, then you like videos with that song... it's the combined power of earworms and arousal. If they somehow make it taste like cheesecake we will have achieved full wireheading.
- newen 6y ago
  Smell-o-vision. The next revolution in smartphone technology!
Yhippa 6y ago
I'm an adult and normally don't get got by these types of apps but I found myself doing the same. I had to uninstall.

corford 6y ago

Very insightful read. Loved the part describing TikTok's algorithm as essentially a zero friction, global 'market maker' for content producers and consumers. Really clicked with me.

intended 6y ago

Does this dodge the problem of political speech ?

m3kw9 6y ago

Am I the only one not hooked on TikTok? It’s just a lot of good looking people repeating similar jokes. Maybe their algorithm haven’t figured me out yet.

auganov 6y ago

> But in the reverse direction, America has been almost as impenetrable to Chinese companies because of what might be though of as America’s cultural firewall Why would they even try? You've got a nice big domestic tech ecosystem which would not have existed without protectionism. Most of these products aren't competitive at all. Nothing to do with culture, same reason Canada, UK etc aren't known for big tech successes. Musical.ly is unusual in that they did actually work on an original idea and decided to launch the product in America where it took off first. Ended up not playing the China game early on. > With it, a massive team made up mostly by people who’ve never left China, and many who never will, grabbed massive marketshare in cultures and markets they’d never experienced firsthand. To a cultural determinist like myself, that feels like black magic. Pretty much every big American tech company does that. Nothing notable about this. > I don’t think the Chinese product teams I’ve met in recent years in China are much further ahead than the ones I met in 2011 when it comes to understanding foreign cultures like America. But what the Bytedance algorithm did was it abstracted that problem away. Understanding in what way? You work on a product and you optimize it for your users. If you work in a 100% domestically focused ecosystem obviously you'll design your products a little differently. What happened here is a Chinese company was for a change actually working on a product with a global userbase and they had to make them happy. The whole thing is borderline insulting to the founders.

webninja 6y ago

The author of this article mentions that the TikTok algorithm alone is worth 30B. One TikTok user claims to have figured it out. 1 rewatch = 6 points 1 completion = 5 points Share = 3 points Comments = 2 points Like = 1 point Videos with the most points above a minimum threshold are shown to an increasingly wider audience. Full video: https://vm.tiktok.com/Jjv2cXC/ https://vm.tiktok.com/Jjv2cXC/