22 ms·

Behind TikTok's boom: A legion of traumatized, $10-a-day content moderators

237 points by nixcraft 4y ago

georgebarnett 4y ago
I find the generic “workers are important” quotes remarkable, in the sense that they do nothing but contribute to the sense that these companies really don’t care. It would almost be better to say nothing.
- gsatic 4y ago
  The ppl in the companies care. But they have become small replaceable cogs/machine parts given the scales the machine has reached. The machine has become mindless and uncontrollable. Larger things scale less control over everything the chimp troupe with their 6 inch chimp brains have. Its the lesson of the last 2O years - we quickly loose control and enter Jurassic Park style reality.
  - cinntaile 4y ago
    The people working at a company are the company culture.
    - throwaway773748 4y ago
      Yes, but the people at the top define what that culture is.
      - rented_mule 4y ago
        Too often the people at the top don't define the culture explicitly. Instead, they do it accidentally through behaviors they incentivize, often without even realizing it. Sometimes they do this in direct conflict with the culture they are trying to explicitly champion (e.g., saying "we believe in transparency" while subtly punishing people for being open, maybe for well-intentioned reasons). And once a given company culture takes hold, it can be very hard to change because people who thrive in that culture tend to stay longer and get promoted more. Those people end up being a kind of "momentum" for the proliferation of said culture. In my experience, it takes people in positions of power, and with real skills for cultivating culture, to deeply change an existing work culture for the better. I've seen a few leaders with such skills make wonderful changes at their scale of influence. But I've seen far more people in leadership positions who act as if they are largely unaware of the nuance and importance of good company culture. Such changes are also hard because meaningful cultural improvements often conflict with short term revenue/profit. It takes a lot of discipline in senior leadership to maintain the needed resolve given ever-present pressures to produce in the short term.
    - Manu40 4y ago
      This. It only gets the way it is, because the people have allowed it. I know it's not ideal to have to go through the pain and trouble of being the good and righteous person who stands up against bad employers; but you HAVE to do it, or this is what happens.
    - michaelt 4y ago
      There are different perspectives on this. One perspective is that large institutions have a sort of life of their own, caused by the interaction of internal factions and priorities, and almost beyond the ability of any individual to change. Nobody in America is in favour of school shootings, and yet they get loads of them. Nobody at Google wants people to avoid their products expecting them to be cancelled, and yet they have that reputation. Another perspective is that of course the institution's actions are the result of individuals' actions, what else could it possibly be? That multinational that appears to have committed manslaughter would like us to believe nobody was responsible and it was just a tragic series of misunderstandings and communication breakdowns, so nobody should face any punishment. Are we fools to believe that?
      - TeMPOraL 4y ago
        > Another perspective is that of course the institution's actions are the result of individuals' actions, what else could it possibly be? Of course, but the naive part is believing all those individuals are spherical people in a vacuum, making independent decisions that are all well-thought-out and optimize for globally best outcome. > That multinational that appears to have committed manslaughter would like us to believe nobody was responsible and it was just a tragic series of misunderstandings and communication breakdowns, so nobody should face any punishment. Are we fools to believe that? Yes and no. It's foolish to believe individuals have much agency in this setting. Everyone, from the bottom tier to C-suite, is entangled in a web of interlocking incentives, that are ultimately anchored outside of any one company or institution[0]. Some people are handling large levers and could almost unilaterally make the company change course, but at a great cost to themselves[1], and thus it's foolish to expect them to become heroes, especially before an issue hits the news cycle. It's the same kind of error in thinking that rests behind ideas like "voting with your wallet". That said, the conclusion that "nobody should face any punishment" is also wrong. Punishment is a powerful incentive that can cut through the tangled web. Even if you believe nobody is by themselves responsible for a bad thing, targeted punishment (or threat of it) at e.g. people holding the levers can encourage them to be more eager to pull those levers and steer the entire system away from causing the bad outcome. ---- [0] - Like, desire to keep your current standard of living, whatever it is, which may be driven not by your own need for comfort, but a desire to not disappoint your spouse and/or children. [1] - And highly likely it would be reversed the moment they got fired for operating the lever wrong. Checks and balances :).
Zealotux 4y ago
Just watching regular TikTok for a few days was enough to make me feel depressed. I can't imagine seeing the dark side of it.
- lm28469 4y ago
  The content and the traction it gets is insane, especially considering the target audience. I don't get why it's not banned or heavily restricted. It's a cess pool of fakes, propaganda, semi hidden advertisement, and other bottom of the barrel grade content
  - airpoint 4y ago
    The content is highly customised and targeted to one's liking. I've never seen any of the types you mention, besides an occasional pro-China fake news propaganda haha. This is not to say TT doesn't contain such content, but it should be considered the feed you're seeing might be *significantly* different to someone else's feed…
    - lm28469 4y ago
      I don't have a tt account nor the app so all I see is from their front page
  - pennaMan 4y ago
    >I don't get why it's not banned or heavily restricted What's up with this authoritarian discourse on HN? Since when are we casually calling for censorship of things that are "disliked"? The megacorps are having videos taken down off so-called private clouds for calling out the rulling class too much. I worry for the Orwellian direction we are heading into.
    - ceejayoz 4y ago
      > Since when are we casually calling for censorship of things that are "disliked"? Some would argue: since technological progress outpaced society's capacity to cope with it. One of the possible answers for the Fermi Paradox is "everyone invents Facebook shortly after computers", and it seems a lot more plausible than it did ten years ago. It's hard not to see surveillance capitalism as a variant of what Orwell was warning about - Facebook etc. are the telescreens and two-minutes-hate from 1984. As megacorps start to rival governments in reach and power, who exactly controls them is somewhat less important than their overall impact on the civilization.
therusskiy 4y ago
Looks like it more or less corresponds to the minimum wage in Colombia, which is roughly $10/day* [1] https://biz30.timedoctor.com/average-salary-in-colombia/ https://biz30.timedoctor.com/average-salary-in-colombia/
- kdmsk 4y ago
  Noooo don’t you know a content moderator should be paid like an SRE?
  - ceejayoz 4y ago
    There's a vast gap between $10/day and a SRE's salary, and minimum wage doesn't sound reasonable for something that's this traumatizing.
    - kdmsk 4y ago
      You get paid depending on how hard you are to replace. This kind of labour can be done by basically anybody.
      - ceejayoz 4y ago
        Sure; companies have long gotten away from dumping their externalities out on the societies that host them. Criticism of this sort of thing isn't arguing that it's not legal; the argument is that it's bad.
      - micromacrofoot 4y ago
        As Amazon is discovering with delivery drivers, at one point you start running out of "basically anybody"
      - Bakary 4y ago
        Yes, so the conclusion is that political agitation is necessary if the goal is to change those working conditions
twoeruuoi234 4y ago
Erm, don't think many of Facebook's "moderators" in India are paid any better. I think most BPO/call-centre jobs pay as much (it's about the GDP-per-capita of "rich" states like Karnataka).
- KaiserPro 4y ago
  I mean thats not really an argument is it? forcing people to do shitty jobs that make that unwell, for shit all cash isn't a great look. Arguing that its ok because they make slightly above the median income for the place they live, even less so.
  - dzhiurgis 4y ago
    Maybe we should blame people who produce horrible content instead?
    - ceejayoz 4y ago
      In many cases, that's a shitty, low-paying job too. https://restofworld.org/2021/the-chinese-content-farms-behind-factory-tiktok/ https://restofworld.org/2021/the-chinese-content-farms-behin... Same for the folks at click farms boosting those pieces of content. https://www.theguardian.com/technology/2013/aug/02/click-farms-appearance-online-popularity https://www.theguardian.com/technology/2013/aug/02/click-far... It's fascinating to me how many people really want to blame the cogs in the machine over the designers and owners of it.
      - IshKebab 4y ago
        Those aren't creating traumatising content though. Totally different thing.
        ceejayoz 4y ago
        Then I'm at a loss as to what the parent poster wanted; acts of child porn and murder and whatnot are already illegal and fairly regularly punished. Those folks get plenty of (fair) blame.
Proven 4y ago
carrolldunham 4y ago
Is it too trite to say that if you only need to check for a video being deletion-worthy, you can do it without watching the video? I click around a paused video if in doubt, and/or make it really small. This has saved me several ruined days. Couldn't they do the job effectively with a preview strip? playing the video for clarification if unclear
raspyberr 4y ago
I watched some TikToks recently by scrolling on the site and it was easily some of the lowest quality stuff I've ever seen in my life. They were literally just someone's face, a caption, and some random music for about 15 seconds. I've tried a few times now to search for tags of things I like and it is just the lowest denominator stuff I've ever seen on the internet. It's one step away from AI generated (and even that has started doing pretty great things now). I'm sure it's algorithm is amazing but is using TikTok any different to reading reddit or watching YouTube really? Does anyone actually transfer the stuff they read/watch from short term to long term memory?
- permo-w 4y ago
  if you think tiktok is bad, you should try instagram reels
- gjulianm 4y ago
  The "defaults" on TikTok are fairly terrible. However, after spending maybe one hour at most scrolling and liking/disliking things to tune the algorithm, and search tags, it works fairly well. I have - Learned several cooking recipes and some tricks - Cleaning advice I've actually put to use - How to use some woodworking tools. Also, general woodworking ideas - Cat behavioral advice In my experience, it's far better than Reddit (mostly because there's far more niche content, and it isn't limited to "recently posted") and the discovery is far easier than on YouTube.
  - Tomte 4y ago
    "However, after spending maybe one hour at most scrolling and liking/disliking things to tune the algorithm, and search tags, it works fairly well." Can someone support that from Germany? I suspect that's highly locale-dependent. I have given TikTok several tries, spent hours searching for interesting stuff and liking things. I always get generic (and really bad) cooking videos and drunken people doing stupid things.
    - overlordalex 4y ago
prvit 4y ago
$10/day is probably a lot for this, I wonder how much they could cut their moderation costs by just launching a public platform where anyone anywhere in the world can get paid to do moderation.
- corobo 4y ago
  Wouldn't that necessitate being a distributor of anything you could be trying to moderate?
  - prvit 4y ago
    Why would that be any different than the current situation? In fact, I think conducting this more publicly would have huge benefits. Perhaps you would attract more motivated staff, like pedophiles to go through CSAM?
    - corobo 4y ago
      Because they can control what happens to the content once it has been found. That second point is kinda daft and would be an absolute PR nightmare at best.
      - prvit 4y ago
        > Because they can control what happens to the content once it has been found. No? The current moderators work from home, what control can TikTok have?
        corobo 4y ago
        FTA > Carolina, a former TikTok moderator who worked remotely for Teleperformance between June and September 2020, said supervisors asked her to be on camera continuously during her night shift. She was also warned that nobody else should be in view of the camera, and that her desk should be empty, apart from a drink in a transparent cup.
club_tropical 4y ago
Content moderators and their pay should be a footnote - the real story is Tiktok's treasure trove of criminal evidence, presumably with a lot of PII. It is telling that there isn't any effort to refer evidence automatically to local authorities; the crimefighting would be like shooting fish in a barrel. It would even help the content moderators by reducing caseload after a few highly publicized cases of Tiktok criminals automatically getting caught and prosecuted. That this is not done indicates Tiktok is a giant kompromat honeypot.
- onphonenow 4y ago
  This is I think false - the will to prosecute by authorities is often low. Almost every online platform has great evidence of various frauds and crime. I heard Facebook reports 20 million images or more PER YEAR. I’ve had a few experiences. In one someone was cashing bogus checks and we got them to provide the account they’d been asked to deposit them to. There was zero interest in taking a report or looking at whose account that was (person in org who’d been cutting the checks stopped showing up and we were pretty sure was involved). Endless card fraud on tape at merchants- again crickets.
  - dannyw 4y ago
    Yeah, the failure is unlikely to be on the platform side. I visited a police station in person to report a $3000 fraud with clear evidence and also document forgery. Police told me to get a lawyer, gave me the fraud report but told me I'm wasting my time if I fill it out. I've also had a stolen laptop, tracked with Find my Mac, and again, police handed me the report form and told me to contact my insurance.
    - club_tropical 4y ago
      I hear you guys on the (relatively small time) financial fraud / theft, but the article mentions far worse. If you went to the police with a video and PII of a violent crime, would they still not open an investigation?
hodgesrm 4y ago
The scenes from this article exceed any dystopian novel I've ever read. Humans have spent centuries building up social structures to eliminate our worst excesses. Social media is tearing those structures down again.
- hoseja 4y ago
  Or maybe the purpose of the social structures is only to perpetuate the social structure and any excesses that don't threaten it are ignored.
- luckylion 4y ago
  Are you sure they're not just made visible to a wider audience, but would be there just the same if social media didn't exist?
  - hodgesrm 4y ago
    Some of what appear would have happened anyway. Social media also normalizes behavior (and weird beliefs) to the extent that they become destructive to the rest of society. QAnon is an example.
    - luckylion 4y ago
      Even QAnon and similar stuff looks like something which you previously just wouldn't have noticed. Lots of people believe in extremely crazy things, sects worked just fine before social media, but unless you had a family member drawn into it, or lived next to members, you probably didn't see much of it unless some journalists decided that it was a topic worthy of an article.
      - Loughla 4y ago
        You didn't actually respond to the point being made: >Social media also normalizes behavior (and weird beliefs) to the extent that they become destructive to the rest of society. Yes, there have always been loonies. Yes. But the issue is that social media has brought them together and given them a voice they have never had before. When in the history of the US has there been a massively mainstream movement predicated on the fact that Democrats are literally agents of the devil, that wealthy elites harvest chemicals from tortured children, and that a shady New York businessman might be God's representative on earth?
Havoc 4y ago
Every time content moderation comes up I’m horrified anew. So sad that this sort of job is necessary. I’m curious though why YouTube never seems to feature? Most of the stories I’ve seen seen to be FB
- wffurr 4y ago
  The YouTube PR machine is more powerful and better at squashing bad press.
- Nasrudith 4y ago
  People seem to actually understand its large scale and evergrowing backlog better, along with "everybody can upload" better. I suspect that is motivated reasoning shining through. They don't want to give it up for a "whitelisted youtube". At most they kvetch about search algorithms.
- starik36 4y ago
  > necessary Maybe necessary, but is it worth the problems that it's creating? I would argue that the armies of scarred people is worse than not moderating at all.
  - chrisbolt 4y ago
    Then what do you do with traumatizing content, just keep showing it to everyone and traumatizing more people to a lesser degree?
    - starik36 4y ago
      I don't have a solution. I am not sure that there is one. Just don't want to subject a group of people to trauma.
      - togs 4y ago
        I think the solution is that these jobs need to be compensated in proportion to their value, with applicants understanding what they've signed up for. Without moderation, there are no services. Governments that prioritize the protection of children among other things should enforce this.
JadoJodo 4y ago
I’ll say first that I think it’s horrible that we have this problem at all. I wonder if a way to avoid traumatizing workers with entire videos would be to capture single frames from videos, and divide them across all the workers with them still rating the frame for content violations. Obviously, some frames would be worse than others to see, but I would have to imagine it would be drastically lower in terms of impact for each employee than watching video.
- ceejayoz 4y ago
  A still frame from a video can lack extremely important context you'd get from the whole thing.
whywhywhywhy 4y ago
Disingenuous to just name TikTok for this as if it’s some new extra evil social network. FB/IG/Twitter all do this and have for over a decade.
- tejohnso 4y ago
  > Disingenuous That's true. I'd call it western hypocrisy. https://strategic-culture.org/news/2016/10/16/western-hypocrisy-why-makes-world-dangerous-place/ https://strategic-culture.org/news/2016/10/16/western-hypocr...
  - ceejayoz 4y ago
    I'd call that "didn't bother to check". Plenty of coverage of this sort of thing at Facebook: https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona https://www.theverge.com/2019/2/25/18229714/cognizant-facebo... Facebook had to pay up, too: https://www.theverge.com/2020/5/12/21255870/facebook-content-moderator-settlement-scola-ptsd-mental-health https://www.theverge.com/2020/5/12/21255870/facebook-content...
- ceejayoz 4y ago
  There's been this sort of coverage of Facebook's moderation woes, too. Running a story about one murder doesn't mean you don't care about the other murders. It just means the article is about one particular case.
- pluc 4y ago
  Don't forget PornHub, there have been articles about their content moderation teams facing the same problems obviously exacerbated by the platform. Plus the tragic thing there is if you see something fucked up... it might not even mean you should remove it.
powerapple 4y ago
TikTok is the best thing happened in the internet. Why do I say so? Remember Facebook allowing you to reconnect with your lost friends. TikTok is doing this by connecting the world. I have seen many content I would not have seen from any other platform. Interesting food, interesting places, interesting personality. Unfortunately, the same side effect happened to Facebook is also happening on TikTok. But I have to say, TikTok made me seeing more things around the world. It is great in this aspect.
_trampeltier 4y ago
How they make sure, they don't have just gore fans and other strange persons as moderators?
arthurofbabylon 4y ago
What a horror. We have an underclass citizenry subject to abuse or at best terribly life-changing work conditions, without the resources (personal, cultural, societal) to protect themselves or change position. It’s horrific. It is a shared problem. Colombia is not so far away, neither geographically nor as a gig-heavy economic model. We should be able to solve this – not from a “content moderation” perspective but from a human work conditions perspective. Counterintuitively, solving content moderation probably requires solving the human work conditions. If the workers were paid considerably better and well-supported, the costs might rise to the point of successfully incentivizing the ideal automated solutions.
arthurofbabylon 4y ago
Question – at what point does content moderation become necessary? Small publishing tools successfully use passive mechanisms to prevent illegal/horrific content (for example: community enforcement and platform culture, identity verification, cost/pricing). At what point do bad actors and unwanted content arrive? At what point do the passive mechanisms fail?
- deleted 4y ago
  [deleted]
bpicolo 4y ago
How will services like tiktok ever move past this without strict identity verification and a supporting global legal system? This seems like a common issue for every photo / video content provider. We've seen this exact same article about Facebook content moderation. They have talented AI developers. They have a massive amount of harmful known content from moderation. They still need to surface harmful content to humans to check.
- micromacrofoot 4y ago
  They could try training and compensating moderators more like software developers, for one. Ultimately maybe it just doesn't make sense for social networks to be centralized monoliths.
  - ackfoobar 4y ago
    Unlike software, human moderation work is literally non-scalable. There just isn't enough value they make per person to compensate them anywhere like software developers. Besides, demand and supply.
    - micromacrofoot 4y ago
      There's probably something workable between "$200k salary and great benefits" and "$10 a day to be traumatized" that works better than the current system.
- pjc50 4y ago
  > global legal system Every time you hear "social media should be unmoderated and it should be impossible to deplatform people", you should think of all the traumatizing-but-not-actually-illegal material here which people want shown to everybody.
  - xfitm3 4y ago
    I haven't found anything traumatizing. It's the real world, living in a bubble serves no one other than small children.
groestl 4y ago
How about: * Identity verification for all uploaders * Suing for punitive damages for every upload that traumatizes a worker
- tantalor 4y ago
  Aaaaaand you killed the business.
  - NoGravitas 4y ago
    I see no downside here.
    - kderbyma 4y ago
      I wish I could upvote this more ....what value does tiktok offer??? I honestly wonder...I would happily bet money that it's net negative across the world.....
  - noasaservice 4y ago
    You mean that "Socializing the costs, and privatizing the gains" fails when you start forcing companies to pay for externalities? And that's social media, pollution, and climate change in a nutshell.
- digdigdag 4y ago
  Won't happen. It would cut out a lot of creative, legitimate content from people who would otherwise be unwilling to divulge their personal information to a CCP-governed application.
dvngnt_ 4y ago
I did a paper in undergrad years ago about facebook moderation it was pretty much the same with underpaid foreign contractors with ptsd
z9znz 4y ago
What's most disturbing is the realization that a measurable percentage of humans are just f*cked up. Considering how many reports there are of Facebook, Youtube, Tiktok, and other sites where content moderators are suffering from having to perform their jobs, it suggests that there's a great deal of really terrible stuff going on - and worse, that the people involved are filming and attempting to share it. This is a serious thing, and it paints humanity as being far darker than it would seem from the surface. It also suggests that the apocalyptic movies may not be so far off when they suggest that humanity will revert to open barbarism if we're faced with a catastrophy large enough.
- cainxinth 4y ago
  > What's most disturbing is the realization that a measurable percentage of humans are just f*cked up. I already knew that. I even know the percentage roughly. According to the Pareto principle, 80% of people are not totally fucked up, but the 20% that are cause 80% of the problems.
  - omgomgomgomg 4y ago
    Whats the gender segmentation on that, any info?
  - Bakary 4y ago
    There's a nested Pareto in that 20% of that 20% causes 80% of the 80%. Considering that psychopathy (or whatever the updated term is) runs at a bit more of a percent of the population...
  - throwamon 4y ago
    > I even know the percentage roughly. No, you don't. This "principle" is not a law of physics, it's just a rule of thumb that may or may not be wildly off the mark for an arbitrary distribution. Its common usage is like statistics horoscope.
    - andreilys 4y ago
      Having worked as a data scientist at multiple companies (From FANG to startup), the first thing I look at when I get my hands on data is the existence of the Pareto principle. I still haven’t found one company where this principle didn’t show up.
thenerdhead 4y ago
The sad reality is that these $10-a-day content moderators make more money than medium sized creators who don't know how to monetize and are hoping the TikTok creator fund will do them justice. It's a very manipulative platform on all sides. Creator, Consumer, and Moderator.
- andreyk 4y ago
  To be fair, this is true for solo artists in general - musicians, writers, indie game devs, youtubes. Even substack writers. More often than not it takes a long time to build up an audience and all.
- Dma54rhs 4y ago
  What makes you think tiktok even needs profesional paid content creators?
calme_toi 4y ago
Similarly, behind every pair of cheap jeans/t-shirts/any other "fast fashion" stuff, there are many more workers paid even less than 10USD per day.
- throwamon 4y ago
  Sure, but I bet they don't have to watch all kinds of traumatizing events as they do it.
yalogin 4y ago
Wow, I thought I was expecting nasty stuff to be mentioned there, but what is mentioned I would not even imagine in my wildest. Humans are capable of truly nasty and even more so put it on line for every one to see? I would love to know from people working in this space how they handle illegal videos. Are they responsible to report it to the authorities in the respective countries? The company will not have a presence in many of the countries the app is used in, so even if they wanted to report to the authorities it's impossible to keep up. Do they just delete and move on? The legal, ethical implications here are staggering.
- comfypotato 4y ago
  This reply was therapeutic for me. I wish the post had a trigger warning. I was very caught off guard by the first vulgar content that was mentioned, and I’m a little upset by the terribleness of it. I immediately stopped reading, but I wish I hadn’t started. I was reading through these comments looking for validation for my feelings and this comment helped. Thanks. My naivety got the best of me I think.
ck2 4y ago
One step up from China and North Korea's prisoners forced to "mine gold" online https://www.theguardian.com/world/2011/may/25/china-prisoners-internet-gaming-scam https://www.theguardian.com/world/2011/may/25/china-prisoner...
MisterTea 4y ago
This is all unnecessary. Social media is a soulless advertising platform that destroys peoples sense of self and being. I don't use them and I live a nice life devoid of vapid self promotion and other nonsense. Resist. Delete your social media accounts.
jmyeet 4y ago
It doesn't surprise me there's really disturbing content. This is the case for any platform. But how much do moderators spend dealing with that rather BS reports? If you spend any time on Tiktok you'll quickly notice that reporting content is heavily brigaded and weaponized. The general process is: 1. Mass reports, probably from automated fake accounts, mass report one or more videos and/or the creator themselves; 2. As with almost all such systems, a certain number of reports triggers an automated response, such as taking down the video; 3. The creator then has to manually appeal the takedown. Sometimes they win, sometimes they lose. It seems to be really inconsistent; 4. Regardless of the outcome of the appeal, a certain number of reports will trigger a community guidelines violation, possibly locking or even completely nuking the account. The fact that every appeal is won seems to be irrelevant. This problem is so bad that you can duet a video, say nothing and get reported for hate speech or harassment and lose your account. What Tiktok doesn't do (like pretty much everyone else) is identify false reports and people who make false reports. It tends to be pretty easy too. Defining characteristics tend to be: 1. A default username ("user123239842"); 2. Only follows 1 person; 3. No PFP. Other sites (eg Twitch) make efforts to at least identify ban evaders. If you don't like someone's content Tiktok should make it hard for you to find it again and when you do, they should just shadowban you. Your comments don't appear to anyone else (which reduces harassment) and your reports are essentially ignored (while appearing to have gone through). I'm actually amazed at just how easily and how often these systems, which supposedly exist to take down offensive content and protect users, are weaponized for simply saying something someone just doesn't like. This would surely reduce the time wasted for moderators (who often rule inconsistently on these issue) so the truly abhorrent content gets immediately nuked.
DrNosferatu 4y ago
This "Teleperformance" company keeps showing up at the mention of sweatshops. Remember that name.
BrainVirus 4y ago
It's almost as if building a hypercentralized faux-communication system that tries to encourage mindless behavior and doesn't have any notion of locality is an inherently bad idea that will always lead to misery and suffering. Hm. Nah, that's can't be true. That would mean a lot of things I've read on the internet recently are lies, which is completely impossible. Yeah, I think this is just a sign that people suck and we need more centralization to control them. If we tied all TikTok accounts to some kind of social credit score, that would probably fix everything.
- deleted 4y ago
  [deleted]
pupppet 4y ago
Whenever I browse TikTok it's like flipping through the channels during the 80's, except every channel is only ever showing commercials.
deleted 4y ago
[deleted]
more_corn 4y ago
At YouTube the content moderation team (site quality) had mandatory group therapy. The therapist recommended not bottling up seeing traumatic things so the moderators would sometimes share between each other. One moderator mentioned that he would never open a link sent by a particular other moderator. I asked why and he said “he’s brown listed”. People suck and the internet brings out the worst in us. They were also on one month contracts. If you made it through your initial period you could choose to convert to full time. Most didn’t. I shudder to think what the next generation of moderators is facing.
dirtyid 4y ago
AKA the Chinese/PRC model, you need a shit load of actual mk1 eyeballs to filter internet into hugbox for political/domestic serenity. Why western platforms had to pull out of PRC (again, not banned) after minority riots, because they couldn't stomach the onerous moderation costs required to operate in PRC that other PRC platforms had to endure. A few years later, radicalization on western platforms compelled the same measures - CCP has consistenly been prescient in domains of mass communication. Fringe voices are loudest in an unrestricted enviroment, it takes a lot of resources to put up walls and prune weeds for popular yet harmless mainstream stuff to stand a chance.
bvoq 4y ago
Meanwhile 4chan moderators do this for free, haha
nojvek 4y ago
> “If you’re looking at this from a monetary perspective, then content moderation AI can’t compete with $1.80 an hour,” Carthy said, referring to a typical wage for content moderators based in the global south. “If that’s the only dimension you’re looking at, then no content moderation AI company can compete with that.” I didn't realize human labor was that cheap. I assume Teleperformance is taking a decent chunk of $1.8/hr. 15 seconds to view each video = 240 videos/hr. On AWS, p4d.24xlarge is $32/hr with 8 A100 GPUs or $4/A100 GPU hr. > “The human brain is the most effective tool to identify toxic material,” said Roi Carthy, the chief marketing officer of L1ght, a content moderation AI company. The learning on my end, is better cheaper computer vision would be able to alleviate human traumatization. The flip side is we'd also be able to generate 1000s of human traumatizing videos.

237 points by nixcraft 4y ago

I find the generic “workers are important” quotes remarkable, in the sense that they do nothing but contribute to the sense that these companies really don’t care. It would almost be better to say nothing.

gsatic 4y ago
The ppl in the companies care. But they have become small replaceable cogs/machine parts given the scales the machine has reached. The machine has become mindless and uncontrollable. Larger things scale less control over everything the chimp troupe with their 6 inch chimp brains have. Its the lesson of the last 2O years - we quickly loose control and enter Jurassic Park style reality.
- cinntaile 4y ago
  The people working at a company are the company culture.
  - throwaway773748 4y ago
    Yes, but the people at the top define what that culture is.
    - rented_mule 4y ago
      Too often the people at the top don't define the culture explicitly. Instead, they do it accidentally through behaviors they incentivize, often without even realizing it. Sometimes they do this in direct conflict with the culture they are trying to explicitly champion (e.g., saying "we believe in transparency" while subtly punishing people for being open, maybe for well-intentioned reasons). And once a given company culture takes hold, it can be very hard to change because people who thrive in that culture tend to stay longer and get promoted more. Those people end up being a kind of "momentum" for the proliferation of said culture. In my experience, it takes people in positions of power, and with real skills for cultivating culture, to deeply change an existing work culture for the better. I've seen a few leaders with such skills make wonderful changes at their scale of influence. But I've seen far more people in leadership positions who act as if they are largely unaware of the nuance and importance of good company culture. Such changes are also hard because meaningful cultural improvements often conflict with short term revenue/profit. It takes a lot of discipline in senior leadership to maintain the needed resolve given ever-present pressures to produce in the short term.
  - Manu40 4y ago
    This. It only gets the way it is, because the people have allowed it. I know it's not ideal to have to go through the pain and trouble of being the good and righteous person who stands up against bad employers; but you HAVE to do it, or this is what happens.
  - michaelt 4y ago
    There are different perspectives on this. One perspective is that large institutions have a sort of life of their own, caused by the interaction of internal factions and priorities, and almost beyond the ability of any individual to change. Nobody in America is in favour of school shootings, and yet they get loads of them. Nobody at Google wants people to avoid their products expecting them to be cancelled, and yet they have that reputation. Another perspective is that of course the institution's actions are the result of individuals' actions, what else could it possibly be? That multinational that appears to have committed manslaughter would like us to believe nobody was responsible and it was just a tragic series of misunderstandings and communication breakdowns, so nobody should face any punishment. Are we fools to believe that?
    - TeMPOraL 4y ago
      > Another perspective is that of course the institution's actions are the result of individuals' actions, what else could it possibly be? Of course, but the naive part is believing all those individuals are spherical people in a vacuum, making independent decisions that are all well-thought-out and optimize for globally best outcome. > That multinational that appears to have committed manslaughter would like us to believe nobody was responsible and it was just a tragic series of misunderstandings and communication breakdowns, so nobody should face any punishment. Are we fools to believe that? Yes and no. It's foolish to believe individuals have much agency in this setting. Everyone, from the bottom tier to C-suite, is entangled in a web of interlocking incentives, that are ultimately anchored outside of any one company or institution[0]. Some people are handling large levers and could almost unilaterally make the company change course, but at a great cost to themselves[1], and thus it's foolish to expect them to become heroes, especially before an issue hits the news cycle. It's the same kind of error in thinking that rests behind ideas like "voting with your wallet". That said, the conclusion that "nobody should face any punishment" is also wrong. Punishment is a powerful incentive that can cut through the tangled web. Even if you believe nobody is by themselves responsible for a bad thing, targeted punishment (or threat of it) at e.g. people holding the levers can encourage them to be more eager to pull those levers and steer the entire system away from causing the bad outcome. ---- [0] - Like, desire to keep your current standard of living, whatever it is, which may be driven not by your own need for comfort, but a desire to not disappoint your spouse and/or children. [1] - And highly likely it would be reversed the moment they got fired for operating the lever wrong. Checks and balances :).

Zealotux 4y ago

Just watching regular TikTok for a few days was enough to make me feel depressed. I can't imagine seeing the dark side of it.

lm28469 4y ago
The content and the traction it gets is insane, especially considering the target audience. I don't get why it's not banned or heavily restricted. It's a cess pool of fakes, propaganda, semi hidden advertisement, and other bottom of the barrel grade content
- airpoint 4y ago
  The content is highly customised and targeted to one's liking. I've never seen any of the types you mention, besides an occasional pro-China fake news propaganda haha. This is not to say TT doesn't contain such content, but it should be considered the feed you're seeing might be *significantly* different to someone else's feed…
  - lm28469 4y ago
    I don't have a tt account nor the app so all I see is from their front page
- pennaMan 4y ago
  >I don't get why it's not banned or heavily restricted What's up with this authoritarian discourse on HN? Since when are we casually calling for censorship of things that are "disliked"? The megacorps are having videos taken down off so-called private clouds for calling out the rulling class too much. I worry for the Orwellian direction we are heading into.
  - ceejayoz 4y ago
    > Since when are we casually calling for censorship of things that are "disliked"? Some would argue: since technological progress outpaced society's capacity to cope with it. One of the possible answers for the Fermi Paradox is "everyone invents Facebook shortly after computers", and it seems a lot more plausible than it did ten years ago. It's hard not to see surveillance capitalism as a variant of what Orwell was warning about - Facebook etc. are the telescreens and two-minutes-hate from 1984. As megacorps start to rival governments in reach and power, who exactly controls them is somewhat less important than their overall impact on the civilization.

therusskiy 4y ago

Looks like it more or less corresponds to the minimum wage in Colombia, which is roughly $10/day* [1] https://biz30.timedoctor.com/average-salary-in-colombia/ https://biz30.timedoctor.com/average-salary-in-colombia/

kdmsk 4y ago
Noooo don’t you know a content moderator should be paid like an SRE?
- ceejayoz 4y ago
  There's a vast gap between $10/day and a SRE's salary, and minimum wage doesn't sound reasonable for something that's this traumatizing.
  - kdmsk 4y ago
    You get paid depending on how hard you are to replace. This kind of labour can be done by basically anybody.
    - ceejayoz 4y ago
      Sure; companies have long gotten away from dumping their externalities out on the societies that host them. Criticism of this sort of thing isn't arguing that it's not legal; the argument is that it's bad.
    - micromacrofoot 4y ago
      As Amazon is discovering with delivery drivers, at one point you start running out of "basically anybody"
    - Bakary 4y ago
      Yes, so the conclusion is that political agitation is necessary if the goal is to change those working conditions

twoeruuoi234 4y ago

Erm, don't think many of Facebook's "moderators" in India are paid any better. I think most BPO/call-centre jobs pay as much (it's about the GDP-per-capita of "rich" states like Karnataka).

KaiserPro 4y ago
I mean thats not really an argument is it? forcing people to do shitty jobs that make that unwell, for shit all cash isn't a great look. Arguing that its ok because they make slightly above the median income for the place they live, even less so.
- dzhiurgis 4y ago
  Maybe we should blame people who produce horrible content instead?
  - ceejayoz 4y ago
    In many cases, that's a shitty, low-paying job too. https://restofworld.org/2021/the-chinese-content-farms-behind-factory-tiktok/ https://restofworld.org/2021/the-chinese-content-farms-behin... Same for the folks at click farms boosting those pieces of content. https://www.theguardian.com/technology/2013/aug/02/click-farms-appearance-online-popularity https://www.theguardian.com/technology/2013/aug/02/click-far... It's fascinating to me how many people really want to blame the cogs in the machine over the designers and owners of it.
    - IshKebab 4y ago
      Those aren't creating traumatising content though. Totally different thing.
      - ceejayoz 4y ago
        Then I'm at a loss as to what the parent poster wanted; acts of child porn and murder and whatnot are already illegal and fairly regularly punished. Those folks get plenty of (fair) blame.

Proven 4y ago

carrolldunham 4y ago

Is it too trite to say that if you only need to check for a video being deletion-worthy, you can do it without watching the video? I click around a paused video if in doubt, and/or make it really small. This has saved me several ruined days. Couldn't they do the job effectively with a preview strip? playing the video for clarification if unclear

raspyberr 4y ago

I watched some TikToks recently by scrolling on the site and it was easily some of the lowest quality stuff I've ever seen in my life. They were literally just someone's face, a caption, and some random music for about 15 seconds. I've tried a few times now to search for tags of things I like and it is just the lowest denominator stuff I've ever seen on the internet. It's one step away from AI generated (and even that has started doing pretty great things now). I'm sure it's algorithm is amazing but is using TikTok any different to reading reddit or watching YouTube really? Does anyone actually transfer the stuff they read/watch from short term to long term memory?

permo-w 4y ago
if you think tiktok is bad, you should try instagram reels
gjulianm 4y ago
The "defaults" on TikTok are fairly terrible. However, after spending maybe one hour at most scrolling and liking/disliking things to tune the algorithm, and search tags, it works fairly well. I have - Learned several cooking recipes and some tricks - Cleaning advice I've actually put to use - How to use some woodworking tools. Also, general woodworking ideas - Cat behavioral advice In my experience, it's far better than Reddit (mostly because there's far more niche content, and it isn't limited to "recently posted") and the discovery is far easier than on YouTube.
- Tomte 4y ago
  "However, after spending maybe one hour at most scrolling and liking/disliking things to tune the algorithm, and search tags, it works fairly well." Can someone support that from Germany? I suspect that's highly locale-dependent. I have given TikTok several tries, spent hours searching for interesting stuff and liking things. I always get generic (and really bad) cooking videos and drunken people doing stupid things.
  - overlordalex 4y ago

prvit 4y ago

$10/day is probably a lot for this, I wonder how much they could cut their moderation costs by just launching a public platform where anyone anywhere in the world can get paid to do moderation.

corobo 4y ago
Wouldn't that necessitate being a distributor of anything you could be trying to moderate?
- prvit 4y ago
  Why would that be any different than the current situation? In fact, I think conducting this more publicly would have huge benefits. Perhaps you would attract more motivated staff, like pedophiles to go through CSAM?
  - corobo 4y ago
    Because they can control what happens to the content once it has been found. That second point is kinda daft and would be an absolute PR nightmare at best.
    - prvit 4y ago
      > Because they can control what happens to the content once it has been found. No? The current moderators work from home, what control can TikTok have?
      - corobo 4y ago
        FTA > Carolina, a former TikTok moderator who worked remotely for Teleperformance between June and September 2020, said supervisors asked her to be on camera continuously during her night shift. She was also warned that nobody else should be in view of the camera, and that her desk should be empty, apart from a drink in a transparent cup.

club_tropical 4y ago

Content moderators and their pay should be a footnote - the real story is Tiktok's treasure trove of criminal evidence, presumably with a lot of PII. It is telling that there isn't any effort to refer evidence automatically to local authorities; the crimefighting would be like shooting fish in a barrel. It would even help the content moderators by reducing caseload after a few highly publicized cases of Tiktok criminals automatically getting caught and prosecuted. That this is not done indicates Tiktok is a giant kompromat honeypot.

onphonenow 4y ago
This is I think false - the will to prosecute by authorities is often low. Almost every online platform has great evidence of various frauds and crime. I heard Facebook reports 20 million images or more PER YEAR. I’ve had a few experiences. In one someone was cashing bogus checks and we got them to provide the account they’d been asked to deposit them to. There was zero interest in taking a report or looking at whose account that was (person in org who’d been cutting the checks stopped showing up and we were pretty sure was involved). Endless card fraud on tape at merchants- again crickets.
- dannyw 4y ago
  Yeah, the failure is unlikely to be on the platform side. I visited a police station in person to report a $3000 fraud with clear evidence and also document forgery. Police told me to get a lawyer, gave me the fraud report but told me I'm wasting my time if I fill it out. I've also had a stolen laptop, tracked with Find my Mac, and again, police handed me the report form and told me to contact my insurance.
  - club_tropical 4y ago
    I hear you guys on the (relatively small time) financial fraud / theft, but the article mentions far worse. If you went to the police with a video and PII of a violent crime, would they still not open an investigation?

hodgesrm 4y ago

The scenes from this article exceed any dystopian novel I've ever read. Humans have spent centuries building up social structures to eliminate our worst excesses. Social media is tearing those structures down again.

hoseja 4y ago
Or maybe the purpose of the social structures is only to perpetuate the social structure and any excesses that don't threaten it are ignored.
luckylion 4y ago
Are you sure they're not just made visible to a wider audience, but would be there just the same if social media didn't exist?
- hodgesrm 4y ago
  Some of what appear would have happened anyway. Social media also normalizes behavior (and weird beliefs) to the extent that they become destructive to the rest of society. QAnon is an example.
  - luckylion 4y ago
    Even QAnon and similar stuff looks like something which you previously just wouldn't have noticed. Lots of people believe in extremely crazy things, sects worked just fine before social media, but unless you had a family member drawn into it, or lived next to members, you probably didn't see much of it unless some journalists decided that it was a topic worthy of an article.
    - Loughla 4y ago
      You didn't actually respond to the point being made: >Social media also normalizes behavior (and weird beliefs) to the extent that they become destructive to the rest of society. Yes, there have always been loonies. Yes. But the issue is that social media has brought them together and given them a voice they have never had before. When in the history of the US has there been a massively mainstream movement predicated on the fact that Democrats are literally agents of the devil, that wealthy elites harvest chemicals from tortured children, and that a shady New York businessman might be God's representative on earth?

Havoc 4y ago

Every time content moderation comes up I’m horrified anew. So sad that this sort of job is necessary. I’m curious though why YouTube never seems to feature? Most of the stories I’ve seen seen to be FB

wffurr 4y ago
The YouTube PR machine is more powerful and better at squashing bad press.
Nasrudith 4y ago
People seem to actually understand its large scale and evergrowing backlog better, along with "everybody can upload" better. I suspect that is motivated reasoning shining through. They don't want to give it up for a "whitelisted youtube". At most they kvetch about search algorithms.
starik36 4y ago
> necessary Maybe necessary, but is it worth the problems that it's creating? I would argue that the armies of scarred people is worse than not moderating at all.
- chrisbolt 4y ago
  Then what do you do with traumatizing content, just keep showing it to everyone and traumatizing more people to a lesser degree?
  - starik36 4y ago
    I don't have a solution. I am not sure that there is one. Just don't want to subject a group of people to trauma.
    - togs 4y ago
      I think the solution is that these jobs need to be compensated in proportion to their value, with applicants understanding what they've signed up for. Without moderation, there are no services. Governments that prioritize the protection of children among other things should enforce this.

JadoJodo 4y ago

I’ll say first that I think it’s horrible that we have this problem at all. I wonder if a way to avoid traumatizing workers with entire videos would be to capture single frames from videos, and divide them across all the workers with them still rating the frame for content violations. Obviously, some frames would be worse than others to see, but I would have to imagine it would be drastically lower in terms of impact for each employee than watching video.

ceejayoz 4y ago
A still frame from a video can lack extremely important context you'd get from the whole thing.

whywhywhywhy 4y ago

Disingenuous to just name TikTok for this as if it’s some new extra evil social network. FB/IG/Twitter all do this and have for over a decade.

tejohnso 4y ago
> Disingenuous That's true. I'd call it western hypocrisy. https://strategic-culture.org/news/2016/10/16/western-hypocrisy-why-makes-world-dangerous-place/ https://strategic-culture.org/news/2016/10/16/western-hypocr...
- ceejayoz 4y ago
  I'd call that "didn't bother to check". Plenty of coverage of this sort of thing at Facebook: https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona https://www.theverge.com/2019/2/25/18229714/cognizant-facebo... Facebook had to pay up, too: https://www.theverge.com/2020/5/12/21255870/facebook-content-moderator-settlement-scola-ptsd-mental-health https://www.theverge.com/2020/5/12/21255870/facebook-content...
ceejayoz 4y ago
There's been this sort of coverage of Facebook's moderation woes, too. Running a story about one murder doesn't mean you don't care about the other murders. It just means the article is about one particular case.
pluc 4y ago
Don't forget PornHub, there have been articles about their content moderation teams facing the same problems obviously exacerbated by the platform. Plus the tragic thing there is if you see something fucked up... it might not even mean you should remove it.

powerapple 4y ago

TikTok is the best thing happened in the internet. Why do I say so? Remember Facebook allowing you to reconnect with your lost friends. TikTok is doing this by connecting the world. I have seen many content I would not have seen from any other platform. Interesting food, interesting places, interesting personality. Unfortunately, the same side effect happened to Facebook is also happening on TikTok. But I have to say, TikTok made me seeing more things around the world. It is great in this aspect.

_trampeltier 4y ago

How they make sure, they don't have just gore fans and other strange persons as moderators?

arthurofbabylon 4y ago

What a horror. We have an underclass citizenry subject to abuse or at best terribly life-changing work conditions, without the resources (personal, cultural, societal) to protect themselves or change position. It’s horrific. It is a shared problem. Colombia is not so far away, neither geographically nor as a gig-heavy economic model. We should be able to solve this – not from a “content moderation” perspective but from a human work conditions perspective. Counterintuitively, solving content moderation probably requires solving the human work conditions. If the workers were paid considerably better and well-supported, the costs might rise to the point of successfully incentivizing the ideal automated solutions.

arthurofbabylon 4y ago

Question – at what point does content moderation become necessary? Small publishing tools successfully use passive mechanisms to prevent illegal/horrific content (for example: community enforcement and platform culture, identity verification, cost/pricing). At what point do bad actors and unwanted content arrive? At what point do the passive mechanisms fail?

deleted 4y ago
[deleted]

bpicolo 4y ago

How will services like tiktok ever move past this without strict identity verification and a supporting global legal system? This seems like a common issue for every photo / video content provider. We've seen this exact same article about Facebook content moderation. They have talented AI developers. They have a massive amount of harmful known content from moderation. They still need to surface harmful content to humans to check.

micromacrofoot 4y ago
They could try training and compensating moderators more like software developers, for one. Ultimately maybe it just doesn't make sense for social networks to be centralized monoliths.
- ackfoobar 4y ago
  Unlike software, human moderation work is literally non-scalable. There just isn't enough value they make per person to compensate them anywhere like software developers. Besides, demand and supply.
  - micromacrofoot 4y ago
    There's probably something workable between "$200k salary and great benefits" and "$10 a day to be traumatized" that works better than the current system.
pjc50 4y ago
> global legal system Every time you hear "social media should be unmoderated and it should be impossible to deplatform people", you should think of all the traumatizing-but-not-actually-illegal material here which people want shown to everybody.
- xfitm3 4y ago
  I haven't found anything traumatizing. It's the real world, living in a bubble serves no one other than small children.

groestl 4y ago

How about: * Identity verification for all uploaders * Suing for punitive damages for every upload that traumatizes a worker

tantalor 4y ago
Aaaaaand you killed the business.
- NoGravitas 4y ago
  I see no downside here.
  - kderbyma 4y ago
    I wish I could upvote this more ....what value does tiktok offer??? I honestly wonder...I would happily bet money that it's net negative across the world.....
- noasaservice 4y ago
  You mean that "Socializing the costs, and privatizing the gains" fails when you start forcing companies to pay for externalities? And that's social media, pollution, and climate change in a nutshell.
digdigdag 4y ago
Won't happen. It would cut out a lot of creative, legitimate content from people who would otherwise be unwilling to divulge their personal information to a CCP-governed application.

dvngnt_ 4y ago

I did a paper in undergrad years ago about facebook moderation it was pretty much the same with underpaid foreign contractors with ptsd

z9znz 4y ago

What's most disturbing is the realization that a measurable percentage of humans are just f*cked up. Considering how many reports there are of Facebook, Youtube, Tiktok, and other sites where content moderators are suffering from having to perform their jobs, it suggests that there's a great deal of really terrible stuff going on - and worse, that the people involved are filming and attempting to share it. This is a serious thing, and it paints humanity as being far darker than it would seem from the surface. It also suggests that the apocalyptic movies may not be so far off when they suggest that humanity will revert to open barbarism if we're faced with a catastrophy large enough.

cainxinth 4y ago
> What's most disturbing is the realization that a measurable percentage of humans are just f*cked up. I already knew that. I even know the percentage roughly. According to the Pareto principle, 80% of people are not totally fucked up, but the 20% that are cause 80% of the problems.
- omgomgomgomg 4y ago
  Whats the gender segmentation on that, any info?
- Bakary 4y ago
  There's a nested Pareto in that 20% of that 20% causes 80% of the 80%. Considering that psychopathy (or whatever the updated term is) runs at a bit more of a percent of the population...
- throwamon 4y ago
  > I even know the percentage roughly. No, you don't. This "principle" is not a law of physics, it's just a rule of thumb that may or may not be wildly off the mark for an arbitrary distribution. Its common usage is like statistics horoscope.
  - andreilys 4y ago
    Having worked as a data scientist at multiple companies (From FANG to startup), the first thing I look at when I get my hands on data is the existence of the Pareto principle. I still haven’t found one company where this principle didn’t show up.

thenerdhead 4y ago

The sad reality is that these $10-a-day content moderators make more money than medium sized creators who don't know how to monetize and are hoping the TikTok creator fund will do them justice. It's a very manipulative platform on all sides. Creator, Consumer, and Moderator.

andreyk 4y ago
To be fair, this is true for solo artists in general - musicians, writers, indie game devs, youtubes. Even substack writers. More often than not it takes a long time to build up an audience and all.
Dma54rhs 4y ago
What makes you think tiktok even needs profesional paid content creators?

calme_toi 4y ago

Similarly, behind every pair of cheap jeans/t-shirts/any other "fast fashion" stuff, there are many more workers paid even less than 10USD per day.

throwamon 4y ago
Sure, but I bet they don't have to watch all kinds of traumatizing events as they do it.

yalogin 4y ago

Wow, I thought I was expecting nasty stuff to be mentioned there, but what is mentioned I would not even imagine in my wildest. Humans are capable of truly nasty and even more so put it on line for every one to see? I would love to know from people working in this space how they handle illegal videos. Are they responsible to report it to the authorities in the respective countries? The company will not have a presence in many of the countries the app is used in, so even if they wanted to report to the authorities it's impossible to keep up. Do they just delete and move on? The legal, ethical implications here are staggering.

comfypotato 4y ago
This reply was therapeutic for me. I wish the post had a trigger warning. I was very caught off guard by the first vulgar content that was mentioned, and I’m a little upset by the terribleness of it. I immediately stopped reading, but I wish I hadn’t started. I was reading through these comments looking for validation for my feelings and this comment helped. Thanks. My naivety got the best of me I think.

ck2 4y ago

One step up from China and North Korea's prisoners forced to "mine gold" online https://www.theguardian.com/world/2011/may/25/china-prisoners-internet-gaming-scam https://www.theguardian.com/world/2011/may/25/china-prisoner...

MisterTea 4y ago

This is all unnecessary. Social media is a soulless advertising platform that destroys peoples sense of self and being. I don't use them and I live a nice life devoid of vapid self promotion and other nonsense. Resist. Delete your social media accounts.

jmyeet 4y ago

It doesn't surprise me there's really disturbing content. This is the case for any platform. But how much do moderators spend dealing with that rather BS reports? If you spend any time on Tiktok you'll quickly notice that reporting content is heavily brigaded and weaponized. The general process is: 1. Mass reports, probably from automated fake accounts, mass report one or more videos and/or the creator themselves; 2. As with almost all such systems, a certain number of reports triggers an automated response, such as taking down the video; 3. The creator then has to manually appeal the takedown. Sometimes they win, sometimes they lose. It seems to be really inconsistent; 4. Regardless of the outcome of the appeal, a certain number of reports will trigger a community guidelines violation, possibly locking or even completely nuking the account. The fact that every appeal is won seems to be irrelevant. This problem is so bad that you can duet a video, say nothing and get reported for hate speech or harassment and lose your account. What Tiktok doesn't do (like pretty much everyone else) is identify false reports and people who make false reports. It tends to be pretty easy too. Defining characteristics tend to be: 1. A default username ("user123239842"); 2. Only follows 1 person; 3. No PFP. Other sites (eg Twitch) make efforts to at least identify ban evaders. If you don't like someone's content Tiktok should make it hard for you to find it again and when you do, they should just shadowban you. Your comments don't appear to anyone else (which reduces harassment) and your reports are essentially ignored (while appearing to have gone through). I'm actually amazed at just how easily and how often these systems, which supposedly exist to take down offensive content and protect users, are weaponized for simply saying something someone just doesn't like. This would surely reduce the time wasted for moderators (who often rule inconsistently on these issue) so the truly abhorrent content gets immediately nuked.

DrNosferatu 4y ago

This "Teleperformance" company keeps showing up at the mention of sweatshops. Remember that name.

BrainVirus 4y ago

It's almost as if building a hypercentralized faux-communication system that tries to encourage mindless behavior and doesn't have any notion of locality is an inherently bad idea that will always lead to misery and suffering. Hm. Nah, that's can't be true. That would mean a lot of things I've read on the internet recently are lies, which is completely impossible. Yeah, I think this is just a sign that people suck and we need more centralization to control them. If we tied all TikTok accounts to some kind of social credit score, that would probably fix everything.

deleted 4y ago
[deleted]

pupppet 4y ago

Whenever I browse TikTok it's like flipping through the channels during the 80's, except every channel is only ever showing commercials.

deleted 4y ago

[deleted]

more_corn 4y ago

At YouTube the content moderation team (site quality) had mandatory group therapy. The therapist recommended not bottling up seeing traumatic things so the moderators would sometimes share between each other. One moderator mentioned that he would never open a link sent by a particular other moderator. I asked why and he said “he’s brown listed”. People suck and the internet brings out the worst in us. They were also on one month contracts. If you made it through your initial period you could choose to convert to full time. Most didn’t. I shudder to think what the next generation of moderators is facing.

dirtyid 4y ago

AKA the Chinese/PRC model, you need a shit load of actual mk1 eyeballs to filter internet into hugbox for political/domestic serenity. Why western platforms had to pull out of PRC (again, not banned) after minority riots, because they couldn't stomach the onerous moderation costs required to operate in PRC that other PRC platforms had to endure. A few years later, radicalization on western platforms compelled the same measures - CCP has consistenly been prescient in domains of mass communication. Fringe voices are loudest in an unrestricted enviroment, it takes a lot of resources to put up walls and prune weeds for popular yet harmless mainstream stuff to stand a chance.

bvoq 4y ago

Meanwhile 4chan moderators do this for free, haha

nojvek 4y ago

> “If you’re looking at this from a monetary perspective, then content moderation AI can’t compete with $1.80 an hour,” Carthy said, referring to a typical wage for content moderators based in the global south. “If that’s the only dimension you’re looking at, then no content moderation AI company can compete with that.” I didn't realize human labor was that cheap. I assume Teleperformance is taking a decent chunk of $1.8/hr. 15 seconds to view each video = 240 videos/hr. On AWS, p4d.24xlarge is $32/hr with 8 A100 GPUs or $4/A100 GPU hr. > “The human brain is the most effective tool to identify toxic material,” said Roi Carthy, the chief marketing officer of L1ght, a content moderation AI company. The learning on my end, is better cheaper computer vision would be able to alleviate human traumatization. The flip side is we'd also be able to generate 1000s of human traumatizing videos.