16 ms·
The young people sifting through the internet's worst horrors
- HannahFox 3y ago[flagged]
- john-doe 3y agohttps://archive.is/AT6JK https://archive.is/AT6JK
- lukas099 3y agoI imagine that moderating content online can take a real toll on one's mental health. Sort of like police who have to look through evidence of child porn day after day.
- ycombinete 3y agoThat makes me wonder about the ethics of perhaps having pedophiles and psychopaths put their “competitive advantage” to use in these respective occupations.
- jimnotgym 3y agoWould they be honest about it though, or give other pedophiles a free reign? Wouldn't it just make them more dangerous?
- ycombinete 3y agoTrue, the scope for abuse might be too great a risk.
- rapidaneurism 3y agoHow much would you charge them?
- optimalsolver 3y agoNow there's a vote-winning proposition.
- geraldhh 3y agopolice being bothered by cp seems miniscule in comparison to the emotional strain that comes with the daily domestic violence, physical neglect and war of the roses style cases
- iamflimflam1 3y agoI worked for a dating company for a while. The moderation team was mostly young people predominantly women. Fortunately, compared to what the people in the article deal with, it was relatively tame. But it still took its mental toll - having to look at dick pics all day is not healthy.
- dudeinjapan 3y agoJust imagine how my urologist feels. He needs to look at actual dicks all day.
- mslt 3y ago[flagged]
- aikinai 3y agoI’m not GP, but I assume the point is that it’s part of the job. Coroners look at real dead people all day. EMTs see horrific injuries. Many jobs have unpleasant aspects, some horrible, but this isn’t novel or unique.
- stinos 3y agoThat's your opinion, here's mine: I found it sort of funny, plus it actually raises an interesting point (definitely not useless, plus bonus points for such humor btw): I always wonder how gynaecologists, urologists, ... mentally deal with this exact aspect. For instance: what is sex in the evening like if you've seen x different genitals during the day? How easy is it to separate that? Or is there no separation perhaps?
- extheat 3y agoA lot of attention gets paid to sex abuse content, which it should, but there is much less I feel for all sorts of other abhorrent content. Gore, violence (physical/mental), terrorism, death, destruction, shock content, etc. It's not just NSFW, but also NSFL content that I think is under discussed at times. Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. It's unfortunate the world is a messed up place, and maybe it will stay that way as long as humans are around. This is the exact type of content that I think AI is so crucial for detecting, and it's a bit sad that we often hear so about all the bad things you can generate with AI and so little about all the good. If we need to train models on this type of content to make it easier to detect and remove, to prevent mentally scarring people, then personally I'm all for it -- regardless of the "it can also generate that bad content" cost.
- osobo 3y agoI've been online since the late eighties and over time I've seen some nasty NSFL stuff. I am absolutely certain this has shaped my world view to some degree. Like some other GenX-er said: "Internet is our Vietnam".
- deleted 3y ago[deleted]
- max_ 3y agoIn highschool a friend of mine's dad worked in traffic police. He opened his dad's computer and found so many pictures of dead road accident victims. He described some to me and I found it very disturbing just via verbal description. But he didn't seem disturbed at all.
- patrulek 3y agoI dont see why he should be disturbed at all. Seeing something irl and the same thing on a screen has completely different effect for some people.
- gumballindie 3y agoSome of us are old to remember ogrish.com. Let’s just say I dont ride motorbikes thanks to that website.
- spacechild1 3y agoI did my civil service as a paramedic near a curvy mountain road. I don't ride motorcycles thanks to that...
- bertil 3y agoThat’s probably the most horrifying flip side of that debate. There’s genuine physical abuse at a scale that would make every genocide, or any pandemic look tame, but it’s all dismissed because car companies represent a material amount of ad revenue. But because the images of it (not the reality of losing a loved one, or the consequence of losing a parent) are gory, there’s nothing we can do to highlight we should care about more than ad budgets.
- Havoc 3y agoOne of the few jobs which I'd just never do fullstop no matter how desperate. Going to war seems to mess up a good % of people. This stuff seems to have an even higher rate (near 100%?). >Like other sacked moderators, she confessed to a feeling of withdrawal at being deprived of the graphic content she had grown accustomed to. Quite surprised about that part - hadn't heard that before in other articles about this topic. Maybe that's what drives the people who post all this crap in the first place
- deleted 3y ago[deleted]
- geraldhh 3y agocould we like, pin archive links for paywalled content?
- rapidaneurism 3y agoI would consider any opportunity that pays per hour as much as one makes per day, a highly paid one. Perhaps the higher pay reflects the horrible parts of the job.
- camillomiller 3y agoI would like to point out that this has been a typical recurring high profile reportage, reporting on exactly the same problem, for at least a decade. ABSOLUTELY NOTHING has changed. Not even in Europe, where the case exploded in Germany in 2016. You can translate this piece from the SZ from 2016 to have an idea. https://sz-magazin.sueddeutsche.de/internet/im-netz-des-boesen-85994 https://sz-magazin.sueddeutsche.de/internet/im-netz-des-boes... As someone who reports on media and tech, I have to say: the way nothing changes with Meta and social networks even when their unethical and evil practices are exposed is one of the most frustrating aspects of my career.
- wslh 3y agoOnly a brief comment: I imagine many (and specially the top) social media companies will be sued in the near future like Tobacco companies in the past [1]. It is just that regulations, laws, and politics are slow to adapt to an extremely dynamic world. Here I am not pointing to specific politics, politicians or ideology spectrum but the field of politics itself. [1] https://en.wikipedia.org/wiki/Tobacco_Master_Settlement_Agreement?wprov=sfti1 https://en.wikipedia.org/wiki/Tobacco_Master_Settlement_Agre...
- justinl33 3y agoAm I the only one wondering how any half capable AI classifier would be unable to identify a literal beheading video or man having s*x with a fish? And especially at a cost that would justify using manual labellers? I get that these human agents are most likely providing labelled training data as opposed to actual filtering but still, surely on more edge-case footage where the model is not confident: not things that could very easily and confidently be classified as inappropriate.
- Culonavirus 3y agoHow do you know that already isn't happening? When we're talking about FB, X, Youtube, we're talking about (tens of?) billions of multimedia posts a year. "Edge-case" will still be a massive amount. A quick Google tells me about 1% of the population are psychopaths. There are 80 million people on the planet that have no empathy or conscience. And that's just one angle we could look at this from. The point is, there's a truckload of fucked up shit on the net and AI isn't that good yet.
- justinl33 3y ago> The point is, there's a truckload of fucked up shit on the net and AI isn't that good yet. I would beg to differ. AI is definitely good enough to identify fucked up things. Feed GPT-4 even a subtly fcked up question, or fcked up image and it'll be onto you immediately When I said edge cases, I wasn't arguing that there wouldn't be any edge cases - I'm saying the human agents were being shown videos that were not edge cases - i.e sexual abuse, gore - if it's bad enough to traumatize a human being, it's also high signal enough for a model to pick it up. Yes, human agents should be used for labelling ACTUAL edge cases where a model might be confused - i.e is this a nerf gun fight or a real gun fight, is this person nude or is it just their shirt... not a plain beheading video.
- schroeding 3y agoI would very much hope that the available training data for "man fucking fish" is pretty sparse, maybe too sparse to build a training set. (I'm not as optimistic about the sparseness of beheading videos, though)
- coldblues 3y agoWhat companies are hiring for this kind of stuff? Do they pay well? Serious question. Email on my profile.
- nuc1e0n 3y agoIt seems to me the algorithm being followed where these moderators are introduced to this horrible content maximises the anguish it causes. There's no proper training in advance to fortify them against it and they're straight into the most horrible things from the first day on the job, which is inhumane. Separating the wheat from the chaff of content is difficult and admirable work and these people should be properly supported and rewarded for the risks they take from moderating it on our behalf. A lot of these people are also quite nieve about horrors of the world. Are they really the best ones to do this?
- bertil 3y agoThis article, and every comment on here, every discourse about that problem, takes this tsunami of shocking images as a natural, exogenous phenomenon. No one ever asks: Who shares those? Why? Do they ever stop? Can we discourage them from doing that? People uploading this have reasons to do so. You can talk to them. I did.
- e40 3y agoTease much??? What did they say?
- bertil 3y agoLook around and ask them
- AlienRobot 3y agoOkay. Why are you posting this? If you aren't going to tell us, why even post this? Share the info.
- bertil 3y agoAsk them.
- AlienRobot 3y agoI just did!
- bertil 3y agoUnlike what you are implying, I never posted anything offensive. If you want me to help you with your User experience research, let me know where to send the invoice.
- giantg2 3y agoI've wondered about a career in digital forensics, but I assume it's at least 50% child porn shit and I don't want to deal with or see that.
- batch12 3y agoIt depends. With law enforcement, probably. Abother option working for a cybersecurity firm and focus on post-intrusion forensics, etc. Also, working in incident response for a large enough company will give you this experience too. Source- I held a role responsible for response and some forensics for years.
- giantg2 3y agoYeah, I mostly meant law enforcement. I probably couldn't reskill at this point anyways. I had a couple security courses including a forensics course in grad school. But it seems most places want people to be experts and don't offer any training. Things like CISSP don't seem to have very good hands-on training.
- batch12 3y agoThere are some podcasts like the DFIR podcast that can get you started with some concepts. I agree with the CISSP. That's more of a management level certification and focuses a lot on risk and other topics. It's a mile wide and an inch deep. If you're looking for other training, there are some SANS ($$$) classes or you can look into general courses that will teach you to use things like FTK or others like Magnet Axiom, Encase, etc. If you're interested, I know some folks deeper into forensics than I that can give me some more specific courses if you'd like- let me know.
- giantg2 3y agoThanks but I don't think I'll switch.
- akudha 3y agoWhile reading stories like these, one thing that depresses me the most is this - how little humanity/empathy these suits at corporations have. Sure, their primary (probably only) goal is to make money, but that doesn't mean they have to treat their employees/contractors like trash. Costco has shown repeatedly that it is possible to build a good business, while treating employees/suppliers etc decently. If Costco can do it in retail business (where the margins aren't as lucrative as software business), why can't Meta? It feels as though they go out of their way to be as nasty as possible. Or live in some kind of weird bubble - like this arsehole for example https://www.businessinsider.com/panera-founder-workers-not-motivated-making-money-shareholders-ceo-therapy-2023-12 https://www.businessinsider.com/panera-founder-workers-not-m...
- newswasboring 3y agoI don't understand why everyone is hating on this guy. He didn't say it in a "can you believe it?!!!" way. It was a matter of fact statement about his understanding of motivation of his employees. The tiktokers are satirizing an attitude which was never displayed.
- dudeinjapan 3y agoWhat he actually said: "So much of what running a business is about is figuring out 'how do I connect with people?'" Shaich told BI. "What motivates them, and how do I help them decide to affiliate with what the mission of the enterprise is?" He said therapy is about better understanding ourselves, where we come from, and our own motivations. In turn, that can build our ability to form alliances and find commonalities with other people. "And that makes you a much more powerful manager," Shaich said. "Every one of us brings our own humanity to it. Similarly, I think it makes us more empathetic if we start by being empathetic to ourselves and learning and thinking about ourselves. We're able to be more empathetic, not just to our team members and the people we work with, but also to our customers." ...sounds like a total arsehole to me!
- kylecazar 3y agoI'm actually missing the point as well. It sounds to me like he is agreeing with the sentiment in those memes, that employees aren't motivated by making owners richer -- and you need to empathize with them to understand them.
- hsuduebc2 3y agohttps://archive.is/AT6JK https://archive.is/AT6JK
- mschuster91 3y agoPersonally, as someone active in Twitter's Birdwatch / Community Notes, I don't even want to imagine what the actual paid moderators get to see. What CN gets is a ton of propaganda (mostly from Russian-backed troll farms and their Western accomplices) and only a few bits of gore (most of that purported to be from the current I/P conflict, but actually from other, sometimes many years old wars and conflicts). And that's bad enough.
- hsuduebc2 3y agoThis sounds like early industrialization in England. The way they approach their workers. It's awful and ridiculous that Facebook don't want to pay them 2.2$. Corporations are really necessarily evil. The depersonalization of people which came with these big structures generally is making most of the problems. People usually are not psychos when dealing with people. Usually.
- jampekka 3y agoCorporations are not evil. They just don't care at all about e.g. human suffering or destruction of environment they cause to maximize return on capital. From human perspective they are amoral or chaotic neutral. Like Cthulhu.
- kayodelycaon 3y ago> They just don't care at all about e.g. human suffering or destruction of environment they cause to maximize return on capital. I think you have just described evil.
- jampekka 3y agoIs an asteroid hitting Earth evil?
- NoZebra120vClip 3y agoI am no longer permitted to post images to my Facebook feed. I am also experiencing glitches with Messenger. I am not sure why, but I can't expect Support to be of any help here. Back in 2015, I participated in a genuine knighting ceremony, and it was re-enacted for photo opportunities. So there I was, on my knees receiving a sword to my shoulders from a knight in full regalia. Of course I eagerly posted the photo to Facebook and it was immediately rejected, I assume by AI only, because it was "male pornography". Yeah, I suppose I could see how the confusion arises there. Unfortunately the false-positives of Trust & Safety can be a real hindrance to those of us who have genuine moments to share with others.
- AlienRobot 3y agoIt seems everyone will ban you for anything with no recourse because they expect bad actors to just create new accounts every time. The answer to this is, ironically, and even officially in Google's case, that good actors should just create a new account as well if they get unfairly banned.
- zoklet-enjoyer 3y agoGoatse, tubgirl, rotten, ogrish were viewed by most of my friend group in middle/high school. Sometimes on school computers.
- jackfoxy 3y agoDamn long-form journalism takes forever to get to the point, so my skimming sometimes misses something crucial. With that disclaimer out of the way, (I think) it is distressing and damaging for young people to be doing this job. (And maybe the rest of the article goes on to say something like that.) Still in today's world it is a job that has to be done. If it is to be done it should be by older more mature people with more life experience under their belts. And it should be revealed up front what the job is and how damaging it can be. I think you would have to be very spiritually grounded and take many breaks to refresh your spirit (mental health) to do this.
- ogurechny 3y agoYoung people suffering “Internet's worst horrors”? You mean the social network services? Those exploitative weepy articles about content moderation are just multi-layered hypocrisy. First of all, it is supposed that people are somehow entitled to “good experiences”, and the arbitrary (and moving) cultural boundary between “suitable” and “unsuitable” is as unquestionable as if it was God-given, when in fact, it's just a feature of an entertainment direct-to-screen service available to those from the luckier parts of the world/society. Others deal with the feces for them, as shown here. Then there's the hypocrisy of the reader, who enjoys the thrills articles like those give, but also enjoys the service too much to stop, and leave the system of exploitation. Like, “It's so awful, so awful, but I need my daily dose of filtered cat pictures, so you're gonna get that sad dickpick spam in my stead. It's just the world we live in! The algorithm makes me continue doing that!”, etc. Then there's the talk about values, correctness, and so on, but those decisions are not even personal in the first place. The “SFW” facade is not supported by some die-hard conservatives in power, it is just a business requirement. Say, breastfeeding is considered “problematic” not because of some “clash of cultures”, or “gender conflict”, or “religious opposition”, but because you can't use someone's body in that context to hold advertisements, as stated in contracts. Now go hide yourself in a ditch somewhere, don't spoil our pretty picture. Money happens to be religion here (what an original thought). Some time ago, people naively believed in the future that eliminates travails and death, now we pretend really hard that the “future” is now, and build all kinds of media and social contraptions to make someone else do the “dirty” work (real or “emotional”).
- MerManMaid 3y ago>Then there's the hypocrisy of the reader, who enjoys the thrills articles like those give, but also enjoys the service too much to stop, and leave the system of exploitation. Like, “It's so awful, so awful, but I need my daily dose of filtered cat pictures, so you're gonna get that sad dickpick spam in my stead. It's just the world we live in! The algorithm makes me continue doing that!”, etc. I get you're angry but if everyone who complains about how awful social media can be while they themselves participate in social media are hypocrites and therefore shouldn't speak up (because... ???), how do you honestly see society being able to address the problem? Heck even now, both you and I are participating in a limited form of social media, aren't we to hypocrites for complaining about the harm given your interpretation? I mean sure we aren't looking at "filtered cat pictures" (or maybe we are :P) but I don't see how what we're doing here is fundamentally different.
- Denote6737 3y agoTo everyone saying AI is the solution to this. You are right but only kind of. Much of the work of modern AI is due to backend labelling and tagging of datasets by humans on services like Mechanical Turk. So to train the model you have to get hundreds of thousands, or even millions of images of the kind you wish to filter. Then get thousands of humans to look at them and label them. These people will suffer just the same as those interviewed in this article.