10 ms·

A lot of attention gets paid to sex abuse content, which it should, but there is much less I feel for all sorts of other abhorrent content. Gore, violence (phys

A lot of attention gets paid to sex abuse content, which it should, but there is much less I feel for all sorts of other abhorrent content. Gore, violence (physical/mental), terrorism, death, destruction, shock content, etc. It's not just NSFW, but also NSFL content that I think is under discussed at times. Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. It's unfortunate the world is a messed up place, and maybe it will stay that way as long as humans are around. This is the exact type of content that I think AI is so crucial for detecting, and it's a bit sad that we often hear so about all the bad things you can generate with AI and so little about all the good. If we need to train models on this type of content to make it easier to detect and remove, to prevent mentally scarring people, then personally I'm all for it -- regardless of the "it can also generate that bad content" cost.

osobo 3y ago
I've been online since the late eighties and over time I've seen some nasty NSFL stuff. I am absolutely certain this has shaped my world view to some degree. Like some other GenX-er said: "Internet is our Vietnam".
- meindnoch 3y ago
  >Internet is our Vietnam Probably don't say this in front of actual veterans.
  - Jerrrry 3y ago
    >actual veterans >actual Probably don't say this
    - deleted 3y ago
      [deleted]
    - ghodith 3y ago
      As opposed to 4chan veterans?
      - Jerrrry 3y ago
        im a veteran of the war on drugs. i served cod:b0 - mw3 now give me my wendys discount please
      - I_Am_Nous 3y ago
        I remember the civil war between those who accepted Queen Boxxy and those who rejected her outright
  - qwertox 3y ago
    Agreed, there's a stark difference between seeing someone getting decapitated on screen and then closing the tab instead of seeing this IRL and being scared of being the next one in line or realizing that the person standing next to you is just doing this. This is referring to the "GenX" who watched this out of his own free will, which is also not comparable to what the people in this article have to deal with.
  - Der_Einzige 3y ago
    Vietnam was a bad war that shouldn’t have happened and anyone who went without being drafted is not worth respecting. The hippies were right.
- stinos 3y ago
  > I am absolutely certain this has shaped my world view to some degree On the other hand many people, if I had to guess I'd say the vast majority, have not seen any of that stuff on the internet. I have, but because I actively went looking for it. Likewise if I ask around amongst friends the only ones who saw the real gore all admitted having actively searched for it. In other words: even before we saw it there might have been something different in our world view already.
  - viraptor 3y ago
    It depends how terminally online you are too. Even not actively looking for it, you'll eventually see it. Probably less likely these days than on the wild west internet ~y2k. But visiting enough forums and going to enough personal pages at the time eventually put you in front of some NSFL content. Or at least a link that you didn't actively look for but now have an option of clicking purely out of curiousity.
    - stinos 3y ago
      visiting enough forums That's sort of the point though: we do that, but whereas around y2k forums were the online communities, I don't exactly have the impression that is still the case. And it's not like you're going to see much NSFL content on, say, TikTok.
    - RF_Savage 3y ago
      At this current moment it is surprisingly hard at times to avoid combat footage from drones. It still sure is a wild west in many places.
  - resupply1554 3y ago
    > In other words: even before we saw it there might have been something different in our world view already. Agreed. I was brought up in a troubled home and spent a lot (if not all) of my childhood/early teens on 4chans /b/ board as a result (mid-late 2000s, early 10s). Looking back at it, it seems like it was THE place that brought together all the damaged individuals. The stuff I've seen there desensitized me to a basically every disgusting/horrid/tragic thing I had to witness later on in life, other than losses of loved ones in of themselves.
eurekin 3y ago
To provide a more expanded horizon: it was a hobby of a peculiar group of kids in my middle school (and earlier). They teased each other, who can be more cool and unfazed (eat a sandwich for example), while watching it. One of them is a paramedic now. I imagine, surgeons might go through similar character development.
- geraldhh 3y ago
  > character development pretty much sums it up; anything that does not break you, will make you stronger. ever chewed on your food, breathing thru the nose, while cleaning diarrhea diapers and being screamed at? your parents did.
  - BLKNSLVR 3y ago
    Man that stirs up some great memories. Sometimes memories are great merely for the fact that they're the past ;)
    - geraldhh 3y ago
      exactly
  - not_alexb 3y ago
    > anything that does not break you, will make you stronger. Not even remotely true. Plenty of people have to spend years in therapy to undo their parent's trauma. Let's put it another way, do you think a rape victim in college will have better outcomes than someone who is not raped in college?
    - deleted 3y ago
      [deleted]
    - tester756 3y ago
      >anything that does not break you I think in your cases it "broke" them
startupsfail 3y ago
This is not where the mainstream is heading. Content like that is illegal to store in some jurisdictions, handling it invokes GDPR issues, no one in their right mind would touch it with a long stick. LAION case is a good recent example. This could be a reasonable business for a company in the countries where it is not illegal to moderate content like that.
intended 3y ago
We are using AI and GenAI to get better at detection, training and policy. However, There can never exist a tool that can correctly detect harmful content. You would need the intent of a submission to get to that level of accuracy. Everything else is: AI filters -> Human review. Your offenders are 2 types: Bad actors and Adversaries. Both evolve to find ways around your detection processes. Detection and obfuscation are linked.
geraldhh 3y ago
totally support the "outsource soul-crushing jobs to Machines". Thou we have to be careful not to outlaw content or viewpoints else our collective ability to cope will diminish.
muragekibicho 3y ago
Related to your point - Meta got sued by the human team employed to detect gore, violence and death. They fired the entire human content team at their biggest content moderation site in Nairobi, Kenya after the team fought to unionize. Here's the story - https://techcrunch.com/2022/03/30/meta-and-sama-face-legal-action-in-kenya-for-alleged-poor-work-conditions/ https://techcrunch.com/2022/03/30/meta-and-sama-face-legal-a...
- muragekibicho 3y ago
  I think big tech companies are pushing towards AI to avoid somewhat similar court cases. I can't imagine how *less stressful* it is for employers who avoid legal tussles with their subcontractors. In this case, Meta bore the brunt of their contractor, Samasource's, underdealings!
  - dbspin 3y ago
    Large tech companies employ subcontractors specifically to take flak for this kind of thing. They save money, but more importantly deflect criticism by allowing subcontractors to engage in employment practices that break their own policies. Your sympathy should never default to a large wealthy company, but to the exploited, usually under paid, employees and 'contractors' forced to do unpalatable things out of the eye of regulation.
    - Snow_Falls 3y ago
      See: "we didn't know we had slaves making our clothes, it was those dastardly factories that we contracted with!"
      - hackerlight 3y ago
        There's an important distinction between what Meta is doing here and other types of low wage work such as clothes manufacturing. We have to consider where the work lies on the spectrum of voluntary vs coercive. The problem with Meta's gore content moderation job is that the people signing up for it didn't know what they were getting themselves into. Or if they did, they weren't aware of the long term health consequences (but Meta surely were). So it's closer to coercive than voluntary. Meta were abusing the information asymmetry. If Meta gave each new hire a dossier outlining how bad the health impacts will be before the contract is signed, then I wouldn't have a problem with it, because anyone who signs up knows what they're getting into. But Meta doesn't do that. They obscure this for new hires in order to exploit the information asymmetry.
ycdxvjp 3y ago
The issue is less about detection and more about the sheer volume of content. Even if your false positives and negative rates are low the numbers are huge. Which is why everyone ends up hiring more and more people as content volume increases. What has changed over the years is - the cost to spam the entire world with zillion messages a second is tending to ZERO. While this is an achievement it's also a disaster cause it's akin to everyone having an almost free high power broadcast transmitter sitting on their roof. We regulate and auction radio spectrum and don't allow everyone on the planet to broadcast simultaneously on what ever frequency they want. Cause the noise would make signal drop to 0. But on the internet it's allowed and it's almost free.
- j16sdiz 3y ago
  Agree. At this volume, nobody could possibly win. Social networks need to break down quite a bit to get something manageable.
- phkahler 3y ago
  >> What has changed over the years is - the cost to spam the entire world with zillion messages a second is tending to ZERO. One solution to that is whitelisting. But we don't want 3rd party censorship, so a federated system where we rate sources and content ourselves seems like a good idea. Unfortunately kids won't filter anything - they are drawn to the unusual and forbidden.
  - chongli 3y ago
    How is whitelisting going to work? You can’t whitelist Instagram photos because that would prevent people from posting new ones. You also can’t whitelist users because bad actors just hack their accounts. All of the disallowed content I’ve come across (pornography on Facebook or Instagram) was posted by an obviously hacked account. If you add an extra checkmark people have to get to be able to post then hackers will just target those people.
    - shadowgovt 3y ago
      It turns out this is actually a very hard problem which is why centralized social media companies get it wrong so often (and why that might still be preferable to the alternatives).
Podgajski 3y ago
I agree everything you said. This is horrific and no person should have to go through seeing all that disgusting material for my benefit of using the Internet. It’s really something that pushes me further and further away from using the Internet. You can put it on par to driving an electric car, knowing that the lithium was mined by nine and 10-year-olds in Africa somewhere. I’m not a religious man, but I think I’m becoming one. Mostly because I feel the appearance of all this stuff correlates with the decline of religion. I only knew which one to choose that didn’t have as much behind it as the Internet does I would, for surely be a devout follower.
- chubs 3y ago
  I’ve heard it said that the existence of objective evil is only explained by the existence of objective good. And yeah… some of what happens in the world just has to be a step beyond subjective evil. Please stay curious :)
  - Snow_Falls 3y ago
    But no one can agree on what's good, let alone what's evil.
- zztop44 3y ago
  “All this stuff” correlates with a lot of other things too. And is happening all over the world, including places where religion is in rude health indeed.
- coryrc 3y ago
  Lithium comes from Australia, Chile, and Bolivia. My EV's batteries have no elements from Africa.
CM30 3y ago
I agree that it's the ideal use case for the AI, it's just the issue there is that it's an area where you can't afford any false positives or negatives. All hell will break loose if sexual abuse content or footage of terrorism/cartel incidents ends up on random people's Facebook timelines, and another kind of hell will break loose if innocent people see their accounts nuked and their details reported to authorities because the system thought something they posted was illegal content. We've already had both things happen in recent times, and it's been a legal and PR nightmare for affected companies. So while using AI would be great to avoid harm to any human moderators, the system would have to be perfect (or nearly perfect) in order to avoid any side effects that would cause serious harm to innocent people and the company/platform in question.
- gpderetta 3y ago
  It could be an additional line of defence. It would also ideally run on your user agent.
- pixl97 3y ago
  >where you can't afford any false positives I disagree with that part of the statement. Companies would just defer to the AI rather than human choices and say "Well, computer says no"
  - CM30 3y ago
    Sadly you're probably not wrong here. But the consequences for that sort of attitude when it comes to this particular field are extremely high, and I dread to think of the outcomes in said cases.
    - pixl97 3y ago
      Oh, we already predicted the outcome years ago... https://en.wikipedia.org/wiki/Computers_Don%27t_Argue https://en.wikipedia.org/wiki/Computers_Don%27t_Argue
- Denote6737 3y ago
  So long as we aren't using the AI to punish the posters. Vis-a-vis the google incident last year with the guy loosing his entire account because of telemedicine. I'm okay with false positives. You took down my video of a sphinx cat because your AI fagged it as NSFW. fine. I can't think of a scenario where random people being able to post things getting blocked occasionally for things being incorrectly flagged is a risk. The risk comes when they then use that to ban and delete accounts. If they just lock the account in a non-destructive way, i.e. block new posting/shadow ban. and reverse it on appeal. This is of course excluding the use of such tools in a non-good faith way.
sBqQu3U0wH 3y ago
[flagged]
- deleted 3y ago
  [deleted]
sneak 3y ago
> Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. What is the source for this seemingly widely held belief? Bad images on a screen aren’t inherently harmful to many people. I grew up (10+ hours a day for a decade) on the edgy, uncensored internet, and I don’t think it broke me or harmed me in any way.
- throwanem 3y ago
  I too have met the guy at the party who can pound vodka shots like water and never notice a difference. I'd say "I don't go home with guys like that any more", but I was never quite naïve enough to go home with guys like that in the first place.
  - sneak 3y ago
    We can measure objectively the damage ethanol does to the tissues in the body, both short and long term, and the impairment it causes to judgement. We have studied its use both intermittently and chronic, and its effects are well-understood. Can we say the same about bad images on a screen? All of this “if you look at enough of them you become broken” seems like folk “wisdom” to me. I don’t want to be the guy holding up the “[citation needed]” sign, but I am truly skeptical of this claim. Just because it’s gross and unpleasant to look at doesn’t mean that lots of exposure causes permanent damage.
    - throwanem 3y ago
      Yes, and just because it's impossible ethically to construct a study showing one way or the other doesn't mean that it doesn't.
  - rvba 3y ago
    OP is writing to completely ban vodka though.
    - throwanem 3y ago
rvba 3y ago
Ah yes censorship, "for the good of children", that will be later used on everyone else 1984 style. Allready is. Not sure if you ever been a child, but kids will be searching for porn and violence on their own
huytersd 3y ago
Eh it’s all a matter of perspective. We get used to animal gore with no mental side effects. If violence is normalized in your society and not directed at you directly (for instance in ancient Mayan/incan societies), it not going to bother you all that much.
- Snow_Falls 3y ago
  Do we get used to animal gore though? Do you know why there are legal time limits on how long you can work at an abbatoir? Its because anyone who works there for more than ~3 years tends to become a serial killer. We're used to seeing meat because its out of the context of a slaughtered animal., in the same way that a mortician is used to seeing corpses in caskets, but would freak out if there was one in their shed. If people had to kill their own cows, I imagine vegetarianism would take off.
  - coryrc 3y ago
    I do not think anything you said is true. Most places do not have "time limits". I find no evidence of abattoir workers becoming more likely to be serial killers. People used to kill their own animals and weren't vegetarians.
    - Der_Einzige 3y ago
      I think all of what they said is true and I trivially found sources for all their claims. Try harder next time
      - wetmore 3y ago
        Could you elaborate here? Spurred by you and the person you're replying to painting such different pictures of what sources are out there, I took a look myself. I didn't find it so trivial to find sources backing up the person's claims. > Most places do not have "time limits". I couldn't find anything about slaughterhouses having lifelong limits for employees and would love a link or search term to use because it's an interesting concept to me. > I find no evidence of abattoir workers becoming more likely to be serial killers. I also could not find evidence for this. Here is a meta analysis of many studies, none of which mention slaughterhouse workers being more likely to be serial killers. Some studies show higher crime rates in towns with slaughterhouses, but not for violent crimes. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10009492/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10009492/ > People used to kill their own animals and weren't vegetarians. This is trivially true?
zaik 3y ago
> the bad things you can generate with AI and so little about all the good Generative AI and detection/classification are very different. The bad news of the last few months are almost exclusively about generative AI. Your good example is a detection AI.
wegfawefgawefg 3y ago
[flagged]
- lttlrck 3y ago
  The is so ignorant I don't know where to start. Look up PSTD, the causes and the broad spectrum of people it affects without them even realizing what's happening, the years of suffering it can cause. "probably" smh.
  - wegfawefgawefg 3y ago
    PTSD is real. I once fled a forest fire. Thats not what im talking about. I'm referring to the assumption that looking at porn, or violent pictures will "absolutely break you". Which I don't believe is the case. Maybe if the person was really into an orthodox conservative religion I could see the possibility of an extreme stress response from seeing the pictures. But in that case its their brainwashing, not the pictures that caused the trauma.
    - Nevermark 3y ago
      Human being are simply not that uniform. Not even close.
      - wegfawefgawefg 3y ago
        Of course, but I think the average person is actually more emotionally resilient, easily desensitized, and capable of killing than the average American generally assumes. History has plenty of examples of this. Modern times do as well. If filtering these was decently well payed, and was a job that had been around for 50 years, 99% of them would be totally fine. Just clock in and clock out. No it is not guaranteed to give every person ptsd if they see gore, which is what the person who started this comment chain claimed, and was what I was responding to.
johnnyanmac 3y ago
I think its undertalked about because platforms regulate that pretty well. If only because adverts would absolutely hate being associated with any of that. Meanwhile, sexuality is much more ephemeral and easier to slip in. >This is the exact type of content that I think AI is so crucial for detecting, and it's a bit sad that we often hear so about all the bad things you can generate with AI and so little about all the good most companies care about making money, so there's a lot more focus on how it can streamline the workflow/replace labor. Not as much business on how to use it as an auto-moderator.
shrimp_emoji 3y ago
> Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. At the end of the day, we evolved to endure much worse than sitting at a computer and watching stuff happen, and our ancestors have. If anything, being too coddled and sheltered is a way to produce way more harm. Even being raped is more traumatic when you're socially trained to expect to be traumatized.
xkcd1963 3y ago
There has always been violence, its just that with internet and a gazillion of cameras everything becomes more visible

osobo 3y ago

I've been online since the late eighties and over time I've seen some nasty NSFL stuff. I am absolutely certain this has shaped my world view to some degree. Like some other GenX-er said: "Internet is our Vietnam".

meindnoch 3y ago
>Internet is our Vietnam Probably don't say this in front of actual veterans.
- Jerrrry 3y ago
  >actual veterans >actual Probably don't say this
  - deleted 3y ago
    [deleted]
  - ghodith 3y ago
    As opposed to 4chan veterans?
    - Jerrrry 3y ago
      im a veteran of the war on drugs. i served cod:b0 - mw3 now give me my wendys discount please
    - I_Am_Nous 3y ago
      I remember the civil war between those who accepted Queen Boxxy and those who rejected her outright
- qwertox 3y ago
  Agreed, there's a stark difference between seeing someone getting decapitated on screen and then closing the tab instead of seeing this IRL and being scared of being the next one in line or realizing that the person standing next to you is just doing this. This is referring to the "GenX" who watched this out of his own free will, which is also not comparable to what the people in this article have to deal with.
- Der_Einzige 3y ago
  Vietnam was a bad war that shouldn’t have happened and anyone who went without being drafted is not worth respecting. The hippies were right.
stinos 3y ago
> I am absolutely certain this has shaped my world view to some degree On the other hand many people, if I had to guess I'd say the vast majority, have not seen any of that stuff on the internet. I have, but because I actively went looking for it. Likewise if I ask around amongst friends the only ones who saw the real gore all admitted having actively searched for it. In other words: even before we saw it there might have been something different in our world view already.
- viraptor 3y ago
  It depends how terminally online you are too. Even not actively looking for it, you'll eventually see it. Probably less likely these days than on the wild west internet ~y2k. But visiting enough forums and going to enough personal pages at the time eventually put you in front of some NSFL content. Or at least a link that you didn't actively look for but now have an option of clicking purely out of curiousity.
  - stinos 3y ago
    visiting enough forums That's sort of the point though: we do that, but whereas around y2k forums were the online communities, I don't exactly have the impression that is still the case. And it's not like you're going to see much NSFL content on, say, TikTok.
  - RF_Savage 3y ago
    At this current moment it is surprisingly hard at times to avoid combat footage from drones. It still sure is a wild west in many places.
- resupply1554 3y ago
  > In other words: even before we saw it there might have been something different in our world view already. Agreed. I was brought up in a troubled home and spent a lot (if not all) of my childhood/early teens on 4chans /b/ board as a result (mid-late 2000s, early 10s). Looking back at it, it seems like it was THE place that brought together all the damaged individuals. The stuff I've seen there desensitized me to a basically every disgusting/horrid/tragic thing I had to witness later on in life, other than losses of loved ones in of themselves.

eurekin 3y ago

To provide a more expanded horizon: it was a hobby of a peculiar group of kids in my middle school (and earlier). They teased each other, who can be more cool and unfazed (eat a sandwich for example), while watching it. One of them is a paramedic now. I imagine, surgeons might go through similar character development.

geraldhh 3y ago
> character development pretty much sums it up; anything that does not break you, will make you stronger. ever chewed on your food, breathing thru the nose, while cleaning diarrhea diapers and being screamed at? your parents did.
- BLKNSLVR 3y ago
  Man that stirs up some great memories. Sometimes memories are great merely for the fact that they're the past ;)
  - geraldhh 3y ago
    exactly
- not_alexb 3y ago
  > anything that does not break you, will make you stronger. Not even remotely true. Plenty of people have to spend years in therapy to undo their parent's trauma. Let's put it another way, do you think a rape victim in college will have better outcomes than someone who is not raped in college?
  - deleted 3y ago
    [deleted]
  - tester756 3y ago
    >anything that does not break you I think in your cases it "broke" them

startupsfail 3y ago

This is not where the mainstream is heading. Content like that is illegal to store in some jurisdictions, handling it invokes GDPR issues, no one in their right mind would touch it with a long stick. LAION case is a good recent example. This could be a reasonable business for a company in the countries where it is not illegal to moderate content like that.

intended 3y ago

We are using AI and GenAI to get better at detection, training and policy. However, There can never exist a tool that can correctly detect harmful content. You would need the intent of a submission to get to that level of accuracy. Everything else is: AI filters -> Human review. Your offenders are 2 types: Bad actors and Adversaries. Both evolve to find ways around your detection processes. Detection and obfuscation are linked.

geraldhh 3y ago

totally support the "outsource soul-crushing jobs to Machines". Thou we have to be careful not to outlaw content or viewpoints else our collective ability to cope will diminish.

muragekibicho 3y ago

Related to your point - Meta got sued by the human team employed to detect gore, violence and death. They fired the entire human content team at their biggest content moderation site in Nairobi, Kenya after the team fought to unionize. Here's the story - https://techcrunch.com/2022/03/30/meta-and-sama-face-legal-action-in-kenya-for-alleged-poor-work-conditions/ https://techcrunch.com/2022/03/30/meta-and-sama-face-legal-a...

muragekibicho 3y ago
I think big tech companies are pushing towards AI to avoid somewhat similar court cases. I can't imagine how *less stressful* it is for employers who avoid legal tussles with their subcontractors. In this case, Meta bore the brunt of their contractor, Samasource's, underdealings!
- dbspin 3y ago
  Large tech companies employ subcontractors specifically to take flak for this kind of thing. They save money, but more importantly deflect criticism by allowing subcontractors to engage in employment practices that break their own policies. Your sympathy should never default to a large wealthy company, but to the exploited, usually under paid, employees and 'contractors' forced to do unpalatable things out of the eye of regulation.
  - Snow_Falls 3y ago
    See: "we didn't know we had slaves making our clothes, it was those dastardly factories that we contracted with!"
    - hackerlight 3y ago
      There's an important distinction between what Meta is doing here and other types of low wage work such as clothes manufacturing. We have to consider where the work lies on the spectrum of voluntary vs coercive. The problem with Meta's gore content moderation job is that the people signing up for it didn't know what they were getting themselves into. Or if they did, they weren't aware of the long term health consequences (but Meta surely were). So it's closer to coercive than voluntary. Meta were abusing the information asymmetry. If Meta gave each new hire a dossier outlining how bad the health impacts will be before the contract is signed, then I wouldn't have a problem with it, because anyone who signs up knows what they're getting into. But Meta doesn't do that. They obscure this for new hires in order to exploit the information asymmetry.

ycdxvjp 3y ago

The issue is less about detection and more about the sheer volume of content. Even if your false positives and negative rates are low the numbers are huge. Which is why everyone ends up hiring more and more people as content volume increases. What has changed over the years is - the cost to spam the entire world with zillion messages a second is tending to ZERO. While this is an achievement it's also a disaster cause it's akin to everyone having an almost free high power broadcast transmitter sitting on their roof. We regulate and auction radio spectrum and don't allow everyone on the planet to broadcast simultaneously on what ever frequency they want. Cause the noise would make signal drop to 0. But on the internet it's allowed and it's almost free.

j16sdiz 3y ago
Agree. At this volume, nobody could possibly win. Social networks need to break down quite a bit to get something manageable.
phkahler 3y ago
>> What has changed over the years is - the cost to spam the entire world with zillion messages a second is tending to ZERO. One solution to that is whitelisting. But we don't want 3rd party censorship, so a federated system where we rate sources and content ourselves seems like a good idea. Unfortunately kids won't filter anything - they are drawn to the unusual and forbidden.
- chongli 3y ago
  How is whitelisting going to work? You can’t whitelist Instagram photos because that would prevent people from posting new ones. You also can’t whitelist users because bad actors just hack their accounts. All of the disallowed content I’ve come across (pornography on Facebook or Instagram) was posted by an obviously hacked account. If you add an extra checkmark people have to get to be able to post then hackers will just target those people.
  - shadowgovt 3y ago
    It turns out this is actually a very hard problem which is why centralized social media companies get it wrong so often (and why that might still be preferable to the alternatives).

Podgajski 3y ago

I agree everything you said. This is horrific and no person should have to go through seeing all that disgusting material for my benefit of using the Internet. It’s really something that pushes me further and further away from using the Internet. You can put it on par to driving an electric car, knowing that the lithium was mined by nine and 10-year-olds in Africa somewhere. I’m not a religious man, but I think I’m becoming one. Mostly because I feel the appearance of all this stuff correlates with the decline of religion. I only knew which one to choose that didn’t have as much behind it as the Internet does I would, for surely be a devout follower.

chubs 3y ago
I’ve heard it said that the existence of objective evil is only explained by the existence of objective good. And yeah… some of what happens in the world just has to be a step beyond subjective evil. Please stay curious :)
- Snow_Falls 3y ago
  But no one can agree on what's good, let alone what's evil.
zztop44 3y ago
“All this stuff” correlates with a lot of other things too. And is happening all over the world, including places where religion is in rude health indeed.
coryrc 3y ago
Lithium comes from Australia, Chile, and Bolivia. My EV's batteries have no elements from Africa.

CM30 3y ago

I agree that it's the ideal use case for the AI, it's just the issue there is that it's an area where you can't afford any false positives or negatives. All hell will break loose if sexual abuse content or footage of terrorism/cartel incidents ends up on random people's Facebook timelines, and another kind of hell will break loose if innocent people see their accounts nuked and their details reported to authorities because the system thought something they posted was illegal content. We've already had both things happen in recent times, and it's been a legal and PR nightmare for affected companies. So while using AI would be great to avoid harm to any human moderators, the system would have to be perfect (or nearly perfect) in order to avoid any side effects that would cause serious harm to innocent people and the company/platform in question.

gpderetta 3y ago
It could be an additional line of defence. It would also ideally run on your user agent.
pixl97 3y ago
>where you can't afford any false positives I disagree with that part of the statement. Companies would just defer to the AI rather than human choices and say "Well, computer says no"
- CM30 3y ago
  Sadly you're probably not wrong here. But the consequences for that sort of attitude when it comes to this particular field are extremely high, and I dread to think of the outcomes in said cases.
  - pixl97 3y ago
    Oh, we already predicted the outcome years ago... https://en.wikipedia.org/wiki/Computers_Don%27t_Argue https://en.wikipedia.org/wiki/Computers_Don%27t_Argue
Denote6737 3y ago
So long as we aren't using the AI to punish the posters. Vis-a-vis the google incident last year with the guy loosing his entire account because of telemedicine. I'm okay with false positives. You took down my video of a sphinx cat because your AI fagged it as NSFW. fine. I can't think of a scenario where random people being able to post things getting blocked occasionally for things being incorrectly flagged is a risk. The risk comes when they then use that to ban and delete accounts. If they just lock the account in a non-destructive way, i.e. block new posting/shadow ban. and reverse it on appeal. This is of course excluding the use of such tools in a non-good faith way.

sBqQu3U0wH 3y ago

[flagged]

deleted 3y ago
[deleted]

sneak 3y ago

> Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. What is the source for this seemingly widely held belief? Bad images on a screen aren’t inherently harmful to many people. I grew up (10+ hours a day for a decade) on the edgy, uncensored internet, and I don’t think it broke me or harmed me in any way.

throwanem 3y ago
I too have met the guy at the party who can pound vodka shots like water and never notice a difference. I'd say "I don't go home with guys like that any more", but I was never quite naïve enough to go home with guys like that in the first place.
- sneak 3y ago
  We can measure objectively the damage ethanol does to the tissues in the body, both short and long term, and the impairment it causes to judgement. We have studied its use both intermittently and chronic, and its effects are well-understood. Can we say the same about bad images on a screen? All of this “if you look at enough of them you become broken” seems like folk “wisdom” to me. I don’t want to be the guy holding up the “[citation needed]” sign, but I am truly skeptical of this claim. Just because it’s gross and unpleasant to look at doesn’t mean that lots of exposure causes permanent damage.
  - throwanem 3y ago
    Yes, and just because it's impossible ethically to construct a study showing one way or the other doesn't mean that it doesn't.
- rvba 3y ago
  OP is writing to completely ban vodka though.
  - throwanem 3y ago

rvba 3y ago

Ah yes censorship, "for the good of children", that will be later used on everyone else 1984 style. Allready is. Not sure if you ever been a child, but kids will be searching for porn and violence on their own

huytersd 3y ago

Eh it’s all a matter of perspective. We get used to animal gore with no mental side effects. If violence is normalized in your society and not directed at you directly (for instance in ancient Mayan/incan societies), it not going to bother you all that much.

Snow_Falls 3y ago
Do we get used to animal gore though? Do you know why there are legal time limits on how long you can work at an abbatoir? Its because anyone who works there for more than ~3 years tends to become a serial killer. We're used to seeing meat because its out of the context of a slaughtered animal., in the same way that a mortician is used to seeing corpses in caskets, but would freak out if there was one in their shed. If people had to kill their own cows, I imagine vegetarianism would take off.
- coryrc 3y ago
  I do not think anything you said is true. Most places do not have "time limits". I find no evidence of abattoir workers becoming more likely to be serial killers. People used to kill their own animals and weren't vegetarians.
  - Der_Einzige 3y ago
    I think all of what they said is true and I trivially found sources for all their claims. Try harder next time
    - wetmore 3y ago
      Could you elaborate here? Spurred by you and the person you're replying to painting such different pictures of what sources are out there, I took a look myself. I didn't find it so trivial to find sources backing up the person's claims. > Most places do not have "time limits". I couldn't find anything about slaughterhouses having lifelong limits for employees and would love a link or search term to use because it's an interesting concept to me. > I find no evidence of abattoir workers becoming more likely to be serial killers. I also could not find evidence for this. Here is a meta analysis of many studies, none of which mention slaughterhouse workers being more likely to be serial killers. Some studies show higher crime rates in towns with slaughterhouses, but not for violent crimes. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10009492/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10009492/ > People used to kill their own animals and weren't vegetarians. This is trivially true?

zaik 3y ago

> the bad things you can generate with AI and so little about all the good Generative AI and detection/classification are very different. The bad news of the last few months are almost exclusively about generative AI. Your good example is a detection AI.

wegfawefgawefg 3y ago

[flagged]

lttlrck 3y ago
The is so ignorant I don't know where to start. Look up PSTD, the causes and the broad spectrum of people it affects without them even realizing what's happening, the years of suffering it can cause. "probably" smh.
- wegfawefgawefg 3y ago
  PTSD is real. I once fled a forest fire. Thats not what im talking about. I'm referring to the assumption that looking at porn, or violent pictures will "absolutely break you". Which I don't believe is the case. Maybe if the person was really into an orthodox conservative religion I could see the possibility of an extreme stress response from seeing the pictures. But in that case its their brainwashing, not the pictures that caused the trauma.
  - Nevermark 3y ago
    Human being are simply not that uniform. Not even close.
    - wegfawefgawefg 3y ago
      Of course, but I think the average person is actually more emotionally resilient, easily desensitized, and capable of killing than the average American generally assumes. History has plenty of examples of this. Modern times do as well. If filtering these was decently well payed, and was a job that had been around for 50 years, 99% of them would be totally fine. Just clock in and clock out. No it is not guaranteed to give every person ptsd if they see gore, which is what the person who started this comment chain claimed, and was what I was responding to.

johnnyanmac 3y ago

I think its undertalked about because platforms regulate that pretty well. If only because adverts would absolutely hate being associated with any of that. Meanwhile, sexuality is much more ephemeral and easier to slip in. >This is the exact type of content that I think AI is so crucial for detecting, and it's a bit sad that we often hear so about all the bad things you can generate with AI and so little about all the good most companies care about making money, so there's a lot more focus on how it can streamline the workflow/replace labor. Not as much business on how to use it as an auto-moderator.

shrimp_emoji 3y ago

> Seeing the worst output of humanity non-stop can (and _will_) completely break you mentally if consumed at a sufficient amount, as we are at the end of the day humans. At the end of the day, we evolved to endure much worse than sitting at a computer and watching stuff happen, and our ancestors have. If anything, being too coddled and sheltered is a way to produce way more harm. Even being raped is more traumatic when you're socially trained to expect to be traumatized.

xkcd1963 3y ago

There has always been violence, its just that with internet and a gazillion of cameras everything becomes more visible