5 ms·

> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through. Agai

> but as Wired notes, it relies on "automated tools" as part of the process, which may have been how the more than 50 ads TTP discovered made it through. Again, and again, and again, these technologies have proven ineffective to moderate content. User posted and otherwise. Until we have a legal mechanism in place to hold these companies meaningfully liable for this, it will not stop. Every single one of these has the same damn list of excuses about cost of hiring human moderators, but I promise everyone reading this if you make the executives of a social network legally liable for distributing CSAM on their platforms, they will find all kinds of money to get human moderators really fucking quick.

duped 2mo ago
But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. --- This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.
- john_strinlai 2mo ago
  >But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. you say sarcasm, but i think this is literally true. this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
  - lokar 2mo ago
    Don’t they have like an 80% gross margin?
    - john_strinlai 2mo ago
      the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that. it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale. (again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
      - gambiting 2mo ago
        Right, but the article isn't about posts, but about a paid ad campaign - surely those are started in numbers that could be moderated?
        john_strinlai 2mo ago
        >Right, but the article isn't about posts, but about a paid ad campaign the comment that started this thread says: "these technologies have proven ineffective to moderate content. User posted and otherwise" which is what i (and i assumed the parent to my comment) was replying to. i agree that paid ad campaigns alone is more tractable.
        wmf 2mo ago
        No, because most ads are created by automated systems (many of which are scam/spam ads) and Meta doesn't want to lose revenue by slowing the system down to human speed.
      - deleted 2mo ago
        [deleted]
      - 2mo ago
alex1138 2mo ago
Yeah and/but also the case it's not bidirectional enough The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down
- ToucanLoucan 2mo ago
  > The eternal question of tech platforms - should we be responsible for what users post? I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible. Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts. Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
  - alex1138 2mo ago
    Yeah sure, I think we basically agree I'm just bemoaning the fact FB can't even pretend they care when they're so busy banning people for nothing
    - alex1138 2mo ago
      Including this by the way Part a) https://news.ycombinator.com/item?id=24201306 https://news.ycombinator.com/item?id=24201306 part b) https://news.ycombinator.com/item?id=24776748 https://news.ycombinator.com/item?id=24776748
malshe 2mo ago
I think people have been demanding this for ages but lobbying by social media companies against any change in the law is enormous. As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.
lstodd 2mo ago
FB might or might not have a right to exist in someone's eyes, but the question remains: is 0.5c-like latency messaging across the world acceptable? Because facebook or not, given internets this is what you will end up with. People are people. On the other hand, now try to ban that telecom. Gods will laugh.
danudey 2mo ago
The article also mentions that a ton of even worse ads were posted between when they contacted Meta and when they were ready to go live with the article, and some of the offending ads which were taken down had just been re-uploaded again and automatically approved. Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.
bigbuppo 2mo ago
More specifically, these were for ads. They received compensation to distribute this matierial.
ratelimitsteve 2mo ago
We used to say computers can't make decisions because they can't meaningfully be held responsible for those decisions. Now the vibe seems to be that computers should make decisions because instead of anyone being held responsible we can just say "the computer made a mistake and it is very sorry" and effectively shield anyone from being responsible. But the thing is, machines don't do stuff. Not from a responsibility standpoint. My car doesn't drive, I do. It may provide the means, but I decide where we're going and how we're going there. Even with the active cruise on, the car didn't speed, I did. The car may have hit the wall, but I caused the accident. the car ran that person over, but I killed someone, not the car. The machine did what someone told it to do. If someone told it to do a bad thing, or someone did a bad job telling it not to a bad thing, that person is responsible. When people started using Grok to make CSAM the platform's response was to paywall image generation. If we don't start making the driver responsible for what the car does, the coder responsible for what the code does, the CEO responsible for what the company does, they will continue to exploit people in increasingly horiffic ways.

duped 2mo ago

But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. --- This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.

john_strinlai 2mo ago
>But they couldn't possibly do it at that the scale they're operating at. They couldn't hire enough people in the countries they operate, or know the language, or trust that who they do hire aren't corrupt or working against their interest. And they can't manually approve every ad, or make sure targeting isn't illegal or otherwise harmful. you say sarcasm, but i think this is literally true. this isn't a defense of facebook, i think that their scale makes it actually impossible (while remaining a going concern). the result should be that they arent allowed to operate at a scale that they cant handle, instead of being able to brush it under the rug every ~6 months when an incident hits the news cycle.
- lokar 2mo ago
  Don’t they have like an 80% gross margin?
  - john_strinlai 2mo ago
    the amount of content posted per second to facebook is unfathomable. a quick google (just for a rough order of magnitude) says ~65,000 posts per second. add in instagram (which is worse for CSAM) and its gotta be at least double that. it would require millions on millions of employees, spread across ~every country. between the payroll, buildings and property, equipment, etc. i just dont think its possible at that scale. (again, emphasizing that im not saying this as an excuse for facebook, but that i dont really think facebook/instagram should exist at all)
    - gambiting 2mo ago
      Right, but the article isn't about posts, but about a paid ad campaign - surely those are started in numbers that could be moderated?
      - john_strinlai 2mo ago
        >Right, but the article isn't about posts, but about a paid ad campaign the comment that started this thread says: "these technologies have proven ineffective to moderate content. User posted and otherwise" which is what i (and i assumed the parent to my comment) was replying to. i agree that paid ad campaigns alone is more tractable.
      - wmf 2mo ago
        No, because most ads are created by automated systems (many of which are scam/spam ads) and Meta doesn't want to lose revenue by slowing the system down to human speed.
    - deleted 2mo ago
      [deleted]
    - 2mo ago

alex1138 2mo ago

Yeah and/but also the case it's not bidirectional enough The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down

ToucanLoucan 2mo ago
> The eternal question of tech platforms - should we be responsible for what users post? I would never say a platform has to be responsible for what a user posts, and to be clear, I mean either a user of the platform, or a user of the platform's ad tooling, or anything else. Anything that arrives via an upload button of some sort. However: If you as a platform then distribute what has been posted, yes, you are responsible for it. Your service is now broadcasting that content. If that content is harmful, you should be held responsible. Who else conceivably could be? And if the response to this is "well we have way too much stuff coming in to moderate it" okay, then restrict how much stuff comes in. Require approval on individual posts or on accounts. Like, this is such basic shit. If a newspaper printed CSAM on the front page, they would be arrested within the day. But because Facebook is real super big they just don't have to take care of their house? Ridiculous.
- alex1138 2mo ago
  Yeah sure, I think we basically agree I'm just bemoaning the fact FB can't even pretend they care when they're so busy banning people for nothing
  - alex1138 2mo ago
    Including this by the way Part a) https://news.ycombinator.com/item?id=24201306 https://news.ycombinator.com/item?id=24201306 part b) https://news.ycombinator.com/item?id=24776748 https://news.ycombinator.com/item?id=24776748

malshe 2mo ago

I think people have been demanding this for ages but lobbying by social media companies against any change in the law is enormous. As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.

lstodd 2mo ago

FB might or might not have a right to exist in someone's eyes, but the question remains: is 0.5c-like latency messaging across the world acceptable? Because facebook or not, given internets this is what you will end up with. People are people. On the other hand, now try to ban that telecom. Gods will laugh.

danudey 2mo ago

The article also mentions that a ton of even worse ads were posted between when they contacted Meta and when they were ready to go live with the article, and some of the offending ads which were taken down had just been re-uploaded again and automatically approved. Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.

bigbuppo 2mo ago

More specifically, these were for ads. They received compensation to distribute this matierial.

ratelimitsteve 2mo ago

We used to say computers can't make decisions because they can't meaningfully be held responsible for those decisions. Now the vibe seems to be that computers should make decisions because instead of anyone being held responsible we can just say "the computer made a mistake and it is very sorry" and effectively shield anyone from being responsible. But the thing is, machines don't do stuff. Not from a responsibility standpoint. My car doesn't drive, I do. It may provide the means, but I decide where we're going and how we're going there. Even with the active cruise on, the car didn't speed, I did. The car may have hit the wall, but I caused the accident. the car ran that person over, but I killed someone, not the car. The machine did what someone told it to do. If someone told it to do a bad thing, or someone did a bad job telling it not to a bad thing, that person is responsible. When people started using Grok to make CSAM the platform's response was to paywall image generation. If we don't start making the driver responsible for what the car does, the coder responsible for what the code does, the CEO responsible for what the company does, they will continue to exploit people in increasingly horiffic ways.