4 ms·

Where Facebook's AI Slop Comes From

71 points by colinprince 2y ago

snowwrestler 2y ago
Long but absolutely worth the read. People in other countries with much lower costs of living are essentially working full time to post meaningless crap to Facebook, because the “creator” payouts from Facebook, while tiny by U.S. standards, are meaningful sums in these other countries. And this has in turn spawned “how to get rich quick” influencer content which itself is also monetized the same way, as well as pretty sophisticated software to automate it all. This story has surprisingly altered the way I feel about this AI spam on Facebook. While it’s disconcerting and annoying content to look at… I have to give props to these folks for finding clever ways to suck money out of lazy rich tech companies. Respect for the international hustle. The part of the ecosystem missing from this story is the revenue stream to Facebook. Are advertisers actually ok with their ads showing up next to this crap, and getting charged by Facebook for the privilege? Maybe? Not all ads are run by big conservative image-conscious consumer companies. If ads get real clicks, maybe that’s enough for some advertisers, regardless of how silly the AI content next door.
- ToucanLoucan 2y ago
  I mean there's always an advertiser ready to advertise on your site. Low-rent ads have been a staple of the internet experience since there was an internet. I remember crawling music piracy websites as a young lad (satirically in the video game minecraft) and seeing all kinds of horrendous shit advertised. Pills that make your dick bigger, hot russian singles in my area, y'all know the rest. Twitter's had a remarkable fall from grace in this particular regard, what with it once being very normal to find extremely high-budget and noteworthy products advertised there, to now being almost 100% grift in one form or another: crypto, AI shlock, NFTs, the works. That said I will say, having grown up with that culture as a part of me, when I see websites that roll ads like those, it definitely makes me much more apprehensive about engaging with them, especially in commerce. So it might well be true that there is always an advertiser ready to advertise, but in so doing, people pay attention to what you're willing to put up next to your content and judge your site/platform accordingly.
- pants2 2y ago
  I despise all platforms with "creator payouts." Influencers churning out hyper-optimized slop will flood your platform and drown out real creators who are actually posting original content. If you do manage to post good content, expect 100s of bots to repost your stuff endlessly and steal what little earnings you might've had.
  - myaccountonhn 2y ago
    Genuine creators earning a living from these platforms also get punished for not putting the exact content the algorithm has deemed the people want to see. Friend of mine who works as an influencer was condemned to forever record herself putting things on a shelf, since any other content she made wouldn't even be shown. I think about that when I see some streamers who get famous for playing a certain game. If they play anything else, it doesn't generate the same buzz so they are just stuck playing the same game for hours on end. Feels like a miserable job imo.
jazzyjackson 2y ago
good reporting but what I really want to see is a profile on the kind of people who actually like and share this content, let alone follow the page. Are they kids that think it's funny? senile pensioners who think that's actually a photo of a wounded veteran on his birthday? are the people of facebook OK?
- deleted 2y ago
  [deleted]
- yifanl 2y ago
  Seems obvious to me that once the incentive is made that maximizing views at all cost occurs, the platform will (slowly or quickly) eventually cater only to the types of people most eager to provide views. Are they okay or not isn't even the right question, it's more like, is a site that mainlines their eyeballs a reasonable platform to exist? (Revenue numbers seem to say no, but something something irrational markets)
- barryrandall 2y ago
  They're okay. But they want to believe a lot of things, and the market for telling people what they want to hear is large and evergreen.
CM30 2y ago
Ah, it's the same issue that every user created content site and service with a payout system has. Because the cost of living is much lower in other countries, there's an incentive for people there to exploit these systems with useless crap in an attempt to get rich quick. I saw it on forums with revenue sharing all the way back in the 90s and 00s. You had people from all kinds of less well off countries signing up and posting junk en masse because the tiny amount that Adsense paid out was a king's ransom in those parts of the world. And it's been a constant ever since. 'Web 3.0' games that paid people to play had these people turn them into full-time jobs, Quora has an endless deluge of junk being posted because of their contributor program, and the same is true of everything from Medium to YouTube to TikTok and Instagram. Any time there's a way to 'make money from home', then it turns into a get rich quick scheme/full-time business for those in low cost regions or with few other options. But hey, it's inevitable. If posting junk online pays better than local businesses and jobs do, then of course you're going to see an increasing number of people trying to get in on the hustle.
- lupire 2y ago
  It's an economic welfare program. Maybe not the most efficient, but it's very Keynesian. Giving poor people money improves the overall economy.
  - ToucanLoucan 2y ago
    Except we could just do that, because giving people the resources they need to live is a good and moral thing to do, and just giving it to them comes with the added side benefit of not having them turn the Internet into an utter cesspool of meaningless shit.
    - jazzyjackson 2y ago
      Seems click fraud is an elaborate way to fool the wealthy into thinking they're getting something out of the trade We had a movement that encouraged the wealthy to become altruistic and give most effectively and look how that ended up in other news, the head of Bagledesh's interim government is the microfinance dude, "access to credit is a human right", I used kiva.org quite a bit back in the day, felt much less depressing than gofundme, like people's future might actually be looking up
yimmothathird 2y ago
Pretty much what I expected. It was very obvious to me it was just engagement hacking for money when I saw damn near every one has a list of irrelevant celebrity hashtags attached at the end. Still, there are ones even stranger than the trash AI images targeting the attention of idiots. I saw a post seemingly advertising a real book, but for some reason, in the content of the post. The claimed the book was written by [Person A] while the photo of the book they posted clearly noted the author as [Person B]. Didn't seem like a simple accident either. The actual author had a name that looked to be of middle eastern origin, while the fale author was some west-African insert.
lfmunoz4 2y ago
non paywall link? https://web.archive.org/web/20240806165521/https://www.404media.co/where-facebooks-ai-slop-comes-from/ https://web.archive.org/web/20240806165521/https://www.404me...
ilrwbwrkhv 2y ago
This is fantastic! The hacker in me really loves the thinking, coordination and arbitrage these guys are doing. More power to them.
iamleppert 2y ago
So Facebook is basically paying people in third world countries to spam people with low quality AI content? Isn't this technically advertiser fraud? They are diluting the value of the content on the platform, thereby decreasing the signal to noise ratio. I can't imagine the effect on advertisers can be good, unless you're Temu peddling more junk.
- advisedwang 2y ago
  It's not fraud if the people viewing the AI junk are real people actually seeing the ads. Facebook doesn't make a promise about the quality of the content alongside the ads. Of course you are still right that advertisers don't want to be seen next to junk. I also believe/hope that people will not actually sit scrolling through pages of AI garbage so there will also be less users/views over time.
bradleykingz 2y ago
[dead]
manx 2y ago
The question now becomes: how to classify useful content. I doubt that's possible without downvotes and some more advanced algorithms than majority voting. Right now, those systems are full of probably unintended feedback loops.
manx 2y ago
So many stakeholders with different incentives: advertisers, facebook, regular users, paid content creators, ... How could we design incentives which produce a positive outcome for everyone?

71 points by colinprince 2y ago

Long but absolutely worth the read. People in other countries with much lower costs of living are essentially working full time to post meaningless crap to Facebook, because the “creator” payouts from Facebook, while tiny by U.S. standards, are meaningful sums in these other countries. And this has in turn spawned “how to get rich quick” influencer content which itself is also monetized the same way, as well as pretty sophisticated software to automate it all. This story has surprisingly altered the way I feel about this AI spam on Facebook. While it’s disconcerting and annoying content to look at… I have to give props to these folks for finding clever ways to suck money out of lazy rich tech companies. Respect for the international hustle. The part of the ecosystem missing from this story is the revenue stream to Facebook. Are advertisers actually ok with their ads showing up next to this crap, and getting charged by Facebook for the privilege? Maybe? Not all ads are run by big conservative image-conscious consumer companies. If ads get real clicks, maybe that’s enough for some advertisers, regardless of how silly the AI content next door.

ToucanLoucan 2y ago
I mean there's always an advertiser ready to advertise on your site. Low-rent ads have been a staple of the internet experience since there was an internet. I remember crawling music piracy websites as a young lad (satirically in the video game minecraft) and seeing all kinds of horrendous shit advertised. Pills that make your dick bigger, hot russian singles in my area, y'all know the rest. Twitter's had a remarkable fall from grace in this particular regard, what with it once being very normal to find extremely high-budget and noteworthy products advertised there, to now being almost 100% grift in one form or another: crypto, AI shlock, NFTs, the works. That said I will say, having grown up with that culture as a part of me, when I see websites that roll ads like those, it definitely makes me much more apprehensive about engaging with them, especially in commerce. So it might well be true that there is always an advertiser ready to advertise, but in so doing, people pay attention to what you're willing to put up next to your content and judge your site/platform accordingly.
pants2 2y ago
I despise all platforms with "creator payouts." Influencers churning out hyper-optimized slop will flood your platform and drown out real creators who are actually posting original content. If you do manage to post good content, expect 100s of bots to repost your stuff endlessly and steal what little earnings you might've had.
- myaccountonhn 2y ago
  Genuine creators earning a living from these platforms also get punished for not putting the exact content the algorithm has deemed the people want to see. Friend of mine who works as an influencer was condemned to forever record herself putting things on a shelf, since any other content she made wouldn't even be shown. I think about that when I see some streamers who get famous for playing a certain game. If they play anything else, it doesn't generate the same buzz so they are just stuck playing the same game for hours on end. Feels like a miserable job imo.

jazzyjackson 2y ago

good reporting but what I really want to see is a profile on the kind of people who actually like and share this content, let alone follow the page. Are they kids that think it's funny? senile pensioners who think that's actually a photo of a wounded veteran on his birthday? are the people of facebook OK?

deleted 2y ago
[deleted]
yifanl 2y ago
Seems obvious to me that once the incentive is made that maximizing views at all cost occurs, the platform will (slowly or quickly) eventually cater only to the types of people most eager to provide views. Are they okay or not isn't even the right question, it's more like, is a site that mainlines their eyeballs a reasonable platform to exist? (Revenue numbers seem to say no, but something something irrational markets)
barryrandall 2y ago
They're okay. But they want to believe a lot of things, and the market for telling people what they want to hear is large and evergreen.

CM30 2y ago

Ah, it's the same issue that every user created content site and service with a payout system has. Because the cost of living is much lower in other countries, there's an incentive for people there to exploit these systems with useless crap in an attempt to get rich quick. I saw it on forums with revenue sharing all the way back in the 90s and 00s. You had people from all kinds of less well off countries signing up and posting junk en masse because the tiny amount that Adsense paid out was a king's ransom in those parts of the world. And it's been a constant ever since. 'Web 3.0' games that paid people to play had these people turn them into full-time jobs, Quora has an endless deluge of junk being posted because of their contributor program, and the same is true of everything from Medium to YouTube to TikTok and Instagram. Any time there's a way to 'make money from home', then it turns into a get rich quick scheme/full-time business for those in low cost regions or with few other options. But hey, it's inevitable. If posting junk online pays better than local businesses and jobs do, then of course you're going to see an increasing number of people trying to get in on the hustle.

lupire 2y ago
It's an economic welfare program. Maybe not the most efficient, but it's very Keynesian. Giving poor people money improves the overall economy.
- ToucanLoucan 2y ago
  Except we could just do that, because giving people the resources they need to live is a good and moral thing to do, and just giving it to them comes with the added side benefit of not having them turn the Internet into an utter cesspool of meaningless shit.
  - jazzyjackson 2y ago
    Seems click fraud is an elaborate way to fool the wealthy into thinking they're getting something out of the trade We had a movement that encouraged the wealthy to become altruistic and give most effectively and look how that ended up in other news, the head of Bagledesh's interim government is the microfinance dude, "access to credit is a human right", I used kiva.org quite a bit back in the day, felt much less depressing than gofundme, like people's future might actually be looking up

yimmothathird 2y ago

Pretty much what I expected. It was very obvious to me it was just engagement hacking for money when I saw damn near every one has a list of irrelevant celebrity hashtags attached at the end. Still, there are ones even stranger than the trash AI images targeting the attention of idiots. I saw a post seemingly advertising a real book, but for some reason, in the content of the post. The claimed the book was written by [Person A] while the photo of the book they posted clearly noted the author as [Person B]. Didn't seem like a simple accident either. The actual author had a name that looked to be of middle eastern origin, while the fale author was some west-African insert.

lfmunoz4 2y ago

non paywall link? https://web.archive.org/web/20240806165521/https://www.404media.co/where-facebooks-ai-slop-comes-from/ https://web.archive.org/web/20240806165521/https://www.404me...

ilrwbwrkhv 2y ago

This is fantastic! The hacker in me really loves the thinking, coordination and arbitrage these guys are doing. More power to them.

iamleppert 2y ago

So Facebook is basically paying people in third world countries to spam people with low quality AI content? Isn't this technically advertiser fraud? They are diluting the value of the content on the platform, thereby decreasing the signal to noise ratio. I can't imagine the effect on advertisers can be good, unless you're Temu peddling more junk.

advisedwang 2y ago
It's not fraud if the people viewing the AI junk are real people actually seeing the ads. Facebook doesn't make a promise about the quality of the content alongside the ads. Of course you are still right that advertisers don't want to be seen next to junk. I also believe/hope that people will not actually sit scrolling through pages of AI garbage so there will also be less users/views over time.

bradleykingz 2y ago

[dead]

manx 2y ago

The question now becomes: how to classify useful content. I doubt that's possible without downvotes and some more advanced algorithms than majority voting. Right now, those systems are full of probably unintended feedback loops.

manx 2y ago

So many stakeholders with different incentives: advertisers, facebook, regular users, paid content creators, ... How could we design incentives which produce a positive outcome for everyone?