5 ms·
AI-generated spam is starting to fill social media. Here's why
- pjc50 3y agoYes, this is one of the reasons people have been so hostile to AI: it's the ultimate spam engine.
- dmix 3y agoThe ultimate spam engine would be one that figures out how to bypass account creation and reputation systems. Just like here on HN, some random can create a hundred accounts but their green username posting AI junk isn’t going to rank them. Nor will the various spam detection algorithms around networks and user behaviour be suddenly bypassed. Nor will user reporting if they do get ranked. The “write believable texts” thing is only one small portion of a very complicated problem. The only real usecase is a human using it to juice their own feeds or paying decent $$ for real accounts (there aren’t that many on the market IRL). Which isn’t automated spam that we need to fear taking over the internet. That doesn’t scale.
- afavour 3y agoIMO a bigger problem is that the social media giants will be perfectly happy to allow AI-generated content as long as it makes them money. My Facebook feed these days is a graveyard of nonsense loosely connected to things I clicked on once. The AI generated text is useful filler for them.
- deleted 3y ago[deleted]
- pjc50 3y ago> their green username posting AI junk isn’t going to rank them You're going to see a substantial overlap between the least original users and the smartest AI, assuming the AI people actually deliver on their improvement promises. They see their job as making "Turing test" discrimination non-viable. This kind of thing leaves you with an increasing proportion of "users" on the platform who aren't real but have garnered enough crumbs of interest and upvote to appear real.
- erikerikson 3y agohttps://xkcd.com/810/ https://xkcd.com/810/
- the_snooze 3y agoThe general publicly-accessible internet will turn into AI grey goo. The real value will retreat to invite-only private chats where human-scale social norms maintain a high signal-to-noise ratio. Asking a friend to recommend a good burger place is way more productive than sifting through the sludge of "best burgers near me" on Google and the like.
- blackeyeblitzar 3y agoQuestion: if the future is everyone having agents and assistants, won’t this be expected and even okay? Why do I need to lose time if my agent can represent my voice? And if so does that mean all social media will lose anonymity to avoid spam or influence campaigns? Or is there some middle ground?
- klyrs 3y agoIf social media is read and written by agents, can we just put our phones down and get back to living in meatspace? Perhaps it's the best thing to ever happen to social media.
- tdb7893 3y agoSocial media is already a lot of spam and influence campaigns and people still use it. I think it'll keep chugging along as it has been with large amounts of spam and then AI to fight the spam.
- spacebanana7 3y agoThere are ways of maintaining a chain of trust without losing total anonymity. For example DNS, email and Marshall Island companies. Karma systems and IP reputation can come into play too. (I know the examples aren't truly anonymous, but they're all broadly comparable to using a Twitter without your real name).
- tdb7893 3y agoThe internet already isn't truly anonymous with how people actually use it. People are logged into a bunch of accounts that are traceable to them and also rarely mask their IP. With how much stuff I've paid for over the internet there's probably a double digit number of companies with my entire name and current address.
- blackeyeblitzar 3y agoI just mean they may force you to share ID documents to make an account. I have trouble believing I would be anonymous at that point forward?
- malfist 3y agoSo we really need an article explaining why? It's obvious, because the spammer profits from it
- kayodelycaon 3y agoMoney isn't necessarily the goal. It could be someone building an audience to spread disinformation to affect public opinion. The fact it's happening is reason enough for the article to exist.
- kube-system 3y ago> But many of the pages don't have a clear financial motivation, Goldstein said. They seem to simply be accumulating an audience for unknown purposes.
- deleted 3y ago[deleted]
- itslennysfault 3y agoProbably just to sell the account. There is a market for social media accounts that already have 1000s of followers.
- notTooFarGone 3y agoThis is like saying "startups have no clear financial motivation, they just gather customers and bleed money" The purpose is always money and influence. There's rarely someine in "for the lulz"
- ck2 3y agoNow just imagine a month or two before November elections in USA It's going to be an absolute sh*t show even on the one or two platforms that do actually want to stop it
- ceejayoz 3y agoI'm presuming a lot of these pages exist to accrue followers now and start posting disinformation closer to November, yeah.
- deleted 3y ago[deleted]
- slowmovintarget 3y agoHere's the amplification of the noise over signal we expected from AI. Scams, spam... all expected. It makes you wonder if there's not more at work like active sabotage. There will come a point where no one is able to believe anything. When that happens, consensus reality goes out the window. Massive social upheaval follows. We know Tik Tok was used specifically for these effects. Where is all this "content" coming from? Does Meta (for example) even know? Is it all from a handful of accounts?
- pc86 3y ago> We know Tik Tok was used specifically for these effects. I know no such thing, and this is the first I'm hearing of anyone saying it's used for social upheaval, denial of consensus reality, or sabotage.
- ceejayoz 3y agoPoe's Law? https://finance.yahoo.com/news/us-spy-chief-cannot-rule-162722979.html https://finance.yahoo.com/news/us-spy-chief-cannot-rule-1627... > Lawmakers have long voiced concerns that the Chinese government could access user data or influence what people see on the app, including pushing content to stoke U.S. political divisions. https://www.npr.org/2024/04/26/1247347363/china-tiktok-national-security https://www.npr.org/2024/04/26/1247347363/china-tiktok-natio... > National security is at the heart of bipartisan concerns in Washington motivating the law. Lawmakers say they're worried the Chinese government could lean on ByteDance in order to use TikTok to suck up Americans' data, surveil them, and spread false and misleading claims to U.S. voters.
- TremendousJudge 3y agoyeah, only American companies should be allowed to do that!
- ceejayoz 3y agoI mean, China's way ahead on this one. https://apnews.com/article/tiktok-bytedance-ban-china-india-376f32d78861e14e65ec4bc78e808a0d https://apnews.com/article/tiktok-bytedance-ban-china-india-... > TikTok has never been available in mainland China, a fact that CEO Shou Chew has mentioned in testimony to U.S. lawmakers. ByteDance instead offers Chinese users Douyin, a similar video-sharing app that follows Beijing’s strict censorship rules. TikTok also ceased operations in Hong Kong after a sweeping Chinese national security law took effect.
- idontwantthis 3y agoThe mutilated children asking for birthday wishes is beyond the pale. That got me to leave facebook.
- neallindsay 3y agoLive by the engagement-optimized timeline, die by the engagement-optimized timeline.
- neverokay 3y agoI’m becoming more of the mind that humans communicate more like birds or whales. There’s a general sound or frequency that everyone is humming. The spam isn’t fodder, it’s the collective “song” of humans at the moment. I don’t think people say anything unique, and LLMs are proving it. We’re all kinda humming the same tunes. It’s why you won’t read anything new here or anywhere, you’ll read the same old story or hear the same old song. I guess I’m saying humans in general communicate with spam, a buzz.
- simianparrot 3y agoSounds like you’re not out in the “real world” enough if you think that. At least it’s absolutely not what I observe offline. But online sure; because most of it is algorithm driven
- neverokay 3y agoI wouldn’t read too much into it. Most of it is coming from why LLMs even hallucinate. In the simplest terms I’m just spitting out Noun noun verb verb noun etc And so are you If you try to read into what the actual noun was, you’ll miss the forest for the trees. People are mostly humming the same tunes as they always have been humming and filling the same roles they have always been filling. Just take our exchange: Opinion Retort Another Retort ——- We’re just dancing. You think there’s something unique going on here but there isn’t, hah. It’s a dance and pattern both me and you know from wherever, and we are sorta humming the song out. Not too different when we try to figure out if there is a real magic to an LLM output.
- c22 3y agoIt's fine to study the forest, but it's also fun to look at individual trees. On another level we are deterministic chemical reactions and all language is just flapping meat sounds. So what?
- KittenInABox 3y ago> I’m becoming more of the mind that humans communicate more like birds or whales. There’s a general sound or frequency that everyone is humming. sorry to um actually here... but neither whales nor birds all have a "general sound" the way you're describing. Both whales and birds actually have strict regional differences and grammars in their singing and communication patterns.
- bena 3y agoBecause it's easy to do and gives people a sense of creative expression. It gives them an easy "sense of pride and accomplishment".
- deleted 3y ago[deleted]
- ChrisArchitect 3y agoRelated: Slop is the new name for unwanted AI-generated content https://news.ycombinator.com/item?id=40301490 https://news.ycombinator.com/item?id=40301490
- skybrian 3y agoI wonder why Facebook doesn't just turn off these recommendations until they fix it?
- pjc50 3y agoThey can still show ad views against them, and have long abandoned any "is the user actually satisfied with their feed?" metrics.
- bashinator 3y agoBecause it drives engagement.
- karaterobot 3y ago> "She'd be like, 'Wouldn't this be so cool for your daughter?'" McVay said. "And I'm like, 'That's not real, though.'" Other than not being able to buy an AI generated item, what's the functional difference between a clickbait image generated by AI, and one generated by a human? My assumption is that people endlessly scrolling for dopamine hits are not overly concerned with the epistemology of it all. In these circumstances, something is true or real if it is amusing or distracting, because it's all just content, and that is what content is for. AI is so good at generating content that this is probably exactly the right use for it.
- pjc50 3y ago> In these circumstances, something is true or real if it is amusing or distracting, because it's all just content "Ten Hours of Jingling Keys" Perhaps, but ultimately replacing meaning with meaningless isn't going to go well? These people still have votes.
- karaterobot 3y agoFair enough but: how's it going giving people regular clickbait? How's it going trying to get people to stop caring about clickbait? My point is that I can't really see the difference between what we already have and what we're going to have with AI, other than that AI will do it better. The idea that meaning matters in social networks does not pass the smell test for me; I haven't been able to confirm it.
- llamaimperative 3y agoIt's probably a good idea in general to have clear-ish boundaries between real and not real unless someone is knowingly opting into an experience that is designed to blur those boundaries.
- ChoHag 3y ago[dead]
- hobotime 3y agoAi assisted rage farming is something I'm part of and it scares the crap out of me. Nothing gets attention like opposite AI personalities raging against each other.
- bashinator 3y agoCare to elaborate? Are you a rage farmer, or trying to mitigate them?
- hobotime 3y agoRage farmer. It's sucks how well it works. Basically, we have AI personas with one side always over-reaching and looking slightly stupid. This is to bolster the side we think is correct. Works really well and we're swaying the opinion of a rather local community.
- bashinator 2y agoThanks for your honesty there. Why do you do it, just the money? A cause you actually believe in?
- lawlessone 3y agoIs it hard to figure out why? It's cheap and easy.