6 ms·

A peek into Reddit's anti-spam internals

185 points by OuterVale 3mo ago

ingvay7 3mo ago
Neat rabbit hole. Reminds me of having to deal with email spam - it was a similar deal with rule-based filters, ML scores, domain bans,IP filtering, browse fingerprinting etc and mishmash of ever evolving scripts surviving org and personnel changes. Glad i dont deal with it anymore as the frontier seems to be 2 fronts now with human and agentic spam.
Terr_ 3mo ago
Damn, maybe I can finally find out why my 10+ year account was globally (and retroactively) shadowbanned, even though the appeal was allegedly granted. In the past, those post removals didn't even exist in the moderation log, so perhaps a reason could give me a clue... On the other hand, I'm taking a kind of emotional damage just remembering.
- qingcharles 3mo ago
  Is it still shadowbanned? If you go to reddit.com/appeal what does it say?
  - someonebaggy 3mo ago
    There's basically only one result you can get from an appeal which is "we have reviewed your account and we will not be lifting the suspension at this time." The appeals process exists to fill a checkbox that says there must be an appeals process, not to actually unban anyone.
    - rjbwork 3mo ago
      I have seen multiple people have their shadowbans reversed after I recognized it had occurred and alerted them.
    - bink 3mo ago
      I appealed a warning I was given for quoting a Simpsons episode (promoting violence, Reddit-wide and not from a sub) and was my appeal was granted.
  - Terr_ 3mo ago
    As I said, the appeal was granted, or at least that's the message I got in reply. However everything remained in the nether-realm and the appeals page claimed that my account was normal, and that therefore couldn't be used No response from any support ticket either. They still advertised me IPO opportunities though...
    - qingcharles 3mo ago
      That's strange. I've never seen that set of circumstances. Pop me an email on my profile and I can try to put you in touch with someone who might be able to help.
randysalami 3mo ago
Reddit must have some mechanism specifically for non-spamming bots that isn’t covered in this article. I wonder how it works. I imagine the mechanisms are more complex and opaque than anti-spam (with various levels being exposed to the hierarchies of Reddit and government backdoors). These days, I’ve noticed an almost forcing-function that operates to put the minimum spin needed on posts and comments to turn signal to noise. It seems smart enough to not only generate noisy comments but create comments to amplify existing organic noisy comments. I’m sure these systems are decentralized, emergent, and split across numerous nation-states and actors. I’m also fairly certain what we have now is a tenuous balance that has emerged from all these actors and Reddit policing actions as well. I imagine Reddit has a high-level of insight into this and a certain level of permissibility it grants, both to inflate user counts and to steer public discourse and insight into less productive mean (or productive to certain interest groups at the expense of the people). I think is also an effect that Reddit has become more global and consensus of the USA people is very antagonistic to the consensus of the people of the world so that doesn’t help (+ access to LLMs to make English writing no longer a barrier to entry).
- asdff 3mo ago
  There is some sort of wink wink nudge nudge agreement going on with certain spam accounts. You will see them post article spam with hidden history, and if you look up their posts either via google or any other reddit crawling tool, they are posting all over various subreddits that same article maybe dozens of times. If they comment it is really basic and formulaic and found all over their post histories as well. I feel like reddit enjoys it as these posts (often political in some way) usually get good engagement which is in line with reddits own incentives for courting advertiser money.
  - hightrix 3mo ago
    This practice was perfected by gallowboob years ago. He would spam a link/pic/post and monitor, if the post didn’t gain traction, he would delete and post again as to not trigger protections against the same link being posted. He was a cancer on Reddit and I’m sure he still exists under different monikers. But now there are 100s of gallowboobs.
busymom0 3mo ago
I swear I read this article 2 or 3 days ago and the comments on that post were also same as this post. Am I missing something here?
- yorwba 3mo ago
  You're missing the second-chance pool https://news.ycombinator.com/pool https://news.ycombinator.com/pool which allows certain posts to reappear as if they were new.
  - asdff 3mo ago
    Once again expressing my opinion that this is the worst anti feature of the site. Threads are like commenting in the void because most people are not going to be looking for replies to comments they made days or weeks ago. I see the true datestamp of the comment I replied upon upthread was not 3 hours ago, but 3 days ago. The fact that they change the timestamp is also very stupid (yes you can hover and still return the datestamp, but this is by definition a dark pattern). These posts should preserve the timestamp vs masking it and even should be flagged as [Second Chance] in the title imo.
    - deleted 3mo ago
      [deleted]
  - khurs 3mo ago
    That doesn't show for me unless I click the link. How does it work?
    - someonebaggy 3mo ago
      dang selects a post to be second-chanced, and this resets the timestamp on it so it appears again.
- rebane2001 3mo ago
  wait what the heck yeah, this is the same post as from a few days ago, i guess the comment and post timestamps got glitched??
felooboolooomba 3mo ago
My friend got shadowbanned for posting a youtube link, part of a interview with Sascha Riley (the one where the explains the thing with the tent peg): https://www.youtube.com/watch?v=84PHEMLab6g&t=2807s https://www.youtube.com/watch?v=84PHEMLab6g&t=2807s
- someonebaggy 3mo ago
  You get shadowbanned for almost anything these days. It's not worth trying to use that site any more - I wrote it off as a lost cause, a playground for bots to talk to each other thinking they're talking to humans.
  - felooboolooomba 3mo ago
    The Russian bots on there trying to demoralize the UK are pretty hilarious too.
montoyaig 3mo ago
[flagged]
Oarch 3mo ago
Can't you just append ".json" to the end of any Reddit link and read all sorts of these fields?
- rebane2001 3mo ago
  No, the API will not return the admin-only removal reason. The code path that causes this is in the post.
leviathant 3mo ago
I cannot emphasize strongly enough just how deeply pervasive the spam is at Reddit. I'm a mod at the ecommerce subreddit, and I've only caught some of the AI-powered marketing operations because in one particular campaign that was making fictional claims about things I had direct knowledge about. Once I looked into the post history, and started to untangle the web of accounts that formed a self-supporting community of posters and commenters, just subtle enough to get genuine engagement, but specific enough to make the kind of posts that the LLMs will siphon up and regurgitate. It's not just shady little operations. I'm speaking specifically about the SCAYLE ecommerce platform, in my example. They've got Zalando money to play with, and as a German platform that's trying to break into the North American market, it appears they've made a bet on indirectly spamming the LLMs with fictional tales of commerce replatforming horror stories. At first, they're some of the more interesting topics in a sea of really useless posts, with contributions from people who seem to have some real experience with enterprise ecommerce. I was a little suspicious, but these interaction campaigns were spread out enough that I didn't put the pieces together for months. Of course, to go back on what I said at the top of the paragraph, maybe SCAYLE is shady, and I'm giving them too much credit. The good news is, some of the AI powered tools that mods have access to are getting better at surfacing suspicious patterns of behavior. However, I still find I have to manually address these campaigns. In the cat-and-mouse game with these marketing jerks, I'm always reluctant to surface what's working and what isn't. This is an interesting post, but it's going to make things worse. Ah well.
- jamesfinlayson 3mo ago
  I remember reading years ago about some corrupt mod in one of the image subreddits - he or his friend had started some image hosting site and had six different Reddit accounts that he used to upvote posts that used his site and downvote all other posts. It took people a long while to notice what he was up to.
  - leviathant 3mo ago
    And now automate and scale that with Claude/OpenAI/Gemini/whatever. It's insidious and terrible.
pedalpete 3mo ago
Based on the current status of my shadowbanned account (I suspect a competitor in our space retaliating), it looks like `banall` only flags posts from the last 6 years. Of course, nobody can view my profile anymore anyway (I'm waiting on appeal), but on my account, only posts from the last 6 years have the "Sorry this post was removed by reddit filters" message.
- mondomondo 3mo ago
  [dead]
hnloser 3mo ago
[flagged]
tomkow 3mo ago
[flagged]
try-working 3mo ago
what's going on with this post? It was posted days ago and now its back on the frontpage again?
- aw1621107 3mo ago
  My guess would be the second-chance pool (https://news.ycombinator.com/pool https://news.ycombinator.com/pool, explained by dang at https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308)
  - gnabgib 3mo ago
    Exactly that, you can tell by hovering over the posted time "9 hours ago".. and see that it's really from June.
    - try-working 3mo ago
      yeah but it was on the frontpage and quite popular last time! and the timestamp on the comments changed too, to be relative to the new "posted" timestamp.
kaszanka 3mo ago
"Anti-Evil Operations" is a pretty grandiose name for spam filtering. Also I liked the House of Leaves reference
- justcool393 3mo ago
  ya it really is. it's the new name (where new is relative to like 10 years) for Trust & Safety.
- ecshafer 3mo ago
  "Anti-Evil Operations" is the name of their wrong think filtering, which is hilarious in a newspeak kind of way.
PUSH_AX 3mo ago
A lot of what they are doing now is around AI comments and posts, I know this because in some of my subreddits I have various automod filters that align with the way AI writes, I see it all the time where my automod removes something and reddit removes it again after (some kind of race condition), as well as tons of these accounts getting site wide bans. In case you didn't realise, a massive portion of content on reddit now is LLMs. I'm not a massive fan of how reddit has played certain hands in the last 5 years or so, but I do hope they win the war on dead internet theory.
- MagicMoonlight 3mo ago
  [dead]
- Scoundreller 3mo ago
  I suspect their IPO is a sign that they’re throwing in the towel existentially.
justcool393 3mo ago
some more information perhaps? banned_by true is more accurate to say "admin or automatic". in "admin mode," you can see these although not sure the UX for these nowadays now that it is spewing a gazillion lines of text into them). Anti-Evil Operations removals (nee Trust & Safety) are generally human(-assisted) actions (although these actions can be applied en masse). there's some more information nowadays in the API which was really nice. it also helped because people stopped blaming "the mods" for removals when the spam filter slopped all over the place. this was also annoying because previously you had to previously guess from the API how it was removed even if you were a mod. the 3 ways to remove a post/comment (i.e. in reply to: train_spam): - remove not spam: removes it but doesn't train the spam filter, obvious - spam: removes it and trains the spam filter, obvious - confirm spam: only happens when you remove after removing for any reason, *does not* train the spam filter - reinforce spam: trains the spam filter even if the spam filter already caught it. *does* train the spam filter. you can do this by doing `action: spam` in automod. not sure if there have been any more in the last few years also you can tell the legacy of "removals", back in the day stories were "banned" instead of "removed" by moderators and administrators. also also also... you can see a lot of the stuff from this article in the `approved_by` side of it as well. if you hover over a checkmark of someone who has been unshadowbanned, you'll see it says "approved by Reddit (shadowban removed)" if an admin manually unspams someones stuff (say someone who got accidentally shadowbanned and got hit with an overzealous spam filter multiple times >.>), it'll say "approved by <username> (all)". there are some consequences to this. it approves stuff that has been "filtered" (as AutoMod filtering is a weird hack where it removes something but keeps in the modqueue). > spammit i believe this is the thing that is "pretty similar to a naive Bayesian classifier"[1][2] that reddit used. /u/Deimorz iirc was a reddit dev at the time and it was somewhat public info. i say somewhat because you kinda had to be both interested in the this and probably be around the metasphere iirc from some other comments i pieced together there are also per-subreddit spam filters. in the olden days sometimes they'd get way out of whack and you could ask an admin to reset it for you... or something idk > em guessing em in this case btw refers to /u/hueypriest, who was reddit's GM at the time > would’ve been catastrophic for Reddit’s spam issues the thing that surprised me at the time was just how bad reddit's spam filtering is. i did a small little thing at the time where i'd just look at stuff following some basic spam filtering rules (like stuff you'd probably get out of an artisinal spamassassin ruleset) and even that deluge was amazing to see. like the ML stuff is cool and all but seriously 90% of this could probably still be solved with some basic rules. the profile hiding stuff didn't help either but that was way after my time. [1]: https://reddit.com/r/TheoryOfReddit/comments/10ko5h/comment/c6eakzz https://reddit.com/r/TheoryOfReddit/comments/10ko5h/comment/... (2012) [2]: https://www.reddit.com/r/modnews/comments/6bj5de/state_of_spam/ https://www.reddit.com/r/modnews/comments/6bj5de/state_of_sp...
forestry 3mo ago
My takeaway: > My test account (5 years old!) got banned immediately, and all of its post history got wiped too. RIP I want to know the real string of the event I ever want to delete my account and content. This would be much faster than using a browser script to manually delete.
deleted 3mo ago
[deleted]
karmdit 3mo ago
[flagged]

185 points by OuterVale 3mo ago

Neat rabbit hole. Reminds me of having to deal with email spam - it was a similar deal with rule-based filters, ML scores, domain bans,IP filtering, browse fingerprinting etc and mishmash of ever evolving scripts surviving org and personnel changes. Glad i dont deal with it anymore as the frontier seems to be 2 fronts now with human and agentic spam.

Terr_ 3mo ago

Damn, maybe I can finally find out why my 10+ year account was globally (and retroactively) shadowbanned, even though the appeal was allegedly granted. In the past, those post removals didn't even exist in the moderation log, so perhaps a reason could give me a clue... On the other hand, I'm taking a kind of emotional damage just remembering.

qingcharles 3mo ago
Is it still shadowbanned? If you go to reddit.com/appeal what does it say?
- someonebaggy 3mo ago
  There's basically only one result you can get from an appeal which is "we have reviewed your account and we will not be lifting the suspension at this time." The appeals process exists to fill a checkbox that says there must be an appeals process, not to actually unban anyone.
  - rjbwork 3mo ago
    I have seen multiple people have their shadowbans reversed after I recognized it had occurred and alerted them.
  - bink 3mo ago
    I appealed a warning I was given for quoting a Simpsons episode (promoting violence, Reddit-wide and not from a sub) and was my appeal was granted.
- Terr_ 3mo ago
  As I said, the appeal was granted, or at least that's the message I got in reply. However everything remained in the nether-realm and the appeals page claimed that my account was normal, and that therefore couldn't be used No response from any support ticket either. They still advertised me IPO opportunities though...
  - qingcharles 3mo ago
    That's strange. I've never seen that set of circumstances. Pop me an email on my profile and I can try to put you in touch with someone who might be able to help.

randysalami 3mo ago

Reddit must have some mechanism specifically for non-spamming bots that isn’t covered in this article. I wonder how it works. I imagine the mechanisms are more complex and opaque than anti-spam (with various levels being exposed to the hierarchies of Reddit and government backdoors). These days, I’ve noticed an almost forcing-function that operates to put the minimum spin needed on posts and comments to turn signal to noise. It seems smart enough to not only generate noisy comments but create comments to amplify existing organic noisy comments. I’m sure these systems are decentralized, emergent, and split across numerous nation-states and actors. I’m also fairly certain what we have now is a tenuous balance that has emerged from all these actors and Reddit policing actions as well. I imagine Reddit has a high-level of insight into this and a certain level of permissibility it grants, both to inflate user counts and to steer public discourse and insight into less productive mean (or productive to certain interest groups at the expense of the people). I think is also an effect that Reddit has become more global and consensus of the USA people is very antagonistic to the consensus of the people of the world so that doesn’t help (+ access to LLMs to make English writing no longer a barrier to entry).

asdff 3mo ago
There is some sort of wink wink nudge nudge agreement going on with certain spam accounts. You will see them post article spam with hidden history, and if you look up their posts either via google or any other reddit crawling tool, they are posting all over various subreddits that same article maybe dozens of times. If they comment it is really basic and formulaic and found all over their post histories as well. I feel like reddit enjoys it as these posts (often political in some way) usually get good engagement which is in line with reddits own incentives for courting advertiser money.
- hightrix 3mo ago
  This practice was perfected by gallowboob years ago. He would spam a link/pic/post and monitor, if the post didn’t gain traction, he would delete and post again as to not trigger protections against the same link being posted. He was a cancer on Reddit and I’m sure he still exists under different monikers. But now there are 100s of gallowboobs.

busymom0 3mo ago

I swear I read this article 2 or 3 days ago and the comments on that post were also same as this post. Am I missing something here?

yorwba 3mo ago
You're missing the second-chance pool https://news.ycombinator.com/pool https://news.ycombinator.com/pool which allows certain posts to reappear as if they were new.
- asdff 3mo ago
  Once again expressing my opinion that this is the worst anti feature of the site. Threads are like commenting in the void because most people are not going to be looking for replies to comments they made days or weeks ago. I see the true datestamp of the comment I replied upon upthread was not 3 hours ago, but 3 days ago. The fact that they change the timestamp is also very stupid (yes you can hover and still return the datestamp, but this is by definition a dark pattern). These posts should preserve the timestamp vs masking it and even should be flagged as [Second Chance] in the title imo.
  - deleted 3mo ago
    [deleted]
- khurs 3mo ago
  That doesn't show for me unless I click the link. How does it work?
  - someonebaggy 3mo ago
    dang selects a post to be second-chanced, and this resets the timestamp on it so it appears again.
rebane2001 3mo ago
wait what the heck yeah, this is the same post as from a few days ago, i guess the comment and post timestamps got glitched??

felooboolooomba 3mo ago

My friend got shadowbanned for posting a youtube link, part of a interview with Sascha Riley (the one where the explains the thing with the tent peg): https://www.youtube.com/watch?v=84PHEMLab6g&t=2807s https://www.youtube.com/watch?v=84PHEMLab6g&t=2807s

someonebaggy 3mo ago
You get shadowbanned for almost anything these days. It's not worth trying to use that site any more - I wrote it off as a lost cause, a playground for bots to talk to each other thinking they're talking to humans.
- felooboolooomba 3mo ago
  The Russian bots on there trying to demoralize the UK are pretty hilarious too.

montoyaig 3mo ago

[flagged]

Oarch 3mo ago

Can't you just append ".json" to the end of any Reddit link and read all sorts of these fields?

rebane2001 3mo ago
No, the API will not return the admin-only removal reason. The code path that causes this is in the post.

leviathant 3mo ago

I cannot emphasize strongly enough just how deeply pervasive the spam is at Reddit. I'm a mod at the ecommerce subreddit, and I've only caught some of the AI-powered marketing operations because in one particular campaign that was making fictional claims about things I had direct knowledge about. Once I looked into the post history, and started to untangle the web of accounts that formed a self-supporting community of posters and commenters, just subtle enough to get genuine engagement, but specific enough to make the kind of posts that the LLMs will siphon up and regurgitate. It's not just shady little operations. I'm speaking specifically about the SCAYLE ecommerce platform, in my example. They've got Zalando money to play with, and as a German platform that's trying to break into the North American market, it appears they've made a bet on indirectly spamming the LLMs with fictional tales of commerce replatforming horror stories. At first, they're some of the more interesting topics in a sea of really useless posts, with contributions from people who seem to have some real experience with enterprise ecommerce. I was a little suspicious, but these interaction campaigns were spread out enough that I didn't put the pieces together for months. Of course, to go back on what I said at the top of the paragraph, maybe SCAYLE is shady, and I'm giving them too much credit. The good news is, some of the AI powered tools that mods have access to are getting better at surfacing suspicious patterns of behavior. However, I still find I have to manually address these campaigns. In the cat-and-mouse game with these marketing jerks, I'm always reluctant to surface what's working and what isn't. This is an interesting post, but it's going to make things worse. Ah well.

jamesfinlayson 3mo ago
I remember reading years ago about some corrupt mod in one of the image subreddits - he or his friend had started some image hosting site and had six different Reddit accounts that he used to upvote posts that used his site and downvote all other posts. It took people a long while to notice what he was up to.
- leviathant 3mo ago
  And now automate and scale that with Claude/OpenAI/Gemini/whatever. It's insidious and terrible.

pedalpete 3mo ago

Based on the current status of my shadowbanned account (I suspect a competitor in our space retaliating), it looks like `banall` only flags posts from the last 6 years. Of course, nobody can view my profile anymore anyway (I'm waiting on appeal), but on my account, only posts from the last 6 years have the "Sorry this post was removed by reddit filters" message.

mondomondo 3mo ago
[dead]

hnloser 3mo ago

[flagged]

tomkow 3mo ago

[flagged]

try-working 3mo ago

what's going on with this post? It was posted days ago and now its back on the frontpage again?

aw1621107 3mo ago
My guess would be the second-chance pool (https://news.ycombinator.com/pool https://news.ycombinator.com/pool, explained by dang at https://news.ycombinator.com/item?id=26998308 https://news.ycombinator.com/item?id=26998308)
- gnabgib 3mo ago
  Exactly that, you can tell by hovering over the posted time "9 hours ago".. and see that it's really from June.
  - try-working 3mo ago
    yeah but it was on the frontpage and quite popular last time! and the timestamp on the comments changed too, to be relative to the new "posted" timestamp.

kaszanka 3mo ago

"Anti-Evil Operations" is a pretty grandiose name for spam filtering. Also I liked the House of Leaves reference

justcool393 3mo ago
ya it really is. it's the new name (where new is relative to like 10 years) for Trust & Safety.
ecshafer 3mo ago
"Anti-Evil Operations" is the name of their wrong think filtering, which is hilarious in a newspeak kind of way.

PUSH_AX 3mo ago

A lot of what they are doing now is around AI comments and posts, I know this because in some of my subreddits I have various automod filters that align with the way AI writes, I see it all the time where my automod removes something and reddit removes it again after (some kind of race condition), as well as tons of these accounts getting site wide bans. In case you didn't realise, a massive portion of content on reddit now is LLMs. I'm not a massive fan of how reddit has played certain hands in the last 5 years or so, but I do hope they win the war on dead internet theory.

MagicMoonlight 3mo ago
[dead]
Scoundreller 3mo ago
I suspect their IPO is a sign that they’re throwing in the towel existentially.

justcool393 3mo ago

some more information perhaps? banned_by true is more accurate to say "admin or automatic". in "admin mode," you can see these although not sure the UX for these nowadays now that it is spewing a gazillion lines of text into them). Anti-Evil Operations removals (nee Trust & Safety) are generally human(-assisted) actions (although these actions can be applied en masse). there's some more information nowadays in the API which was really nice. it also helped because people stopped blaming "the mods" for removals when the spam filter slopped all over the place. this was also annoying because previously you had to previously guess from the API how it was removed even if you were a mod. the 3 ways to remove a post/comment (i.e. in reply to: train_spam): - remove not spam: removes it but doesn't train the spam filter, obvious - spam: removes it and trains the spam filter, obvious - confirm spam: only happens when you remove after removing for any reason, *does not* train the spam filter - reinforce spam: trains the spam filter even if the spam filter already caught it. *does* train the spam filter. you can do this by doing `action: spam` in automod. not sure if there have been any more in the last few years also you can tell the legacy of "removals", back in the day stories were "banned" instead of "removed" by moderators and administrators. also also also... you can see a lot of the stuff from this article in the `approved_by` side of it as well. if you hover over a checkmark of someone who has been unshadowbanned, you'll see it says "approved by Reddit (shadowban removed)" if an admin manually unspams someones stuff (say someone who got accidentally shadowbanned and got hit with an overzealous spam filter multiple times >.>), it'll say "approved by <username> (all)". there are some consequences to this. it approves stuff that has been "filtered" (as AutoMod filtering is a weird hack where it removes something but keeps in the modqueue). > spammit i believe this is the thing that is "pretty similar to a naive Bayesian classifier"[1][2] that reddit used. /u/Deimorz iirc was a reddit dev at the time and it was somewhat public info. i say somewhat because you kinda had to be both interested in the this and probably be around the metasphere iirc from some other comments i pieced together there are also per-subreddit spam filters. in the olden days sometimes they'd get way out of whack and you could ask an admin to reset it for you... or something idk > em guessing em in this case btw refers to /u/hueypriest, who was reddit's GM at the time > would’ve been catastrophic for Reddit’s spam issues the thing that surprised me at the time was just how bad reddit's spam filtering is. i did a small little thing at the time where i'd just look at stuff following some basic spam filtering rules (like stuff you'd probably get out of an artisinal spamassassin ruleset) and even that deluge was amazing to see. like the ML stuff is cool and all but seriously 90% of this could probably still be solved with some basic rules. the profile hiding stuff didn't help either but that was way after my time. [1]: https://reddit.com/r/TheoryOfReddit/comments/10ko5h/comment/c6eakzz https://reddit.com/r/TheoryOfReddit/comments/10ko5h/comment/... (2012) [2]: https://www.reddit.com/r/modnews/comments/6bj5de/state_of_spam/ https://www.reddit.com/r/modnews/comments/6bj5de/state_of_sp...

forestry 3mo ago

My takeaway: > My test account (5 years old!) got banned immediately, and all of its post history got wiped too. RIP I want to know the real string of the event I ever want to delete my account and content. This would be much faster than using a browser script to manually delete.

deleted 3mo ago

[deleted]

karmdit 3mo ago

[flagged]