3 ms·

High quality anecdata are exactly the reason why I love HN. Thanks for posting about it. > fake AI accounts First, how do you identify them? Is it stric

by throwaway2037 6mo ago

High quality anecdata are exactly the reason why I love HN. Thanks for posting about it. > fake AI accounts First, how do you identify them? Is it strictly admins monitoring posts/server-side logs or do users report odd behaviour? Second, what is the purpose of these accounts? Are they basically running submarine adverts, or are they just trolling (to harm the community)?

rramadass 6mo ago
A starting point for study; AI Deception: A Survey of Examples, Risks, and Potential Solutions - https://arxiv.org/abs/2308.14752 https://arxiv.org/abs/2308.14752 Deception Analysis with Artificial Intelligence: An Interdisciplinary Perspective - https://arxiv.org/abs/2406.05724 https://arxiv.org/abs/2406.05724
- rramadass 6mo ago
  Some background (pre-AI) ; Online Deception in Social Media - https://cacm.acm.org/research/online-deception-in-social-media/ https://cacm.acm.org/research/online-deception-in-social-med...
- deleted 6mo ago
  [deleted]
21asdffdsa12 6mo ago
Eagerness.. you let them work through a endless labyrinth of forms. While the human drops out after the 2nd form- the ai is willing to go all the way always.
- spoiler 6mo ago
  Don't you lose really users by making it easy for bots to complete a sign up, but hard for humans?
  - Cyan488 6mo ago
    Maybe 5 required fields and 30 optional fields. Apart from "census dweebs" [1] I don't know of many people that fill in more than the required. [1]: https://thebeaverton.com/2026/05/local-nerd-disappointed-he-didnt-get-the-long-form-census/ https://thebeaverton.com/2026/05/local-nerd-disappointed-he-...
  - kyusan0 6mo ago
    I'm not familiar with the specific example, but as a first hunch, if I had to implement such a system, it'd be part of the flow after the successful signup, not during.
  - 21asdffdsa12 6mo ago
    You could also pretend to pass-through and watch the behavior of the "user" after success.
CrzyLngPwd 6mo ago
We review all signups and their submitted content, and we have a fake account type "AI creator" which we record, so if they tried that first, it's a clue. We also integrate a third-party AI detection tool that checks their uploaded content before they get access to the community, and we use a second third-party tool as a double-check, and if we still doubt ourselves, we'll ask for their workings-outs in some way that is relevant to their creative type. We also ask them to prove they are the author of their work through various time-tested mechanisms. I have no idea why people do the things they do. People are a mystery to me.
- throwaway2037 6mo ago
  > we have a fake account type "AI creator" which we record This is genius! The LLM is so dumb that it falls for the honeypot. > We also ask them to prove they are the author of their work through various time-tested mechanisms. I would love to know more details, but it is not a good idea to share on a public platform. That said, if it is working for you, then hats off: You have found/developed a effective "defense in layers" strategy.

rramadass 6mo ago

A starting point for study; AI Deception: A Survey of Examples, Risks, and Potential Solutions - https://arxiv.org/abs/2308.14752 https://arxiv.org/abs/2308.14752 Deception Analysis with Artificial Intelligence: An Interdisciplinary Perspective - https://arxiv.org/abs/2406.05724 https://arxiv.org/abs/2406.05724

rramadass 6mo ago
Some background (pre-AI) ; Online Deception in Social Media - https://cacm.acm.org/research/online-deception-in-social-media/ https://cacm.acm.org/research/online-deception-in-social-med...
deleted 6mo ago
[deleted]

21asdffdsa12 6mo ago

Eagerness.. you let them work through a endless labyrinth of forms. While the human drops out after the 2nd form- the ai is willing to go all the way always.

spoiler 6mo ago
Don't you lose really users by making it easy for bots to complete a sign up, but hard for humans?
- Cyan488 6mo ago
  Maybe 5 required fields and 30 optional fields. Apart from "census dweebs" [1] I don't know of many people that fill in more than the required. [1]: https://thebeaverton.com/2026/05/local-nerd-disappointed-he-didnt-get-the-long-form-census/ https://thebeaverton.com/2026/05/local-nerd-disappointed-he-...
- kyusan0 6mo ago
  I'm not familiar with the specific example, but as a first hunch, if I had to implement such a system, it'd be part of the flow after the successful signup, not during.
- 21asdffdsa12 6mo ago
  You could also pretend to pass-through and watch the behavior of the "user" after success.

CrzyLngPwd 6mo ago

We review all signups and their submitted content, and we have a fake account type "AI creator" which we record, so if they tried that first, it's a clue. We also integrate a third-party AI detection tool that checks their uploaded content before they get access to the community, and we use a second third-party tool as a double-check, and if we still doubt ourselves, we'll ask for their workings-outs in some way that is relevant to their creative type. We also ask them to prove they are the author of their work through various time-tested mechanisms. I have no idea why people do the things they do. People are a mystery to me.

throwaway2037 6mo ago
> we have a fake account type "AI creator" which we record This is genius! The LLM is so dumb that it falls for the honeypot. > We also ask them to prove they are the author of their work through various time-tested mechanisms. I would love to know more details, but it is not a good idea to share on a public platform. That said, if it is working for you, then hats off: You have found/developed a effective "defense in layers" strategy.