5 ms·

Whether text is AI generated or not doesn’t matter. It’s whether we can detect if the text is low or high quality that matters.

nwoli 4y ago
Yeah I mean if AI HN comments start to appear does that really matter in the end. Ie there’s already a lot of noise in the form of low quality comments, and they get downvoted. If an AI manages to produce high quality comments that people upvote im not sure I care that it wasn’t generated by a human
- jacquesm 4y ago
  It would matter to me because I'm here to read what people write, not what bots write. Besides that being against the HN guidelines noise in the form of high quality comments is still noise.
  - travisjungroth 4y ago
    A comment being noise and high quality seems contradictory.
    - notahacker 4y ago
      "High quality" in HN terms means it's not just a couple of words or a meme, isn't glaringly false or advertising or particularly off topic or antagonistic. "Feedback" that isn't actually feedback, syntheses of other people's comments that claim to be a new person adding their agreement, anecdotes that didn't happen, little known facts that aren't true and pseudo-recommendations that are actually derived largely from the product marketing copy will usually pass this filter, but they're definitely noise
  - chrischen 4y ago
    I'm here to read high quality comments. If I'm unable to distinguish between high and low quality comments then I have another problem... maybe I'm the one that's not human.
    - p1esk 4y ago
      I think what he meant was that an AI generated comment might be high quality bullshit, like for example a story from personal experience that has never existed, or a false fact that looks convincing and/or is hard to check, or a personal opinion written by a statistical meat grinder, etc. And these comments might come to dominate any discussion because they are so easy to generate, unlike human high quality comments.
      - chrischen 4y ago
        If it's now easy to generate nice sounding bullshit then perhaps we should be re-evaluating our goals in reading that. The examples you give is no different from going on TikTok and ingesting mindless entertaining content.
        p1esk 4y ago
        Exactly. We don’t want HN turning into TikTok, do we? What’s worse, there will inevitably appear AI constructed “native” ads or even political propaganda - high quality comments skillfully promoting products or someone’s agenda.
        chrischen 4y ago
        Yes, so downvote comments that are just entertainment. Only keep high quality objective comments. At best problem solved, at worst AI are better at commenting than humans.
- rcme 4y ago
  It definitely matters. Like you can play chess against a bot at any level. It’s not nearly as fun as playing another human.
jacquesm 4y ago
It does matter. For instance, what will you train your future AI on? If AI ends up eating its own tail it will end up amplifying its errors and the end result is that you won't be able to trust anything that wasn't written down before this hit.
- chrischen 4y ago
  You shouldn't be trusting random comments as truth in the first place. Maybe learn to identify high value first? Then you can just train on high value targets. Just because (hypothetically) 4-chan is full of humans doesn't make it suddenly a good training target for AI.
  - jacquesm 4y ago
    Random comments attached to accounts with good reputation count for something. If those get replaced by random comments written by bots attached to accounts with good reputation then that would be a problem. High value has many signals, reputation is one of them. 4chan is/was a very mixed bag, and just like there are garbage HN comments so there were excellent 4chan comments.
  - YetAnotherNick 4y ago
    OpenAI training data oversamples sites which got 3 reddit likes[1], because it improves model. Most large scale training data has some filter on commoncrawl. [1]: https://paperswithcode.com/dataset/webtext https://paperswithcode.com/dataset/webtext
- p1esk 4y ago
  My hope is that future AI will become smarter, not just trained on more data, and as a result it will be capable to perhaps “rate” the quality of any text, based on fact checking, reputation of the commenter, content of all their other comments, frequency of the previous comments, consistency of the style, availability of particular LLMs at the time, etc. It won’t be good enough to detect AI of the same level, but might be good enough to filter out most of the earlier AI’s comments. GPT-5 might be able to detect comments generated by GPT-3, even if its training dataset included a small fraction of GPT-3 generated content, or even if it was trained on the same corpus as GPT-4 (up to 2021).
YPPH 4y ago
There are contexts where it does matter. For instance, plagiarism detection at schools.
- chaxor 4y ago
  Why is "at schools" necessarily important though? Wouldn't it also be nice to know if the scientific articles you are reading, or the code a company is based upon, or a new piece of legislature was constructed from someone just asking an LLM to do it for them?
  - YPPH 4y ago
    Sure, I was only giving one example to counter the claim, which is not to say there aren't other examples.
  - 8note 4y ago
    As usual, "just" pulls a lot of weight - if they're putting the straight LLM output, it probably won't be great regardless. If they put effort into making the ouput good, then chances are the content will be good regardless of whether they wrote it by hand, or wrote it using an LLM. Like, I don't need to know whether you used a bic ballpoint pen, or some other ballpoint pen to write your note on paper; the tool is a tool
- chrischen 4y ago
  Plagiarism can be solved with non-AI methods because plagiarism is defined more or less by exact copying. If you mean detection of AI generated essays at schools, then teachers just have to expect better essays and be able to identify them since students now have a new tool. Think of it like grading math homework before and after the invention of calculators. Once students had calculators at home the problems simply got harder and higher level, since there was no reasonable way to enforce "don't use a calculator at home with these artificially simplistic problems."
waboremo 4y ago
The more I think about this, the less I agree with it. Think of all the online communication you've had. The good parts. The great but messy blog post that changed your mind about something. Communicating with someone on a social network only to share each other's work and establish a great working relationship. Talking to a friend in DMs sharing your pain. Getting feedback from a mentor. Having people tell you how amazing your art is or how it inspired them. The bad parts too. The negative comments from random people. Others mocking your tweet. People calling your HN comment idiotic. All of these things affect us because we assume a person is behind them, and therefore their intent is clear. Without that, we devalue all communication to compensate. Like a more exaggerated effect of how we view likes on social media, where "20k likes" means nothing. Truly believe we're only ok with it now because it's easy to tell what is AI generated, and they're mostly confined. When AI is creating videos on Twitter, creating threads on Reddit, posting blogs on their personal site to be shared to Mastodon, replying to other bots on HN... I'm not sure we'll feel that it's whether we can detect quality anymore. There's also the differentiation theory where in order to separate human connection from bots, we'll do the opposite. We'll start incorporating prompt injections into everything we post, start purposefully being "low quality" because bots are trained against that. Now we're creating a very particular kind of societal mess.
- YetAnotherNick 4y ago
  I definitely don't agree with the GP but I don't agree with you either. Human could and absolutely become attached to non humans, ranging from dog to pen to house. And I believe humans might begin to develop deeper relationship with AI bots very soon.
  - waboremo 4y ago
    I don't know. Part of why we anthropomorphize so much is because we can still feel the physical elements. We pet a dog, we live inside our homes, list goes on. When that is not possible, how does this change people and their valuation of online communication? Would you still love "Anne" when she's now rejected your 50th attempt to meet in person? Would you still reply to comments like mine if the default assumption wasn't "that's another human being I can persuade" and instead "that is a bot"? I'm sure in the long term that differentiation alone would change, you know androids and all that jazz - however in the meantime, we're going to be facing a tremendous shift in how we interact online.
WalterBright 4y ago
If HN develops into a forum dominated by chatbots, I'll leave.
- klipklop 4y ago
  How will you know?
  - WalterBright 4y ago
    Maybe I am the only real person that exists, and everyone else is just a simulation.

nwoli 4y ago

Yeah I mean if AI HN comments start to appear does that really matter in the end. Ie there’s already a lot of noise in the form of low quality comments, and they get downvoted. If an AI manages to produce high quality comments that people upvote im not sure I care that it wasn’t generated by a human

jacquesm 4y ago
It would matter to me because I'm here to read what people write, not what bots write. Besides that being against the HN guidelines noise in the form of high quality comments is still noise.
- travisjungroth 4y ago
  A comment being noise and high quality seems contradictory.
  - notahacker 4y ago
    "High quality" in HN terms means it's not just a couple of words or a meme, isn't glaringly false or advertising or particularly off topic or antagonistic. "Feedback" that isn't actually feedback, syntheses of other people's comments that claim to be a new person adding their agreement, anecdotes that didn't happen, little known facts that aren't true and pseudo-recommendations that are actually derived largely from the product marketing copy will usually pass this filter, but they're definitely noise
- chrischen 4y ago
  I'm here to read high quality comments. If I'm unable to distinguish between high and low quality comments then I have another problem... maybe I'm the one that's not human.
  - p1esk 4y ago
    I think what he meant was that an AI generated comment might be high quality bullshit, like for example a story from personal experience that has never existed, or a false fact that looks convincing and/or is hard to check, or a personal opinion written by a statistical meat grinder, etc. And these comments might come to dominate any discussion because they are so easy to generate, unlike human high quality comments.
    - chrischen 4y ago
      If it's now easy to generate nice sounding bullshit then perhaps we should be re-evaluating our goals in reading that. The examples you give is no different from going on TikTok and ingesting mindless entertaining content.
      - p1esk 4y ago
        Exactly. We don’t want HN turning into TikTok, do we? What’s worse, there will inevitably appear AI constructed “native” ads or even political propaganda - high quality comments skillfully promoting products or someone’s agenda.
        chrischen 4y ago
        Yes, so downvote comments that are just entertainment. Only keep high quality objective comments. At best problem solved, at worst AI are better at commenting than humans.
rcme 4y ago
It definitely matters. Like you can play chess against a bot at any level. It’s not nearly as fun as playing another human.

jacquesm 4y ago

It does matter. For instance, what will you train your future AI on? If AI ends up eating its own tail it will end up amplifying its errors and the end result is that you won't be able to trust anything that wasn't written down before this hit.

chrischen 4y ago
You shouldn't be trusting random comments as truth in the first place. Maybe learn to identify high value first? Then you can just train on high value targets. Just because (hypothetically) 4-chan is full of humans doesn't make it suddenly a good training target for AI.
- jacquesm 4y ago
  Random comments attached to accounts with good reputation count for something. If those get replaced by random comments written by bots attached to accounts with good reputation then that would be a problem. High value has many signals, reputation is one of them. 4chan is/was a very mixed bag, and just like there are garbage HN comments so there were excellent 4chan comments.
- YetAnotherNick 4y ago
  OpenAI training data oversamples sites which got 3 reddit likes[1], because it improves model. Most large scale training data has some filter on commoncrawl. [1]: https://paperswithcode.com/dataset/webtext https://paperswithcode.com/dataset/webtext
p1esk 4y ago
My hope is that future AI will become smarter, not just trained on more data, and as a result it will be capable to perhaps “rate” the quality of any text, based on fact checking, reputation of the commenter, content of all their other comments, frequency of the previous comments, consistency of the style, availability of particular LLMs at the time, etc. It won’t be good enough to detect AI of the same level, but might be good enough to filter out most of the earlier AI’s comments. GPT-5 might be able to detect comments generated by GPT-3, even if its training dataset included a small fraction of GPT-3 generated content, or even if it was trained on the same corpus as GPT-4 (up to 2021).

YPPH 4y ago

There are contexts where it does matter. For instance, plagiarism detection at schools.

chaxor 4y ago
Why is "at schools" necessarily important though? Wouldn't it also be nice to know if the scientific articles you are reading, or the code a company is based upon, or a new piece of legislature was constructed from someone just asking an LLM to do it for them?
- YPPH 4y ago
  Sure, I was only giving one example to counter the claim, which is not to say there aren't other examples.
- 8note 4y ago
  As usual, "just" pulls a lot of weight - if they're putting the straight LLM output, it probably won't be great regardless. If they put effort into making the ouput good, then chances are the content will be good regardless of whether they wrote it by hand, or wrote it using an LLM. Like, I don't need to know whether you used a bic ballpoint pen, or some other ballpoint pen to write your note on paper; the tool is a tool
chrischen 4y ago
Plagiarism can be solved with non-AI methods because plagiarism is defined more or less by exact copying. If you mean detection of AI generated essays at schools, then teachers just have to expect better essays and be able to identify them since students now have a new tool. Think of it like grading math homework before and after the invention of calculators. Once students had calculators at home the problems simply got harder and higher level, since there was no reasonable way to enforce "don't use a calculator at home with these artificially simplistic problems."

waboremo 4y ago

The more I think about this, the less I agree with it. Think of all the online communication you've had. The good parts. The great but messy blog post that changed your mind about something. Communicating with someone on a social network only to share each other's work and establish a great working relationship. Talking to a friend in DMs sharing your pain. Getting feedback from a mentor. Having people tell you how amazing your art is or how it inspired them. The bad parts too. The negative comments from random people. Others mocking your tweet. People calling your HN comment idiotic. All of these things affect us because we assume a person is behind them, and therefore their intent is clear. Without that, we devalue all communication to compensate. Like a more exaggerated effect of how we view likes on social media, where "20k likes" means nothing. Truly believe we're only ok with it now because it's easy to tell what is AI generated, and they're mostly confined. When AI is creating videos on Twitter, creating threads on Reddit, posting blogs on their personal site to be shared to Mastodon, replying to other bots on HN... I'm not sure we'll feel that it's whether we can detect quality anymore. There's also the differentiation theory where in order to separate human connection from bots, we'll do the opposite. We'll start incorporating prompt injections into everything we post, start purposefully being "low quality" because bots are trained against that. Now we're creating a very particular kind of societal mess.

YetAnotherNick 4y ago
I definitely don't agree with the GP but I don't agree with you either. Human could and absolutely become attached to non humans, ranging from dog to pen to house. And I believe humans might begin to develop deeper relationship with AI bots very soon.
- waboremo 4y ago
  I don't know. Part of why we anthropomorphize so much is because we can still feel the physical elements. We pet a dog, we live inside our homes, list goes on. When that is not possible, how does this change people and their valuation of online communication? Would you still love "Anne" when she's now rejected your 50th attempt to meet in person? Would you still reply to comments like mine if the default assumption wasn't "that's another human being I can persuade" and instead "that is a bot"? I'm sure in the long term that differentiation alone would change, you know androids and all that jazz - however in the meantime, we're going to be facing a tremendous shift in how we interact online.

WalterBright 4y ago

If HN develops into a forum dominated by chatbots, I'll leave.

klipklop 4y ago
How will you know?
- WalterBright 4y ago
  Maybe I am the only real person that exists, and everyone else is just a simulation.