4 ms·

I wrote a userscript (with help from ChatGPT) that identifies if comments on HN are written by AI or a human. I based it on https://huggingface.co/openai-detect

I wrote a userscript (with help from ChatGPT) that identifies if comments on HN are written by AI or a human. I based it on https://huggingface.co/openai-detector https://huggingface.co/openai-detector. Its still a little shabby and only works on HN, but I imagine this is going to be required for general non-specific Internet browsing going forward. Looks like this: https://i.imgur.com/BTt1DTh.png https://i.imgur.com/BTt1DTh.png

nicolas_lorenzi 4y ago
Very interesting! How does it work?
- hxugufjfjf 4y ago
  It puts every comment into that GPT output detector and colors and writes a short comment on the HN comment, like you see in the screenshot based on a threshold. >0.7 is probably AI, >0.9 is definitely AI. Lower than that is most likely human. Most comments still appear to be human. It only becomes reliable after about 50 tokens (one token is around 4 characters) so I mark the comments that are too short with gray and make no assessment on those. I've put it on https://github.com/chryzsh/GPTCommentDetector https://github.com/chryzsh/GPTCommentDetector
  - nicolas_lorenzi 4y ago
    I see. My question of how it works was more about the method you were using to identify content as something written by an AI, but from what I saw on your repo, you rely on a set of GPT-specific configurations that identify a percentage of similarity to the content being analyzed?
    - hxugufjfjf 4y ago
      Like desrcibed in the repo, I just feed it to the GPT output detector. I didn't write that tool, but from my understanding they trained a GPT model to recognize itself.
      - nicolas_lorenzi 4y ago
        Okay cool, I had heard the same kind of AI training during the release of DALL.E 2 where one of the AI was dedicated to the generation of the image and another AI which checked if the generated image corresponded to an AI or not.
nicolas_lorenzi 4y ago
+1 star for this repo btw
- hxugufjfjf 4y ago
  Thank you https://github.com/chryzsh/GPTCommentDetector https://github.com/chryzsh/GPTCommentDetector
  - nicolas_lorenzi 4y ago
    With pleasure, we are also working on an open-source project (called Luos engine). Receiving support just by clicking on a star is a quick click for a big effect.

nicolas_lorenzi 4y ago

Very interesting! How does it work?

hxugufjfjf 4y ago
It puts every comment into that GPT output detector and colors and writes a short comment on the HN comment, like you see in the screenshot based on a threshold. >0.7 is probably AI, >0.9 is definitely AI. Lower than that is most likely human. Most comments still appear to be human. It only becomes reliable after about 50 tokens (one token is around 4 characters) so I mark the comments that are too short with gray and make no assessment on those. I've put it on https://github.com/chryzsh/GPTCommentDetector https://github.com/chryzsh/GPTCommentDetector
- nicolas_lorenzi 4y ago
  I see. My question of how it works was more about the method you were using to identify content as something written by an AI, but from what I saw on your repo, you rely on a set of GPT-specific configurations that identify a percentage of similarity to the content being analyzed?
  - hxugufjfjf 4y ago
    Like desrcibed in the repo, I just feed it to the GPT output detector. I didn't write that tool, but from my understanding they trained a GPT model to recognize itself.
    - nicolas_lorenzi 4y ago
      Okay cool, I had heard the same kind of AI training during the release of DALL.E 2 where one of the AI was dedicated to the generation of the image and another AI which checked if the generated image corresponded to an AI or not.

nicolas_lorenzi 4y ago

+1 star for this repo btw

hxugufjfjf 4y ago
Thank you https://github.com/chryzsh/GPTCommentDetector https://github.com/chryzsh/GPTCommentDetector
- nicolas_lorenzi 4y ago
  With pleasure, we are also working on an open-source project (called Luos engine). Receiving support just by clicking on a star is a quick click for a big effect.