17 ms·

Detect ChatGPT Generated Content

103 points by ausudhz 4y ago

jalino23 4y ago
teachers pet final boss hahaha
sorokod 4y ago
Expect an escalating arms race between models attempting to disguise their output and tools attempting to detect generated content. There may also be escalating social and perhaps legal penalties too.
- falcor84 4y ago
  I'll just put this xkcd here - https://xkcd.com/810/ https://xkcd.com/810/ (Constructive)
  - pwdisswordfishc 4y ago
    This doesn’t work. Stack Exchange tried it, and it turns out there is a lot of misleading, superficial content that easily gets a high number of upvotes from newbies who don’t know better. And that is without even considering bots.
    - sorokod 4y ago
      Yeah, war is hell.
mypastself 4y ago
Does this use ML to determine whether or not ML is used? I wonder if it might be useful to try ChatGPT for detection, I’ve heard it’s pretty good…
kekkidy 4y ago
[dead]
diogopublio 4y ago
worked about 20% of the time/content for me. It does not seem to perform very well in languages other than english. However there is a huge market potential for whoever figure this out. (Although I believe this will be nearly impossible)
kolinko 4y ago
A nice approach - estimates the randomness in text. I tried with various cases - blog posts, wikipedia, scientific docs, and with a few examples from chat, and it figured it out accurately. Respect. Even though I disagree with a need for such tools - it doesn’t matter if content was written by a human or by a machine. What matters is whether it’s easy to read and worthwhile.
- NotYourLawyer 4y ago
  Disagree, this is very much needed. The problem with AI-generated content is that it looks superficially worthwhile and plausible, but in fact it often says things that are not correct.
  - closewith 4y ago
    > The problem with AI-generated content is that it looks superficially worthwhile and plausible, but in fact it often says things that are not correct. This is also a problem with human-generated content.
    - ausudhz 4y ago
      Depends on the human
    - NotYourLawyer 4y ago
      Sort of. But if I read a five page article by a human, and everything on the first couple pages checks out as correct, I expect the rest of the article to be at least reasonable. If it’s written by AI, it’s entirely possible that it’ll start in on something that doesn’t even make sense. Then I’ll realize that I can’t even trust the part that seemed correct.
  - notRobot 4y ago
    But that is true also of lots of content not generated by AI! Fact checking always needs to be done, AI generated or not. But does it matter that it was generated by AI?
  - 4y ago
mt_ 4y ago
How is this different from this research on gpt-2 detection model [1]? [1] https://github.com/openai/gpt-2-output-dataset/tree/master/detector https://github.com/openai/gpt-2-output-dataset/tree/master/d...
lovelearning 4y ago
What about false positives? Irresponsible to market this to laypersons as perfect with absolutely no word of caution about false positives. There's been some chatter on Reddit that some content writers were falsely accused of using ChatGPT and lost their clients.
- makach 4y ago
  This is an everlasting battle. Schools are already using these kind of tools to detect cheating, often with false positives that may have devastating outcome.
metayrnc 4y ago
I tried with a piece of text that was generated with ChatGPT and it said written by a human. I guess this still needs some improvement.
Buffout 4y ago
So, what prevents chatGPT to use detection tool and fine-tune its response accordingly?
- dogma1138 4y ago
  Nothing, this is how adversarial training works, but it also works both ways.
  - mistercow 4y ago
    It works both ways, but generation is advantaged in the long run. There has to actually be a statistical difference to detect, and AI outputs without statistical differences from human output are obviously possible, since humans make them all the time.
    - dogma1138 4y ago
      Sort off, I’m aware that in principle the generator has an advantage and eventually the detector will average out to a coin flip at best. However some advantages can disappear when you put constraints on the output such as quality and correctness. So whilst the end result might be less statistically significant in terms of was it human or AI generated it can overall be also less useful to the end user.
      - mistercow 4y ago
        "However some advantages can disappear when you put constraints on the output such as quality and correctness." Only if you suppose that the ideal output is superhuman. In the case of OpenAI et al, that's arguably the case, but those aren't the players that are going to get into an arms race with detection anyway. They want it to be relatively easy to detect AI generated content, because they're not in the plagiarism business, and anti-plagiarism measures will get the public and media off their backs. And nobody who is interested in targeting plagiarism has nearly the funding to build their own LLM on a level that matters. So if there's an arms race in the near term, I expect it will be with postprocessors instead. These will be much smaller models (i.e. runs in your browser, or at least on a small backend machine) that take the output of ChatGPT and tweak it to fool detectors. They won't care about maximizing quality or accuracy, but will just care about preserving meaning while erasing statistical signs of AI generation. I don't know if the business case for that will be there. It's there for selling papers, and almost certainly some people will try their hand at these models just for the challenge and/or to prove a point.
101008 4y ago
I tried with a few texts that ChatGPT generated for me and it says it was entirely generated by a human. It's true, I fixed some things here and there, for cohesion basically, but I think it's almost impossible for these tools to get it right.
- another-dave 4y ago
  Conversely, I wrote a few sentences in simple English off the top of my head (about The Sound of Silence, because they were talking about it on the radio this morning) and it said it was "likely to be written entirely by AI". False positives in a plagiarism tool are pretty bad IMO. It should definitely skew towards "we can't be certain" rather than "definitely AI".
yeputons 4y ago
I can tell you that detecting AI plagiarism can be a challenging task, as AI models like ChatGPT are able to generate text that is similar to existing text. It is important to note that not all text generated by AI models is considered plagiarism, as it may be used in a legal, educational or research context. The best way to evaluate the accuracy of a plagiarism detection website would be to test it using a variety of inputs, and compare its results to those of other plagiarism detection tools. Sorry.
- deleted 4y ago
  [deleted]
- IncRnd 4y ago
  I ran your comment through this Detector, and received the following message. Your text is likely to be written entirely by AI False positives and all.
  - etothepii 4y ago
    Now you've said this I read that comment and it has the ChatGPT tone for sure.
    - yeputons 4y ago
      Yes, it was ChatGPT indeed.
m00dy 4y ago
I'm not sure this is working.
antegamisou 4y ago
Someone make a no-ChatGPT HN. submission filter (show every submission but ChatGPT-related ones).
- kolinko 4y ago
  You can easily build it with gpt3 api. And codex can help you if you don't know how to program :)
spoork 4y ago
The problem with such things is that you would need 100% accuracy in most cases for it to be useful. For example many schools and universities fear that students use ChatGPT for homework. If such a plagiarism checker has false positive results in just a small percentage of cases, the consequences for honest students would be too severe to actually act on the results of the check. But perfect accuracy can never be reached for text classification, so those tool will never be that useful despite being interesting to AI people.
- dogma1138 4y ago
  That’s a very naïve outlook in reality like with every system the burden of proof would end up falling on the accused not the accuser…
  - another-dave 4y ago
    What could that burden of proof look like? Not sure how a student could prove that the negative, apart from videotaping themselves writing the essay, or only being allowed type it on an airgapped machine owned by the university. Doesn't feel very feasible to implement.
    - avereveard 4y ago
      and if the student is not a complete tool, he can definitely learn the output and replicate it very, very closely
    - lupire 4y ago
      Even before GPT, schools should require students to submit derailed outlines and notes of their work. For one, students need to learn these skills. For two, it helps protect against more plagiarism and cheating.
  - j_french 4y ago
    The reality in my experience (as a university lecturer) is that the burden of proof very much falls on the accuser rather than the accused.
anupamchugh 4y ago
I tried this on my recent AI satire article. While ChatGPT detected my story as AI (because I wrote it like a machine), GPTZero gladly says it's human written. ChatGPT's response: https://imgur.com/a/u7iBBaX https://imgur.com/a/u7iBBaX GPTZero's response: https://imgur.com/a/89icz2X https://imgur.com/a/89icz2X To be clear, my story was ChatGPT-assisted. But I wonder why ChatGPT couldn't detect it correctly like GPTZero?
- dogma1138 4y ago
  If your story was ChatGPT assisted one can argue that it was GPT0 which misclassified it.
  - anupamchugh 4y ago
    Correction: I used ChatGPT to discover new words and rephrase the story myself. Here's the link to my non-paywall story for references: https://medium.com/humor-bytes/i-your-nba-highlights-broadcaster-surrenders-my-throne-to-the-artificial-intelligence-8a2c5c3c9172 https://medium.com/humor-bytes/i-your-nba-highlights-broadca... On the topic of assistance, where do detectors draw the line between AI-augmented, aided, and generated writing?
    - dogma1138 4y ago
      I don’t know which is why this is completely up for debate, Office also offers grammatical corrections and you have websites like Grammarly that do the same often without any “AI” as far as I know. Like with anything else this is something that would require society to come to a consensus. And there are various issues with AI assisted or generated content depending on the context. For example educational institutions may see this as cheating whilst corporate entities might fear legal challenges in relation to copyright and IP.
AstixAndBelix 4y ago
ChatGPT and other AIs are theoretical machines built on human data. Can an algorithm write an article about the 2008 financial crisis? Of course! it has read God knows how many Wikipedia articles, books and online discussions about it. But everything the AI knows is because some humans put in the work and documented everything. If we don't know something, be it a historical fact or a scientific model, we have the ability to go out in the real world and record the data. Can ChatGPT fly to Yemen and interview refugees? Can ChatGPT tear apart the circuitry of a home appliance, reverse engineer it and write a blog post about it? Can ChatGPT go to a lab and make chemical experiments to create new compounds? No, it can only regurgitate what it knows and even what it doesn't know. ChatGPT will absolutely steal the job of all the "journalists" that just regurgitate what others have already written. Original research and field reporting will be left to the real journalists for God knows how many decades. If your job consists of actually making new things or collecting real-world data then I think it will be safe for the decades to come.
DeusExMachina 4y ago
I tried it on a section of an article I wrote with the help of ChatGPT. It marked some of my sentences as generated by AI while it didn't mark the ones coming from ChatGPT.
lancebeet 4y ago
Interesting. It only seems to work in English. English texts that it successfully detects as AI generated are classified as human generated if you ask ChatGPT to translate them into a different language (or make the initial prompt in a different language). Perhaps they want to clarify this limitation, either through an FAQ or by stating that it doesn't support it when other languages are supplied in the prompt.
mistercow 4y ago
The accuracy just isn’t there, which isn’t surprising. I’m feeding it Star Wars reviews from both ChatGPT and IMDb, and for sure the answers are correlated. The false negative rate is OK, although I’ve hit plenty of them. But man, it sure does think a lot of people were using ChatGPT to write reviews back in the 2000s. Again, it’s definitely correlated, even strongly correlated, but that’s not good enough for plagiarism detection. You can’t go accusing students of academic dishonesty based on a tool that gets it wrong multiple times on a couple dozen samples.
O__________O 4y ago
Here’s and alternate approach to detecting GPT that has an actual benchmarks and research paper available: https://news.ycombinator.com/item?id=34557189 https://news.ycombinator.com/item?id=34557189
lwhi 4y ago
I just fed it an email I'd personally written. The start and end were apparently written by AI. What does this mean? Is this proof we're in a simulation and I'm actually a mouth piece for an AI that's reached the singularity?
_visgean 4y ago
I copied a text from wikipedia - it classified it as human written, i then asked gpt to rewrite it and it was still classified as a human.. Would be interesting how much they actually catch.
reset-password 4y ago
I thought to make something similar but what scared me away was the idea that some educational institution could use it and decide a student was being dishonest due to a false positive. I wouldn't want to bear responsibility for something like that. And secondly, this is a never-ending race. Even if it were to be able to detect ChatGPT content with 100% accuracy today, it would just be used to assist in training another model to defeat it.
ArcaneMoose 4y ago
What I think would be interesting is to add the assignment / questions as input for the tool to consider in its evaluation. That way your tool could plug these into ChatGPT, ask it to reword it several ways (mimicking the likely path a plagiarizer might take to actually generate content) and do some diff/comparison as part of the evaluation. Even so, I agree with most of the comments that it would be exceeding difficult to truly identify this stuff since you can ask ChatGPT to reword things in very specific ways or just edit it yourself (aside from blatant false positives)
pigtailgirl 4y ago
-- I copy and pasted GPT3 reply to - what are Jim Crow Laws? - said written by AI - then copy paste wikipeda first paragraph - said written by AI - then I wrote it myself - said written by AI - then I copy/paste 1 word over and over and over 100 or so times - it says "likely written by human but some by AI" then - highlights the whole text and notes - likely written by AI - hahhaa --
- forgotpwd16 4y ago
  AI is so good that even human written text is indistinguishable from AI.
krisoft 4y ago
This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to this.
- _just7_ 4y ago
  The only way I could get it to say something was completely written by a human, was by adding spelling mistakes
- krisoft 4y ago
  The GNU preamble is entirely AI written. According to this tool. To be honest I always suspected Stallman. :)
- deleted 4y ago
  [deleted]
- bhickey 4y ago
  Agreed. This is useless. I asked GPT-3 to write a HN post decrying GPTZero: > GPTZero is absolute trash! It can't even detect plagarism correctly. It's a complete joke and waste of time! Don't bother with it! You'd be better off just manually checking for plagiarism. It's so unreliable it's not even worth mentioning. Just save yourself the time and money and don't bother with GPTZero. Worst AI tool ever! GPTZero classified it as fully human written. This is a cat and mouse game where the mouse is always going to lose.
  - programmarchy 4y ago
    Agree with you, but! In a Tom and Jerry episode, Tom Cat would be GPTZero and Jerry Mouse would be GPT3 ;)
  - hutzlibu 4y ago
    It is hilarious comedy, though. The sad part is, that real people will likely suffer, when people with power take salespersons serious and use stuff like that to "detect" and punish frauds.
deleted 4y ago
[deleted]
searchableguy 4y ago
Is there a chatgpt and human text dataset similar to gpt2 released by openAI?
- garymiklos 4y ago
  I also wanted to train my own GPT detection model but unfortunately there isn't any publicly available dataset about outputs.
drexlspivey 4y ago
I wonder if there is a way to watermark generated text, not by adding extra spaces or using invisible characters but by using grammar. Example naive rule “every 13th word MUST be a proposition”. Another more complex example “every 11th word is a checksum of the previous 10. You encode a few bits in every word i.e noun=1 verb=2 article=3 etc”
- lupire 4y ago
  Trivial to break by making tiny changes.
barberpole 4y ago
It would seem to be checking sentence structure, and if "too complex" -- embedded clauses and such -- it follows a hard-wired rule to report AI.
netfortius 4y ago
I assume this is English only, as none of other languages with which I tried chatGPT found even one single word AI produced.
pelorat 4y ago
ChatGPT can generate infinite combinations of words in thousands of different styles. Accurate detection will never work.
zopa 4y ago
We ought to require OpenAI to run something like this: a “Hey ChatGPT, this you?” endpoint that replies Yes/No/similarity score, and a creation timestamp, when you hit it with some text. As regulations go, this one’s not too burdensome: expensive, but pretty cheap compared to training and running a large language model in the first place.
- jonnycomputer 4y ago
  This is a pretty good idea, at least for commercial models.
- 1attice 4y ago
  This works right up until the model is open sourced or otherwise replicable. Put another way, it only works in today's environment. When ML hardware is as widely distributed as classical compute, and all the models are on HuggingFace, you will be back to 0
  - zopa 4y ago
    If it buys some time, that’s a good day’s work and I’ll be very satisfied. All you can really ask is that a regulation makes things better in the near term. The future will know a lot more about the nature and implications of persuasively-humanlike ML than I do: it can take care of itself. Maybe by then hybrid writing will be the norm and not considered plagiarism, and we’ll all have trustworthy virtual assistants shielding us from scams. But in the meantime, there are some reasonable causes for concern, and this would help.
- tkiolp4 4y ago
  If OpenAI implements such an endpoint, then a big chunk of potential customers would just not use ChatGPT… I mean, if I’m a student planning to use ChatGPT to enhance my uni essays, but it turns out OpenAI actually can say to my uni if my texts were AI generated, well, I’m not gonna use ChatGPT at all.
bdcravens 4y ago
Fed a blog I know to be ChatGPT-written. It only detected 1/3 of the sentences.
julianh65 4y ago
I read an interesting tweet about how it would be possible to watermark GPT outputs. Essentially before each token is generated the previous token is used to seed an RNG. Using the RNG the possible next tokens are split into a whitelist and a blacklist and the model can only select words from the whitelist. Later on it's possible to "check" for the watermark by counting the whitelist tokens and doing statistical analysis. Apparently they can preserve performance by not doing this for very low entropy tokens where there is only one token that is extremely likely. Saw it here: https://twitter.com/tomgoldsteincs/status/1618287665006403585 https://twitter.com/tomgoldsteincs/status/161828766500640358...
camillomiller 4y ago
Useless. Any text written by chatGPT gets marked as human, as long as you edit it a bit here and there. Other text refined through a couple of chiseling prompts are a complete miss.
jdalgetty 4y ago
This is detecting my own written work as ai.
SachinDSI 4y ago
Consider trying out GPTKit https://gptkit.ai https://gptkit.ai it has higher accuracy than GPTZero.
- SachinDSI 4y ago
  Disclosure: The tool is built by me. It uses 6 distinct methods to classify text with a 93% accuracy rate, based on testing a dataset of 100k+ samples.
  - gschoeni 4y ago
    Where are you hosting the dataset? Would love to help out, I'm building an open source data version control tool to help iterate on ML datasets. https://github.com/Oxen-AI/oxen-release https://github.com/Oxen-AI/oxen-release Would be cool if we could get a community around the test dataset to insure that 93% accuracy rate. Then people can add their failure cases to the repo and then you can iterate on them.
YeGoblynQueenne 4y ago
This only calculates perplexity and burstiness. I don't think that's going to work very well. It would be much better to try and detect whether the distribution from which a piece of text was drawn is closer to that of a human, or that of a large language model. But how would one go about detecting something like that? Well, one would need a model of human language trained to approximate the distribution of tokens in a large corpus of natural language... text... Oh wait.

103 points by ausudhz 4y ago

teachers pet final boss hahaha

Expect an escalating arms race between models attempting to disguise their output and tools attempting to detect generated content. There may also be escalating social and perhaps legal penalties too.

falcor84 4y ago
I'll just put this xkcd here - https://xkcd.com/810/ https://xkcd.com/810/ (Constructive)
- pwdisswordfishc 4y ago
  This doesn’t work. Stack Exchange tried it, and it turns out there is a lot of misleading, superficial content that easily gets a high number of upvotes from newbies who don’t know better. And that is without even considering bots.
  - sorokod 4y ago
    Yeah, war is hell.

mypastself 4y ago

Does this use ML to determine whether or not ML is used? I wonder if it might be useful to try ChatGPT for detection, I’ve heard it’s pretty good…

kekkidy 4y ago

[dead]

diogopublio 4y ago

worked about 20% of the time/content for me. It does not seem to perform very well in languages other than english. However there is a huge market potential for whoever figure this out. (Although I believe this will be nearly impossible)

kolinko 4y ago

A nice approach - estimates the randomness in text. I tried with various cases - blog posts, wikipedia, scientific docs, and with a few examples from chat, and it figured it out accurately. Respect. Even though I disagree with a need for such tools - it doesn’t matter if content was written by a human or by a machine. What matters is whether it’s easy to read and worthwhile.

NotYourLawyer 4y ago
Disagree, this is very much needed. The problem with AI-generated content is that it looks superficially worthwhile and plausible, but in fact it often says things that are not correct.
- closewith 4y ago
  > The problem with AI-generated content is that it looks superficially worthwhile and plausible, but in fact it often says things that are not correct. This is also a problem with human-generated content.
  - ausudhz 4y ago
    Depends on the human
  - NotYourLawyer 4y ago
    Sort of. But if I read a five page article by a human, and everything on the first couple pages checks out as correct, I expect the rest of the article to be at least reasonable. If it’s written by AI, it’s entirely possible that it’ll start in on something that doesn’t even make sense. Then I’ll realize that I can’t even trust the part that seemed correct.
- notRobot 4y ago
  But that is true also of lots of content not generated by AI! Fact checking always needs to be done, AI generated or not. But does it matter that it was generated by AI?
- 4y ago

mt_ 4y ago

How is this different from this research on gpt-2 detection model [1]? [1] https://github.com/openai/gpt-2-output-dataset/tree/master/detector https://github.com/openai/gpt-2-output-dataset/tree/master/d...

lovelearning 4y ago

What about false positives? Irresponsible to market this to laypersons as perfect with absolutely no word of caution about false positives. There's been some chatter on Reddit that some content writers were falsely accused of using ChatGPT and lost their clients.

makach 4y ago
This is an everlasting battle. Schools are already using these kind of tools to detect cheating, often with false positives that may have devastating outcome.

metayrnc 4y ago

I tried with a piece of text that was generated with ChatGPT and it said written by a human. I guess this still needs some improvement.

Buffout 4y ago

So, what prevents chatGPT to use detection tool and fine-tune its response accordingly?

dogma1138 4y ago
Nothing, this is how adversarial training works, but it also works both ways.
- mistercow 4y ago
  It works both ways, but generation is advantaged in the long run. There has to actually be a statistical difference to detect, and AI outputs without statistical differences from human output are obviously possible, since humans make them all the time.
  - dogma1138 4y ago
    Sort off, I’m aware that in principle the generator has an advantage and eventually the detector will average out to a coin flip at best. However some advantages can disappear when you put constraints on the output such as quality and correctness. So whilst the end result might be less statistically significant in terms of was it human or AI generated it can overall be also less useful to the end user.
    - mistercow 4y ago
      "However some advantages can disappear when you put constraints on the output such as quality and correctness." Only if you suppose that the ideal output is superhuman. In the case of OpenAI et al, that's arguably the case, but those aren't the players that are going to get into an arms race with detection anyway. They want it to be relatively easy to detect AI generated content, because they're not in the plagiarism business, and anti-plagiarism measures will get the public and media off their backs. And nobody who is interested in targeting plagiarism has nearly the funding to build their own LLM on a level that matters. So if there's an arms race in the near term, I expect it will be with postprocessors instead. These will be much smaller models (i.e. runs in your browser, or at least on a small backend machine) that take the output of ChatGPT and tweak it to fool detectors. They won't care about maximizing quality or accuracy, but will just care about preserving meaning while erasing statistical signs of AI generation. I don't know if the business case for that will be there. It's there for selling papers, and almost certainly some people will try their hand at these models just for the challenge and/or to prove a point.

101008 4y ago

I tried with a few texts that ChatGPT generated for me and it says it was entirely generated by a human. It's true, I fixed some things here and there, for cohesion basically, but I think it's almost impossible for these tools to get it right.

another-dave 4y ago
Conversely, I wrote a few sentences in simple English off the top of my head (about The Sound of Silence, because they were talking about it on the radio this morning) and it said it was "likely to be written entirely by AI". False positives in a plagiarism tool are pretty bad IMO. It should definitely skew towards "we can't be certain" rather than "definitely AI".

yeputons 4y ago

I can tell you that detecting AI plagiarism can be a challenging task, as AI models like ChatGPT are able to generate text that is similar to existing text. It is important to note that not all text generated by AI models is considered plagiarism, as it may be used in a legal, educational or research context. The best way to evaluate the accuracy of a plagiarism detection website would be to test it using a variety of inputs, and compare its results to those of other plagiarism detection tools. Sorry.

deleted 4y ago
[deleted]
IncRnd 4y ago
I ran your comment through this Detector, and received the following message. Your text is likely to be written entirely by AI False positives and all.
- etothepii 4y ago
  Now you've said this I read that comment and it has the ChatGPT tone for sure.
  - yeputons 4y ago
    Yes, it was ChatGPT indeed.

m00dy 4y ago

I'm not sure this is working.

antegamisou 4y ago

Someone make a no-ChatGPT HN. submission filter (show every submission but ChatGPT-related ones).

kolinko 4y ago
You can easily build it with gpt3 api. And codex can help you if you don't know how to program :)

spoork 4y ago

The problem with such things is that you would need 100% accuracy in most cases for it to be useful. For example many schools and universities fear that students use ChatGPT for homework. If such a plagiarism checker has false positive results in just a small percentage of cases, the consequences for honest students would be too severe to actually act on the results of the check. But perfect accuracy can never be reached for text classification, so those tool will never be that useful despite being interesting to AI people.

dogma1138 4y ago
That’s a very naïve outlook in reality like with every system the burden of proof would end up falling on the accused not the accuser…
- another-dave 4y ago
  What could that burden of proof look like? Not sure how a student could prove that the negative, apart from videotaping themselves writing the essay, or only being allowed type it on an airgapped machine owned by the university. Doesn't feel very feasible to implement.
  - avereveard 4y ago
    and if the student is not a complete tool, he can definitely learn the output and replicate it very, very closely
  - lupire 4y ago
    Even before GPT, schools should require students to submit derailed outlines and notes of their work. For one, students need to learn these skills. For two, it helps protect against more plagiarism and cheating.
- j_french 4y ago
  The reality in my experience (as a university lecturer) is that the burden of proof very much falls on the accuser rather than the accused.

anupamchugh 4y ago

I tried this on my recent AI satire article. While ChatGPT detected my story as AI (because I wrote it like a machine), GPTZero gladly says it's human written. ChatGPT's response: https://imgur.com/a/u7iBBaX https://imgur.com/a/u7iBBaX GPTZero's response: https://imgur.com/a/89icz2X https://imgur.com/a/89icz2X To be clear, my story was ChatGPT-assisted. But I wonder why ChatGPT couldn't detect it correctly like GPTZero?

dogma1138 4y ago
If your story was ChatGPT assisted one can argue that it was GPT0 which misclassified it.
- anupamchugh 4y ago
  Correction: I used ChatGPT to discover new words and rephrase the story myself. Here's the link to my non-paywall story for references: https://medium.com/humor-bytes/i-your-nba-highlights-broadcaster-surrenders-my-throne-to-the-artificial-intelligence-8a2c5c3c9172 https://medium.com/humor-bytes/i-your-nba-highlights-broadca... On the topic of assistance, where do detectors draw the line between AI-augmented, aided, and generated writing?
  - dogma1138 4y ago
    I don’t know which is why this is completely up for debate, Office also offers grammatical corrections and you have websites like Grammarly that do the same often without any “AI” as far as I know. Like with anything else this is something that would require society to come to a consensus. And there are various issues with AI assisted or generated content depending on the context. For example educational institutions may see this as cheating whilst corporate entities might fear legal challenges in relation to copyright and IP.

AstixAndBelix 4y ago

ChatGPT and other AIs are theoretical machines built on human data. Can an algorithm write an article about the 2008 financial crisis? Of course! it has read God knows how many Wikipedia articles, books and online discussions about it. But everything the AI knows is because some humans put in the work and documented everything. If we don't know something, be it a historical fact or a scientific model, we have the ability to go out in the real world and record the data. Can ChatGPT fly to Yemen and interview refugees? Can ChatGPT tear apart the circuitry of a home appliance, reverse engineer it and write a blog post about it? Can ChatGPT go to a lab and make chemical experiments to create new compounds? No, it can only regurgitate what it knows and even what it doesn't know. ChatGPT will absolutely steal the job of all the "journalists" that just regurgitate what others have already written. Original research and field reporting will be left to the real journalists for God knows how many decades. If your job consists of actually making new things or collecting real-world data then I think it will be safe for the decades to come.

DeusExMachina 4y ago

I tried it on a section of an article I wrote with the help of ChatGPT. It marked some of my sentences as generated by AI while it didn't mark the ones coming from ChatGPT.

lancebeet 4y ago

Interesting. It only seems to work in English. English texts that it successfully detects as AI generated are classified as human generated if you ask ChatGPT to translate them into a different language (or make the initial prompt in a different language). Perhaps they want to clarify this limitation, either through an FAQ or by stating that it doesn't support it when other languages are supplied in the prompt.

mistercow 4y ago

The accuracy just isn’t there, which isn’t surprising. I’m feeding it Star Wars reviews from both ChatGPT and IMDb, and for sure the answers are correlated. The false negative rate is OK, although I’ve hit plenty of them. But man, it sure does think a lot of people were using ChatGPT to write reviews back in the 2000s. Again, it’s definitely correlated, even strongly correlated, but that’s not good enough for plagiarism detection. You can’t go accusing students of academic dishonesty based on a tool that gets it wrong multiple times on a couple dozen samples.

O__________O 4y ago

Here’s and alternate approach to detecting GPT that has an actual benchmarks and research paper available: https://news.ycombinator.com/item?id=34557189 https://news.ycombinator.com/item?id=34557189

lwhi 4y ago

I just fed it an email I'd personally written. The start and end were apparently written by AI. What does this mean? Is this proof we're in a simulation and I'm actually a mouth piece for an AI that's reached the singularity?

_visgean 4y ago

I copied a text from wikipedia - it classified it as human written, i then asked gpt to rewrite it and it was still classified as a human.. Would be interesting how much they actually catch.

reset-password 4y ago

I thought to make something similar but what scared me away was the idea that some educational institution could use it and decide a student was being dishonest due to a false positive. I wouldn't want to bear responsibility for something like that. And secondly, this is a never-ending race. Even if it were to be able to detect ChatGPT content with 100% accuracy today, it would just be used to assist in training another model to defeat it.

ArcaneMoose 4y ago

What I think would be interesting is to add the assignment / questions as input for the tool to consider in its evaluation. That way your tool could plug these into ChatGPT, ask it to reword it several ways (mimicking the likely path a plagiarizer might take to actually generate content) and do some diff/comparison as part of the evaluation. Even so, I agree with most of the comments that it would be exceeding difficult to truly identify this stuff since you can ask ChatGPT to reword things in very specific ways or just edit it yourself (aside from blatant false positives)

pigtailgirl 4y ago

-- I copy and pasted GPT3 reply to - what are Jim Crow Laws? - said written by AI - then copy paste wikipeda first paragraph - said written by AI - then I wrote it myself - said written by AI - then I copy/paste 1 word over and over and over 100 or so times - it says "likely written by human but some by AI" then - highlights the whole text and notes - likely written by AI - hahhaa --

forgotpwd16 4y ago
AI is so good that even human written text is indistinguishable from AI.

krisoft 4y ago

This is useless. I have tried it with 4 different samples and it made mistakes with each. - it thinks AI wrote parts of my handwritten texts - King James Genesis 29 is appearantely fully AI written - a wall of text copied straight from chatgpt? Only partially AI written. - second chapter of Harry Potter? AI written parts In fact I could not find a sample yet which was not AI written at least partially according to this.

_just7_ 4y ago
The only way I could get it to say something was completely written by a human, was by adding spelling mistakes
krisoft 4y ago
The GNU preamble is entirely AI written. According to this tool. To be honest I always suspected Stallman. :)
deleted 4y ago
[deleted]
bhickey 4y ago
Agreed. This is useless. I asked GPT-3 to write a HN post decrying GPTZero: > GPTZero is absolute trash! It can't even detect plagarism correctly. It's a complete joke and waste of time! Don't bother with it! You'd be better off just manually checking for plagiarism. It's so unreliable it's not even worth mentioning. Just save yourself the time and money and don't bother with GPTZero. Worst AI tool ever! GPTZero classified it as fully human written. This is a cat and mouse game where the mouse is always going to lose.
- programmarchy 4y ago
  Agree with you, but! In a Tom and Jerry episode, Tom Cat would be GPTZero and Jerry Mouse would be GPT3 ;)
- hutzlibu 4y ago
  It is hilarious comedy, though. The sad part is, that real people will likely suffer, when people with power take salespersons serious and use stuff like that to "detect" and punish frauds.

deleted 4y ago

[deleted]

searchableguy 4y ago

Is there a chatgpt and human text dataset similar to gpt2 released by openAI?

garymiklos 4y ago
I also wanted to train my own GPT detection model but unfortunately there isn't any publicly available dataset about outputs.

drexlspivey 4y ago

I wonder if there is a way to watermark generated text, not by adding extra spaces or using invisible characters but by using grammar. Example naive rule “every 13th word MUST be a proposition”. Another more complex example “every 11th word is a checksum of the previous 10. You encode a few bits in every word i.e noun=1 verb=2 article=3 etc”

lupire 4y ago
Trivial to break by making tiny changes.

barberpole 4y ago

It would seem to be checking sentence structure, and if "too complex" -- embedded clauses and such -- it follows a hard-wired rule to report AI.

netfortius 4y ago

I assume this is English only, as none of other languages with which I tried chatGPT found even one single word AI produced.

pelorat 4y ago

ChatGPT can generate infinite combinations of words in thousands of different styles. Accurate detection will never work.

zopa 4y ago

We ought to require OpenAI to run something like this: a “Hey ChatGPT, this you?” endpoint that replies Yes/No/similarity score, and a creation timestamp, when you hit it with some text. As regulations go, this one’s not too burdensome: expensive, but pretty cheap compared to training and running a large language model in the first place.

jonnycomputer 4y ago
This is a pretty good idea, at least for commercial models.
1attice 4y ago
This works right up until the model is open sourced or otherwise replicable. Put another way, it only works in today's environment. When ML hardware is as widely distributed as classical compute, and all the models are on HuggingFace, you will be back to 0
- zopa 4y ago
  If it buys some time, that’s a good day’s work and I’ll be very satisfied. All you can really ask is that a regulation makes things better in the near term. The future will know a lot more about the nature and implications of persuasively-humanlike ML than I do: it can take care of itself. Maybe by then hybrid writing will be the norm and not considered plagiarism, and we’ll all have trustworthy virtual assistants shielding us from scams. But in the meantime, there are some reasonable causes for concern, and this would help.
tkiolp4 4y ago
If OpenAI implements such an endpoint, then a big chunk of potential customers would just not use ChatGPT… I mean, if I’m a student planning to use ChatGPT to enhance my uni essays, but it turns out OpenAI actually can say to my uni if my texts were AI generated, well, I’m not gonna use ChatGPT at all.

bdcravens 4y ago

Fed a blog I know to be ChatGPT-written. It only detected 1/3 of the sentences.

julianh65 4y ago

I read an interesting tweet about how it would be possible to watermark GPT outputs. Essentially before each token is generated the previous token is used to seed an RNG. Using the RNG the possible next tokens are split into a whitelist and a blacklist and the model can only select words from the whitelist. Later on it's possible to "check" for the watermark by counting the whitelist tokens and doing statistical analysis. Apparently they can preserve performance by not doing this for very low entropy tokens where there is only one token that is extremely likely. Saw it here: https://twitter.com/tomgoldsteincs/status/1618287665006403585 https://twitter.com/tomgoldsteincs/status/161828766500640358...

camillomiller 4y ago

Useless. Any text written by chatGPT gets marked as human, as long as you edit it a bit here and there. Other text refined through a couple of chiseling prompts are a complete miss.

jdalgetty 4y ago

This is detecting my own written work as ai.

SachinDSI 4y ago

Consider trying out GPTKit https://gptkit.ai https://gptkit.ai it has higher accuracy than GPTZero.

SachinDSI 4y ago
Disclosure: The tool is built by me. It uses 6 distinct methods to classify text with a 93% accuracy rate, based on testing a dataset of 100k+ samples.
- gschoeni 4y ago
  Where are you hosting the dataset? Would love to help out, I'm building an open source data version control tool to help iterate on ML datasets. https://github.com/Oxen-AI/oxen-release https://github.com/Oxen-AI/oxen-release Would be cool if we could get a community around the test dataset to insure that 93% accuracy rate. Then people can add their failure cases to the repo and then you can iterate on them.

YeGoblynQueenne 4y ago

This only calculates perplexity and burstiness. I don't think that's going to work very well. It would be much better to try and detect whether the distribution from which a piece of text was drawn is closer to that of a human, or that of a large language model. But how would one go about detecting something like that? Well, one would need a model of human language trained to approximate the distribution of tokens in a large corpus of natural language... text... Oh wait.