11 ms·

A college student made an app to detect AI-written text

103 points by greenie_beans 4y ago

richardatlarge 4y ago
Yes, but can it prove it? The old fashioned plagiarism problem was solved by finding the original it was copied from. With AI, the falsely attributed work IS original. There in lies…
- throwmeup123 4y ago
  It doesn't and it tags nearly everything not written by an 8th grader as written by AI... even this exact article.
  - informalo 4y ago
    Well maybe the article was written using ChatGPT.
  - henryteeare 4y ago
    Really? Perhaps they updated it, because I got "likely human generated" for the parent article. "Your GPTZero score corresponds to the likelihood of the text being AI generated: 338.59305536714294 Your text is likely human generated!"
- paul_f 4y ago
  AI-generated content can still be traced back to its source. Even though the content may be original, AI-generated works can be identified by their algorithmic patterns, which can be used to determine the source of the work. ChatGPT wrote that, not me. How do we know we aren't simply reading ChatGPT arguing with itself?
  - godelski 4y ago
    > It is certainly possible for AI models to generate text that appears to be self-referential or self-reflexive. However, it is also possible to trace the source of AI-generated text by examining the specific patterns and characteristics of the text. For example, you can analyze the language and style used in the text, as well as the context in which it was generated, to determine whether it was likely produced by a specific AI model or not. In addition, you can also examine the metadata associated with the text, such as the date and time it was created, to help identify its source. ChatGPT wrote that. But I'll say that this is a lot easier said than done. This will always be an increasingly difficult cat and mouse game.
endisneigh 4y ago
Without knowing the false positive and false negative rate this article is meaningless IMHO. Furthermore, even if it had a 0% false negative and false positive rate, it then would be used to train the next iteration of language models. A good idea superficially, but ultimately solves the wrong problem.
- deleted 4y ago
  [deleted]
ttul 4y ago
My son and his friends are already doing their homework using ChatGPT. Teachers will have to get used to this and hopefully embrace it.
- TSiege 4y ago
  No judgement here, but I'm curious how they're using it and what your feelings are. It seems like you've embraced it
  - ttul 4y ago
    Literally everything. They’re using it to help with math homework. Putting together copy for a slide presentation about the history of Pepsico (why? I have no idea). Writing a silly story about an ape man who dominates the world through vicious conquests. “Now write a sequel” apparently works really well.
- jolmg 4y ago
  In the end, (ideally) it's the students that are losing out on education, though. That's unless there's a competition dynamic among students. Maybe embracing it is removing competition among students. EDIT: On second thought, one can just put more emphasis on exams taken in-premises with supervision.
  - Der_Einzige 4y ago
    School is not about getting an "education", this is pedegological propaganda. It's about maximizing your college admissions chances, which then leads to better jobs. Id rather be stupid and rich than "educated" and poor.
    - skor 4y ago
      would you rather be stupid? think of all the things you would have wished to know before you die. seems like a pipe dream to me.
e1g 4y ago
Since TFA mentioned ChatGPT by name, it’s worth noting that OpenAI itself is developing software to detect text generated by ChatGPT https://www.theregister.com/2023/01/09/in_brief_ai/ https://www.theregister.com/2023/01/09/in_brief_ai/
- gnicholas 4y ago
  Ah, there's the true business model. Students can use ChatGPT for free, but schools get charged to be able to detect it. Brilliant! Similarly, OpenAI's tools can be used to create all sorts of content marketing and other webspam. They they'll create a monetizable search engine-killer that knows how to ignore all this fluff (which will probably constitute 99% of the internet within a few years).
  - cloudking 4y ago
    GPT-3 content marketing tools already exist https://jasper.ai https://jasper.ai
    - gnicholas 4y ago
      Oh I know — that's why I assume the web will be choking on AI-generated content marketing in the foreseeable future. The killer app will be the search/information/answer engine that can suitably ignore the AI-generated tripe. I wonder if we'll look back one day and say "remember back before 2020, when the internet was filled with junk, but at least it was human-written junk?".
Der_Einzige 4y ago
No reliable technique exists for AI LM text detection assuming that the LM can be fine tuned and that the decoder settings are available. I doubt a reliable technique for this is even ever possible.
teeray 4y ago
I can’t see how this will be useful unless the false positive rate is 0%. Can you imagine how demoralizing it would be to legitimately write a term paper and have it flagged as being AI-written? How do you disprove it?
- dvngnt_ 4y ago
  that literally just happened on reddit r/art
Al-Khwarizmi 4y ago
Some people are putting a lot of hope in AI text detectors to prevent ChatGPT usage in contexts like student essays, etc. While these detectors are of course interesting from the hacker/academic points of view, I don't think they will really solve much in practice. On the one hand, our (teachers') adversary is not ChatGPT. It's ChatGPT + a human, which will be aware of these techniques and can tweak the text to avoid detection. On the other hand, even if we could detect generated texts with high accuracy, are we going to fail a student just because a black-box AI model says so? Doesn't seem very ethical to me. It's funny how some people worried about ChatGPT's ethical implications seem to be OK with this, just because it goes against their personal demon. I don't think there is any realistic alternative to just assuming that these systems will be used wherever text is required and you can't physically watch the writer, and living with it.
- sarchertech 4y ago
  I took an accounting class and it required you to come in on Saturday for really long tests. My guess is that students just have to spend 3-5 hours on a Saturday writing essays, doing programming assignments etc…
- drbeast 4y ago
  I went through college using Wikipedia to not only make my outline but also provide citations. I'd then rewrite the articles in my own word. This greatly reduced the amount of preparation needed prior to writing. No shame, the majority of research papers done in unis are silly and regurgitation. Really technology is just getting good at eliminating busy work which will be the death kneel of liberal arts. Double rant: college is largely a waste of time for liberal arts classes.
  - brokensegue 4y ago
    >death kneel of liberal arts
    - drbeast 4y ago
      Ah yes, one of those jack offs who disregard arguments because of a phone posting auto correct. Do us a favor and get the COVID vaccines. They're safe and effective!
lr4444lr 4y ago
Just my opinion, but the kinds of writing assignments ChatGPT excels at seem to me the least useful ones for assessing student understanding. Why are we making kids write entire persuasive essays and research papers when in the real world these are almost always the result of collaborative effort?
- namelessoracle 4y ago
  How are they supposed to colloborate of these without doing some of these on their own?
- asdff 4y ago
  Because writing is an important skill in life.
marcell 4y ago
ChatGPT is similar to a calculator. We don't let 3rd graders use calculators on their arithmetic homework, even though there's no way to detect if they used one. The point of a 3rd grader adding fractions is not because we care about the result of 3/4 + 1/3, but because the 3rd grader should learn how to add fractions. If you think of ChatGPT as a fancy calculator, you can just apply the same techniques we've been using for a while: tell students not to use them on homework, make homework only a small % of the grade, and rely on in-class, paper only exams for most the grade.
- namaria 4y ago
  I think text generators will do two things: reduce the average quality of published text, and reduce the average text writing skills. It thus increases the value of writing skills. I count that as a win.
- seanmcdirmid 4y ago
  So like...writing an essay in class. That could be useful, although they won't be able to do any research or anything.
  - janalsncm 4y ago
    Students could print out their sources and turn them in with the essay. Of course this would require knowing ahead of time what sources you’d need.
  - stephencanon 4y ago
    The best class I had in high school (Mr. Hecker’s English class), required writing an essay in class every week. All my college essays were easy by comparison.
    - seanmcdirmid 4y ago
      To each their own. I didn’t really learn how to write until in grad school, when I had things I really wanted to write about. I can imagine how much joy would have been sucked out of writing if I just had to write some essay every week that I didn’t have much interest in.
poulsbohemian 4y ago
I find the texts written by AI systems so clunky and superficial that I would be disappointed if they aren't immediately obvious to an educator.
- Psychoshy_bc1q 4y ago
  The efficacy of ChatGPT is contingent upon one's utilization of its capabilities. Its ability to generate text of exceptional quality is undeniable.
knaik94 4y ago
I don't think cheaters will use any text from ChatGPT verbatim. The majority of "cheating" is going to be similar to how students use Wikipedia as a reference. This might be useful to filter spam from places that initially got bombarded, like stack overflow. I agree with other comments, it's hard to judge the value of this tool without false positive/negative stats. The output isn't completely deterministic though so I wonder how "certain" this tool can be.
- a_bonobo 4y ago
  You'd be surprised how many students turn in Wikipedia-articles run through software that replaces words with (often wrong) synonyms. I've tutored students who paid such find-and-replace services. Zero effort for the majority of students that cheat, very easy to detect, rarely punished (a zero on an essay here or there, no expulsion - students are customers after all). There will be paid ChatGPT services for cheating students, the ideal solution would be to rethink and rehaul how student performance is evaluated, but universities will instead pay something like Turnitin because throwing money at a problem is always easier than thinking.
  - analog31 4y ago
    I taught college for a brief time. Catching cheaters is hard work, and unpaid, and it's perceived that doing it too much affects your chances of getting your contract renewed. "Let the employers or the next course level sort them out" seems to be a more pragmatic strategy.
    - kspacewalk2 4y ago
      Seeig a fellow TA pursue a cheater who copied a major math assignment symbol for symbol from another student, mistakes and all, and fail to break through the bureaucracy and achieve actual punishment was quite an eye opening experience. If this is how brazen cheaters are handled in one of the top Canadian engineering schools, what hope is there?
      - 4y ago
cloudking 4y ago
It's a cool idea, unfortunately it doesn't work very well. I ran a few generated examples through it and they came back as human.
Wowfunhappy 4y ago
Regardless of whether students should be using ChatGPT, I strongly disagree that a ChatGPT-written essay is "plagiarism". ChatGPT is not an author, it's a computer program. We don't expect bibliographies to include Microsoft Word, Grammerly, or iOS Keyboard Predictive Text in the list of references. If I use a piece of software to create an essay, the essay that's created belongs to me.
- elefanten 4y ago
  I guess it depends if you consider the act of writing to be something that can only be done by a human. If a program can produce a work that passes as "writing", then that program could be called the "author" of the work. So a student that takes said output and hands it in is plagiarizing by claiming credit for something written by a different "author". It'll probably take some time for language to evolve around non-human agents. There's no intrinsic reason to prefer separating (or combining) that terminology by humanness of agent, afaik. At least, for common usage. One will win and only time will tell.
- kuhewa 4y ago
  Plagarism is passing off others' ideas or words off as your own without attribution. IMO there is an argument to be made whether conjuring up text from a model of others' words should qualify as "others' ideas or words", but if it isn't technically academic plagarism I reckon it is a distinction without a difference. The concern with academic plagarism isn't primarily IP, so I reckon it doesn't inherently matter whether "others" being borrowed from is a particular author. Also, There's a commonly understood threshold for plagiarism where a single word won't qualify and a phrase probably won't qualify, but a whole paragraph does. That is why Grammarly isn't plagarism, not because it is software.
- andai 4y ago
  It's not plagiarism in the sense that you took stole else's work, it's plagiarism in the sense that you didn't produce it yourself and passed it off as your own. So in an academic context, for grading your understanding, it becomes meaningless to grade something written by a machine instead of by you. In other contexts, it's totally fine.
gnicholas 4y ago
> Separately, burstiness compares the variations of sentences. Humans tend to write with greater burstiness, for example, with some longer or complex sentences alongside shorter ones. AI sentences tend to be more uniform. Although I thought it would be trivial to work around this heuristic by specifying "use some short sentences and some long sentences", that didn't really work for me. When I said "use only short sentences", the average sentence length was a bit shorter. But I was surprised that this type of meta instruction didn't work better (at least in this version).
- astrange 4y ago
  I tried defeating GPT detectors recently by prompting it something like "write with high perplexity" (https://towardsdatascience.com/perplexity-in-language-models-87a196019a94 https://towardsdatascience.com/perplexity-in-language-models...) and although it worked, I also couldn't get it to stop literally using the word "perplexity" in the output.
  - jmugan 4y ago
    That is hilarious.
  - Psychoshy_bc1q 4y ago
    use 'don't mention "perplexety"'
btheshoe 4y ago
The problem isn't chatgpt, it's curriculum. If students were motivated to learn to write (and the skill is genuinely useful, this shouldn't be a hard task) this wouldn't be a problem. But instead for whatever reason we've adopted a coercion based system of education where we assume students innately dislike learning. Personally I think it's a good thing that chatgpt is allowing students to ignore the coercive education system.
zone411 4y ago
None of this will work if you run the generated text through a smaller model designed to rewrite it. I'm betting on schools using expanded remote proctoring software to monitor and record students' homework process and flag potential irregularities for teachers to check. Whether it's positive or negative for students' education, it will be less disruptive to schools than eliminating homework or significantly altering the curriculum.
raylad 4y ago
The Hugging Face GPT-2 detector (https://openai-openai-detector.hf.space/ https://openai-openai-detector.hf.space/) still does a good job identifying generic chatGPT output. But you can fool it by first asking chatGPT if it can write in the style of some known author, letting it output an example, then asking it to write about a given topic in the same style. Here's generic output to "write a one paragraph description of Genghis Khan". The Hugging Face output detector says it's 99.98% likely to be fake (from GPT): > Genghis Khan was a Mongolian warrior and ruler who founded the Mongol Empire, one of the largest empires in history. He is credited with uniting the nomadic Mongol tribes and establishing a powerful and efficient military force that conquered much of Asia and Europe in the 13th century. Genghis Khan is known for his strict code of law and his innovative use of cavalry and military strategy. He is also remembered for his brutal treatment of conquered peoples, as he and his armies were responsible for the deaths of millions of people during their conquests. But this response, in the style of Charles Bukowski, gets a 98.66% real (not GPT) rating by the checker: > Genghis Khan was a man of fire and fury, a conqueror who swept across the land like a storm. His armies were vast and unstoppable, crushing all who opposed him beneath their hooves. He was a man of vision and determination, who united the scattered tribes of the steppes and forged them into a mighty nation. But he was also a man of blood and death, who left a trail of destruction in his wake. Villages were burned and pillaged, their inhabitants slaughtered or taken as slaves. The world trembled at the mention of his name, and even the greatest kings and emperors bowed before him in fear. Such was the power of Genghis Khan, the greatest warrior the world had ever known.
- m463 4y ago
  You might have helped students cheat... But wow, the second response was pretty exciting. I wonder if chatgpt can get present-day authors to up their game?
  - raylad 4y ago
    If I can figure it out in a couple of minutes, everyone else would figure it out too. By the way, here's the response to "can you write in the style of Charles Bukowski?" (the first prompt in the session): > Sure! Here's a little something I came up with: "I woke up this morning with a hangover and a broken heart. The bottle was empty, just like my bank account. Love had come and gone, leaving nothing but pain and regret in its wake. I stumbled out of bed and into the shower, hoping the water would wash away the memories of her. But as I stood there, staring at the peeling paint and the mildew on the walls, I knew that the only thing that would cure me was time and more whisky. So I dressed myself in my finest rags and headed out into the city, searching for the next fix, the next distraction, the next moment of fleeting pleasure that would take me away from the pain. Because in the end, that's all we really have: the present moment and the hope that tomorrow will be better, or at least no worse than today."
s-xyz 4y ago
What strikes me is that even OpenAI has been working on a way to "watermark" GPT-generated text with an "unnoticeable secret signal" to identify its source.
skytrue 4y ago
Disclaimer: I'm working on an app to solve the impending wave of generative content, so I'm somewhat biased. As I'm sure many of us did, I tested out the app here on GPT-3 output. Unmodified, it detected it was GPT-3. Great! However, I added about 10 additional words to the output provided by GPT-3, and it shot up my "human" score by like 60 points, and determined it was human-generated. This is going to be the problem underscoring _any_ model that is trying to identify "AI" generated text. A human can modify it slightly, or subtract words, and it throws the entire thing off. There are other paths that we need to explore to this problem.
- cloudking 4y ago
  What specific problem(s) are you trying to solve?
- godelski 4y ago
  As a generative modeling researcher, thank you for the new adversarial training methods. In all seriousness though, this is really a cat and mouse game. Working in CV I might be biased to think that images are going to be easier to detect than text, but we are going to get better. I'm not saying we shouldn't create detectors (we should), but I think we also need to be aware that this is a cat and mouse game and we need to have true social conversations about this. Though we still fail to have these conversations when it comes to computer security so maybe we're doomed (or maybe this will be the catalyst). It's also important to note that a lot of damage can be done even with images and text that are easy to detect as fake. I've seen plenty of fake Twitter and Linkedin accounts that use StyleGAN profile pictures (with all the telltale signs). I'm curious what kind of model you're using? How interpretable is it?
- AYBABTME 4y ago
  If we go back to the fear of AIs supplanting humans, Accelerando style, and the desire to keep humans relevant. Then assume that we accept the premise that humans-augmented-by-AI work is an acceptable outcome for that future, in an attempt to keep humans relevant. Then is it a problem if a human slightly modified an AI's work? Isn't that the desirable outcome? And if the work itself is so useless that it would be worthy of a zero grade, then perhaps another way of measuring usefulness should be used. In a way, I feel like we can't impose old-world grading techniques to new-world content synthesis technologies.
WalterBright 4y ago
I avoided any and all classes that required writing papers and essays. I was able to satisfy the university liberal arts requirement by taking foreign language and accounting. I have no regrets about that. I wanted to learn math, science, and engineering. I didn't want a degree, I wanted to learn the stuff. What was the point of cheating? I've derived a lifetime of pleasure from knowing that stuff. I don't understand wanting a liberal arts degree and then cheating to get it.
- zmgsabst 4y ago
  A credential for a better job.
  - WalterBright 4y ago
    Then you're facing a career of faking it because you didn't learn the job skills. Also looking forward to complaining a lot about job screening tests, and having your job outsourced. Employers know about the cheaters, fakers, frauds, and charlatans. The job screening tests are an attempt to screen them out. Nobody wants to hire cheaters.
    - zmgsabst 4y ago
      Okay? But the expected value of that choice is positive (for the person making it), as evidenced by the people making it routinely.
      - WalterBright 4y ago
        The expected value may not turn out to be the positive one imagines it would be. See what I wrote about employers trying to filter them out.
        zmgsabst 4y ago
        Right — and yet, there’s a positive expected value. I get you don’t like that, but that’s the reality of US labor market, where useless credentialism has crept in: Many people get degrees they don’t want for jobs that don’t need them to check a box and get hired - because as a society, we’ve built that into our hiring. Particularly liberal arts degrees.
DesiLurker 4y ago
and then makes an AI to fool that AI .. thus the AI cold-wars began!
- slater- 4y ago
  i love this idea.
xyproto 4y ago
Great! These detectors can be used to improve systems like ChatGPT to the point where the output is undetectable again.
jameslk 4y ago
> The college senior isn't alone in the race to rein in AI plagiarism and forgery. OpenAI, the developer of ChatGPT, has signaled a commitment to preventing AI plagiarism and other nefarious applications. Last month, Scott Aaronson, a researcher currently focusing on AI safety at OpenAI, revealed that the company has been working on a way to "watermark" GPT-generated text with an "unnoticeable secret signal" to identify its source. Ah, so magnanimous of them to sell both the plague and the cure. I say that tongue-in-cheek of course, but it seems that's where this is going, whether they make and sell both or let others do it. That is, both sides of the production and detection of this spam are about to profit handsomely from the overwhelming piles of it we're about to be drowning in.
jtxt 4y ago
OpenAI stores all their input and output right? They could make a plagiarism detector type site to test if a text came from them.
luisapic 4y ago
[flagged]

103 points by greenie_beans 4y ago

Yes, but can it prove it? The old fashioned plagiarism problem was solved by finding the original it was copied from. With AI, the falsely attributed work IS original. There in lies…

throwmeup123 4y ago
It doesn't and it tags nearly everything not written by an 8th grader as written by AI... even this exact article.
- informalo 4y ago
  Well maybe the article was written using ChatGPT.
- henryteeare 4y ago
  Really? Perhaps they updated it, because I got "likely human generated" for the parent article. "Your GPTZero score corresponds to the likelihood of the text being AI generated: 338.59305536714294 Your text is likely human generated!"
paul_f 4y ago
AI-generated content can still be traced back to its source. Even though the content may be original, AI-generated works can be identified by their algorithmic patterns, which can be used to determine the source of the work. ChatGPT wrote that, not me. How do we know we aren't simply reading ChatGPT arguing with itself?
- godelski 4y ago
  > It is certainly possible for AI models to generate text that appears to be self-referential or self-reflexive. However, it is also possible to trace the source of AI-generated text by examining the specific patterns and characteristics of the text. For example, you can analyze the language and style used in the text, as well as the context in which it was generated, to determine whether it was likely produced by a specific AI model or not. In addition, you can also examine the metadata associated with the text, such as the date and time it was created, to help identify its source. ChatGPT wrote that. But I'll say that this is a lot easier said than done. This will always be an increasingly difficult cat and mouse game.

endisneigh 4y ago

Without knowing the false positive and false negative rate this article is meaningless IMHO. Furthermore, even if it had a 0% false negative and false positive rate, it then would be used to train the next iteration of language models. A good idea superficially, but ultimately solves the wrong problem.

deleted 4y ago
[deleted]

ttul 4y ago

My son and his friends are already doing their homework using ChatGPT. Teachers will have to get used to this and hopefully embrace it.

TSiege 4y ago
No judgement here, but I'm curious how they're using it and what your feelings are. It seems like you've embraced it
- ttul 4y ago
  Literally everything. They’re using it to help with math homework. Putting together copy for a slide presentation about the history of Pepsico (why? I have no idea). Writing a silly story about an ape man who dominates the world through vicious conquests. “Now write a sequel” apparently works really well.
jolmg 4y ago
In the end, (ideally) it's the students that are losing out on education, though. That's unless there's a competition dynamic among students. Maybe embracing it is removing competition among students. EDIT: On second thought, one can just put more emphasis on exams taken in-premises with supervision.
- Der_Einzige 4y ago
  School is not about getting an "education", this is pedegological propaganda. It's about maximizing your college admissions chances, which then leads to better jobs. Id rather be stupid and rich than "educated" and poor.
  - skor 4y ago
    would you rather be stupid? think of all the things you would have wished to know before you die. seems like a pipe dream to me.

e1g 4y ago

Since TFA mentioned ChatGPT by name, it’s worth noting that OpenAI itself is developing software to detect text generated by ChatGPT https://www.theregister.com/2023/01/09/in_brief_ai/ https://www.theregister.com/2023/01/09/in_brief_ai/

gnicholas 4y ago
Ah, there's the true business model. Students can use ChatGPT for free, but schools get charged to be able to detect it. Brilliant! Similarly, OpenAI's tools can be used to create all sorts of content marketing and other webspam. They they'll create a monetizable search engine-killer that knows how to ignore all this fluff (which will probably constitute 99% of the internet within a few years).
- cloudking 4y ago
  GPT-3 content marketing tools already exist https://jasper.ai https://jasper.ai
  - gnicholas 4y ago
    Oh I know — that's why I assume the web will be choking on AI-generated content marketing in the foreseeable future. The killer app will be the search/information/answer engine that can suitably ignore the AI-generated tripe. I wonder if we'll look back one day and say "remember back before 2020, when the internet was filled with junk, but at least it was human-written junk?".

Der_Einzige 4y ago

No reliable technique exists for AI LM text detection assuming that the LM can be fine tuned and that the decoder settings are available. I doubt a reliable technique for this is even ever possible.

teeray 4y ago

I can’t see how this will be useful unless the false positive rate is 0%. Can you imagine how demoralizing it would be to legitimately write a term paper and have it flagged as being AI-written? How do you disprove it?

dvngnt_ 4y ago
that literally just happened on reddit r/art

Al-Khwarizmi 4y ago

Some people are putting a lot of hope in AI text detectors to prevent ChatGPT usage in contexts like student essays, etc. While these detectors are of course interesting from the hacker/academic points of view, I don't think they will really solve much in practice. On the one hand, our (teachers') adversary is not ChatGPT. It's ChatGPT + a human, which will be aware of these techniques and can tweak the text to avoid detection. On the other hand, even if we could detect generated texts with high accuracy, are we going to fail a student just because a black-box AI model says so? Doesn't seem very ethical to me. It's funny how some people worried about ChatGPT's ethical implications seem to be OK with this, just because it goes against their personal demon. I don't think there is any realistic alternative to just assuming that these systems will be used wherever text is required and you can't physically watch the writer, and living with it.

sarchertech 4y ago
I took an accounting class and it required you to come in on Saturday for really long tests. My guess is that students just have to spend 3-5 hours on a Saturday writing essays, doing programming assignments etc…
drbeast 4y ago
I went through college using Wikipedia to not only make my outline but also provide citations. I'd then rewrite the articles in my own word. This greatly reduced the amount of preparation needed prior to writing. No shame, the majority of research papers done in unis are silly and regurgitation. Really technology is just getting good at eliminating busy work which will be the death kneel of liberal arts. Double rant: college is largely a waste of time for liberal arts classes.
- brokensegue 4y ago
  >death kneel of liberal arts
  - drbeast 4y ago
    Ah yes, one of those jack offs who disregard arguments because of a phone posting auto correct. Do us a favor and get the COVID vaccines. They're safe and effective!

lr4444lr 4y ago

Just my opinion, but the kinds of writing assignments ChatGPT excels at seem to me the least useful ones for assessing student understanding. Why are we making kids write entire persuasive essays and research papers when in the real world these are almost always the result of collaborative effort?

namelessoracle 4y ago
How are they supposed to colloborate of these without doing some of these on their own?
asdff 4y ago
Because writing is an important skill in life.

marcell 4y ago

ChatGPT is similar to a calculator. We don't let 3rd graders use calculators on their arithmetic homework, even though there's no way to detect if they used one. The point of a 3rd grader adding fractions is not because we care about the result of 3/4 + 1/3, but because the 3rd grader should learn how to add fractions. If you think of ChatGPT as a fancy calculator, you can just apply the same techniques we've been using for a while: tell students not to use them on homework, make homework only a small % of the grade, and rely on in-class, paper only exams for most the grade.

namaria 4y ago
I think text generators will do two things: reduce the average quality of published text, and reduce the average text writing skills. It thus increases the value of writing skills. I count that as a win.
seanmcdirmid 4y ago
So like...writing an essay in class. That could be useful, although they won't be able to do any research or anything.
- janalsncm 4y ago
  Students could print out their sources and turn them in with the essay. Of course this would require knowing ahead of time what sources you’d need.
- stephencanon 4y ago
  The best class I had in high school (Mr. Hecker’s English class), required writing an essay in class every week. All my college essays were easy by comparison.
  - seanmcdirmid 4y ago
    To each their own. I didn’t really learn how to write until in grad school, when I had things I really wanted to write about. I can imagine how much joy would have been sucked out of writing if I just had to write some essay every week that I didn’t have much interest in.

poulsbohemian 4y ago

I find the texts written by AI systems so clunky and superficial that I would be disappointed if they aren't immediately obvious to an educator.

Psychoshy_bc1q 4y ago
The efficacy of ChatGPT is contingent upon one's utilization of its capabilities. Its ability to generate text of exceptional quality is undeniable.

knaik94 4y ago

I don't think cheaters will use any text from ChatGPT verbatim. The majority of "cheating" is going to be similar to how students use Wikipedia as a reference. This might be useful to filter spam from places that initially got bombarded, like stack overflow. I agree with other comments, it's hard to judge the value of this tool without false positive/negative stats. The output isn't completely deterministic though so I wonder how "certain" this tool can be.

a_bonobo 4y ago
You'd be surprised how many students turn in Wikipedia-articles run through software that replaces words with (often wrong) synonyms. I've tutored students who paid such find-and-replace services. Zero effort for the majority of students that cheat, very easy to detect, rarely punished (a zero on an essay here or there, no expulsion - students are customers after all). There will be paid ChatGPT services for cheating students, the ideal solution would be to rethink and rehaul how student performance is evaluated, but universities will instead pay something like Turnitin because throwing money at a problem is always easier than thinking.
- analog31 4y ago
  I taught college for a brief time. Catching cheaters is hard work, and unpaid, and it's perceived that doing it too much affects your chances of getting your contract renewed. "Let the employers or the next course level sort them out" seems to be a more pragmatic strategy.
  - kspacewalk2 4y ago
    Seeig a fellow TA pursue a cheater who copied a major math assignment symbol for symbol from another student, mistakes and all, and fail to break through the bureaucracy and achieve actual punishment was quite an eye opening experience. If this is how brazen cheaters are handled in one of the top Canadian engineering schools, what hope is there?
    - 4y ago

cloudking 4y ago

It's a cool idea, unfortunately it doesn't work very well. I ran a few generated examples through it and they came back as human.

Wowfunhappy 4y ago

Regardless of whether students should be using ChatGPT, I strongly disagree that a ChatGPT-written essay is "plagiarism". ChatGPT is not an author, it's a computer program. We don't expect bibliographies to include Microsoft Word, Grammerly, or iOS Keyboard Predictive Text in the list of references. If I use a piece of software to create an essay, the essay that's created belongs to me.

elefanten 4y ago
I guess it depends if you consider the act of writing to be something that can only be done by a human. If a program can produce a work that passes as "writing", then that program could be called the "author" of the work. So a student that takes said output and hands it in is plagiarizing by claiming credit for something written by a different "author". It'll probably take some time for language to evolve around non-human agents. There's no intrinsic reason to prefer separating (or combining) that terminology by humanness of agent, afaik. At least, for common usage. One will win and only time will tell.
kuhewa 4y ago
Plagarism is passing off others' ideas or words off as your own without attribution. IMO there is an argument to be made whether conjuring up text from a model of others' words should qualify as "others' ideas or words", but if it isn't technically academic plagarism I reckon it is a distinction without a difference. The concern with academic plagarism isn't primarily IP, so I reckon it doesn't inherently matter whether "others" being borrowed from is a particular author. Also, There's a commonly understood threshold for plagiarism where a single word won't qualify and a phrase probably won't qualify, but a whole paragraph does. That is why Grammarly isn't plagarism, not because it is software.
andai 4y ago
It's not plagiarism in the sense that you took stole else's work, it's plagiarism in the sense that you didn't produce it yourself and passed it off as your own. So in an academic context, for grading your understanding, it becomes meaningless to grade something written by a machine instead of by you. In other contexts, it's totally fine.

gnicholas 4y ago

> Separately, burstiness compares the variations of sentences. Humans tend to write with greater burstiness, for example, with some longer or complex sentences alongside shorter ones. AI sentences tend to be more uniform. Although I thought it would be trivial to work around this heuristic by specifying "use some short sentences and some long sentences", that didn't really work for me. When I said "use only short sentences", the average sentence length was a bit shorter. But I was surprised that this type of meta instruction didn't work better (at least in this version).

astrange 4y ago
I tried defeating GPT detectors recently by prompting it something like "write with high perplexity" (https://towardsdatascience.com/perplexity-in-language-models-87a196019a94 https://towardsdatascience.com/perplexity-in-language-models...) and although it worked, I also couldn't get it to stop literally using the word "perplexity" in the output.
- jmugan 4y ago
  That is hilarious.
- Psychoshy_bc1q 4y ago
  use 'don't mention "perplexety"'

btheshoe 4y ago

The problem isn't chatgpt, it's curriculum. If students were motivated to learn to write (and the skill is genuinely useful, this shouldn't be a hard task) this wouldn't be a problem. But instead for whatever reason we've adopted a coercion based system of education where we assume students innately dislike learning. Personally I think it's a good thing that chatgpt is allowing students to ignore the coercive education system.

zone411 4y ago

None of this will work if you run the generated text through a smaller model designed to rewrite it. I'm betting on schools using expanded remote proctoring software to monitor and record students' homework process and flag potential irregularities for teachers to check. Whether it's positive or negative for students' education, it will be less disruptive to schools than eliminating homework or significantly altering the curriculum.

raylad 4y ago

The Hugging Face GPT-2 detector (https://openai-openai-detector.hf.space/ https://openai-openai-detector.hf.space/) still does a good job identifying generic chatGPT output. But you can fool it by first asking chatGPT if it can write in the style of some known author, letting it output an example, then asking it to write about a given topic in the same style. Here's generic output to "write a one paragraph description of Genghis Khan". The Hugging Face output detector says it's 99.98% likely to be fake (from GPT): > Genghis Khan was a Mongolian warrior and ruler who founded the Mongol Empire, one of the largest empires in history. He is credited with uniting the nomadic Mongol tribes and establishing a powerful and efficient military force that conquered much of Asia and Europe in the 13th century. Genghis Khan is known for his strict code of law and his innovative use of cavalry and military strategy. He is also remembered for his brutal treatment of conquered peoples, as he and his armies were responsible for the deaths of millions of people during their conquests. But this response, in the style of Charles Bukowski, gets a 98.66% real (not GPT) rating by the checker: > Genghis Khan was a man of fire and fury, a conqueror who swept across the land like a storm. His armies were vast and unstoppable, crushing all who opposed him beneath their hooves. He was a man of vision and determination, who united the scattered tribes of the steppes and forged them into a mighty nation. But he was also a man of blood and death, who left a trail of destruction in his wake. Villages were burned and pillaged, their inhabitants slaughtered or taken as slaves. The world trembled at the mention of his name, and even the greatest kings and emperors bowed before him in fear. Such was the power of Genghis Khan, the greatest warrior the world had ever known.

m463 4y ago
You might have helped students cheat... But wow, the second response was pretty exciting. I wonder if chatgpt can get present-day authors to up their game?
- raylad 4y ago
  If I can figure it out in a couple of minutes, everyone else would figure it out too. By the way, here's the response to "can you write in the style of Charles Bukowski?" (the first prompt in the session): > Sure! Here's a little something I came up with: "I woke up this morning with a hangover and a broken heart. The bottle was empty, just like my bank account. Love had come and gone, leaving nothing but pain and regret in its wake. I stumbled out of bed and into the shower, hoping the water would wash away the memories of her. But as I stood there, staring at the peeling paint and the mildew on the walls, I knew that the only thing that would cure me was time and more whisky. So I dressed myself in my finest rags and headed out into the city, searching for the next fix, the next distraction, the next moment of fleeting pleasure that would take me away from the pain. Because in the end, that's all we really have: the present moment and the hope that tomorrow will be better, or at least no worse than today."

s-xyz 4y ago

What strikes me is that even OpenAI has been working on a way to "watermark" GPT-generated text with an "unnoticeable secret signal" to identify its source.

skytrue 4y ago

Disclaimer: I'm working on an app to solve the impending wave of generative content, so I'm somewhat biased. As I'm sure many of us did, I tested out the app here on GPT-3 output. Unmodified, it detected it was GPT-3. Great! However, I added about 10 additional words to the output provided by GPT-3, and it shot up my "human" score by like 60 points, and determined it was human-generated. This is going to be the problem underscoring _any_ model that is trying to identify "AI" generated text. A human can modify it slightly, or subtract words, and it throws the entire thing off. There are other paths that we need to explore to this problem.

cloudking 4y ago
What specific problem(s) are you trying to solve?
godelski 4y ago
As a generative modeling researcher, thank you for the new adversarial training methods. In all seriousness though, this is really a cat and mouse game. Working in CV I might be biased to think that images are going to be easier to detect than text, but we are going to get better. I'm not saying we shouldn't create detectors (we should), but I think we also need to be aware that this is a cat and mouse game and we need to have true social conversations about this. Though we still fail to have these conversations when it comes to computer security so maybe we're doomed (or maybe this will be the catalyst). It's also important to note that a lot of damage can be done even with images and text that are easy to detect as fake. I've seen plenty of fake Twitter and Linkedin accounts that use StyleGAN profile pictures (with all the telltale signs). I'm curious what kind of model you're using? How interpretable is it?
AYBABTME 4y ago
If we go back to the fear of AIs supplanting humans, Accelerando style, and the desire to keep humans relevant. Then assume that we accept the premise that humans-augmented-by-AI work is an acceptable outcome for that future, in an attempt to keep humans relevant. Then is it a problem if a human slightly modified an AI's work? Isn't that the desirable outcome? And if the work itself is so useless that it would be worthy of a zero grade, then perhaps another way of measuring usefulness should be used. In a way, I feel like we can't impose old-world grading techniques to new-world content synthesis technologies.

WalterBright 4y ago

I avoided any and all classes that required writing papers and essays. I was able to satisfy the university liberal arts requirement by taking foreign language and accounting. I have no regrets about that. I wanted to learn math, science, and engineering. I didn't want a degree, I wanted to learn the stuff. What was the point of cheating? I've derived a lifetime of pleasure from knowing that stuff. I don't understand wanting a liberal arts degree and then cheating to get it.

zmgsabst 4y ago
A credential for a better job.
- WalterBright 4y ago
  Then you're facing a career of faking it because you didn't learn the job skills. Also looking forward to complaining a lot about job screening tests, and having your job outsourced. Employers know about the cheaters, fakers, frauds, and charlatans. The job screening tests are an attempt to screen them out. Nobody wants to hire cheaters.
  - zmgsabst 4y ago
    Okay? But the expected value of that choice is positive (for the person making it), as evidenced by the people making it routinely.
    - WalterBright 4y ago
      The expected value may not turn out to be the positive one imagines it would be. See what I wrote about employers trying to filter them out.
      - zmgsabst 4y ago
        Right — and yet, there’s a positive expected value. I get you don’t like that, but that’s the reality of US labor market, where useless credentialism has crept in: Many people get degrees they don’t want for jobs that don’t need them to check a box and get hired - because as a society, we’ve built that into our hiring. Particularly liberal arts degrees.

DesiLurker 4y ago

and then makes an AI to fool that AI .. thus the AI cold-wars began!

slater- 4y ago
i love this idea.

xyproto 4y ago

Great! These detectors can be used to improve systems like ChatGPT to the point where the output is undetectable again.

jameslk 4y ago

> The college senior isn't alone in the race to rein in AI plagiarism and forgery. OpenAI, the developer of ChatGPT, has signaled a commitment to preventing AI plagiarism and other nefarious applications. Last month, Scott Aaronson, a researcher currently focusing on AI safety at OpenAI, revealed that the company has been working on a way to "watermark" GPT-generated text with an "unnoticeable secret signal" to identify its source. Ah, so magnanimous of them to sell both the plague and the cure. I say that tongue-in-cheek of course, but it seems that's where this is going, whether they make and sell both or let others do it. That is, both sides of the production and detection of this spam are about to profit handsomely from the overwhelming piles of it we're about to be drowning in.

jtxt 4y ago

OpenAI stores all their input and output right? They could make a plagiarism detector type site to test if a text came from them.

luisapic 4y ago

[flagged]