4 ms·

It'll be interesting to see how colleges and universities react to GPT-3. Students will surely use it to write entire assignments.

grogenaut 6y ago
Have you read much gpt3 stuff? While it's coherent in a sentence it is very rambling over paragraphs to pages. It could probably do fine for a grade school or bad highschool paper. I think if you turned it in for college you'd get an f. On an unrelated note my fake daughter is now a TA and the professor lead off saying "we are in a golden age of cheating". They're going for way more short assignments as it's a lot more work to cheat on those than one make or break test.
- SubiculumCode 6y ago
  Have you read college freshman essays? While it's coherent in a sentence it is very rambling over paragraphs to pages.
  - grogenaut 6y ago
    yes I've read them, would those pass an English composition class in college? This comment generated by Gpt3
    - whimsicalism 6y ago
      I think GPT essays could definitely pass a freshman expository writing class. I went to a pretty good university and when we did peer review I was pretty surprised at (what I considered) the low average quality of the writing.
      - bluetwo 6y ago
        Examples?
        whimsicalism 6y ago
        Looking back, I think freshman me was perhaps a bit harsh in my assessment of my peers. Here are two excerpts, one my own writing and one from a peer. Rereading them, I am not sure GPT3 could recreate either of these, but you can judge: Peer: > The Gaza Conflict Gave Hamas what they needed to build an even deeper anti US narrative and anti-israeli narrative. The reasons that Israel was able to act the way it did so during these conflicts were:The civil war/Russian invasion of Ukraine, especially after the July 17 downing of Malaysian Airlines Flight 17; President Assad’s brutal tactics in the Syrian civil war, which seemed to be paying off at the time; and advance of the Islamic State in northwest Iraq and the horrific videos of their executions. As a way to show 0 tolerance for Islamic Radicalism President Obama and his administration gave Israel full support at the start of the Gaza operation. Me: > Eleven years later, peace in Israel and Palestine seems no closer. Despite being ousted by Fatah from the West Bank, continually targeted by economic sanctions, and subject to military action, Hamas seems no closer to disappearing than when it was first founded (Milton-Edwards 212) and continues to be a major political force, particularly in the Gaza strip. What has changed, however, is that Hamas has grown desperate. In “A rare opportunity to influence Hamas,” Daniel Nisman argues that the increased isolation experienced by Hamas due to the international community's pressure is in fact counterproductive.
        nl 6y ago
        Here's some samples using the small GPT-2 (emphasis: small GPT-2!) in AiTextGen, seeded using "The Gaza Conflict Gave Hamas what they needed " (1st 3 samples, not cherry picked): > The Gaza Conflict Gave Hamas what they needed to survive and that's not something Israel can afford. > The Gaza Conflict Gave Hamas what they needed to take control of Gaza. In the past few days I have had a few conversations with people who have been in Gaza who are very critical of the IDF. These people are often very, very angry. They are also very angry about the way the IDF is doing what it is supposed to be doing and what it is supposed to be doing. So in my mind, the issue of Hamas being in control of Gaza is not something that should go away. It is something that needs to be addressed. Q: What have you been hearing about since the war started? A: The IDF has been saying that it is going to allow Hamas to take over Gaza. Q: What has been particularly troubling to you lately? A: I have been hearing about this as well. Q: What has been more troubling to you recently? A: My friends have been saying that they want to call it Operation Protective Edge. Q: And, you know, what do they mean by that? A: They mean that they want to use it to make sure that they have enough force to take control of Gaza. Q: And that means that they want to make sure that they have enough > The Gaza Conflict Gave Hamas what they needed?" The UN is now considering whether to send military advisers to Gaza to assist the country's military forces. The Security Council is considering whether to send the equipment, along with the training, to a military operation in the region. The US is also considering sending a team of US special forces to assist the Palestinian armed forces in the conflict. The United Nations is now considering whether to send military advisers to Gaza to assist the country's military forces. Kerry's comments come as the US has been in touch with the Palestinians to offer support in exchange for a full ceasefire, and as the US continues to support the PA and Hamas, the two groups have been engaged in a long-running conflict with Israel in the Gaza Strip. In January, Kerry condemned Israel's "continued offensive against Gaza," saying the blockade was the "worst violation of international law on the part of the Israeli government and the civilian population of Gaza." The US is now considering whether to send military advisers to Gaza to assist the country's military forces. According to Reuters, the US Secretary of State John Kerry said this week that "there is no guarantee" that the US will send special forces "to the Gaza Strip So yeah - not fantastic, but interestingly not terrible either. The non-factual but coherent nature of it is very troubling.
    - gwern 6y ago
      Well... https://www.eduref.net/features/what-grades-can-ai-get-in-college/ https://www.eduref.net/features/what-grades-can-ai-get-in-co... https://arxiv.org/abs/2009.03300 https://arxiv.org/abs/2009.03300
rjzzleep 6y ago
Even long before GPT-3 a friend of mine did his thesis with generated text, in an engineering university and received a B. This is 6 years ago. I have my own beefs with thesis' in general already, since 2 thirds of it seems to be filled with redundant text to prove that you went to university. I guess it's a little bit different, since back then he had to actually work to generate it and now it's a lot easier
- whimsicalism 6y ago
  > his thesis with generated text 6 years ago = probably LSTM. He wrote an entire thesis with this and got a B? That seems implausible to me, but maybe I'm used to higher grading standards. Did he just use it to fill in parts of it? Also, the plural of thesis is theses, not thesis' which implies the possessive.
- kwhitefoot 6y ago
  How does that work? Don't you have to defend a thesis?
b0rsuk 6y ago
Corporate speeches, sermons, motivational talks, poetry, and political speeches. They are either not required to make sense or no one dares to interrupt.
Der_Einzige 6y ago
Anecdotally, but I know a number of people who do university assignments for money. Many of the clients for the folks I know are at the university level are folks with poorer then average English language skills and are usually in intro writing courses. I'd be terrified if I were one of them right now. GPT-3 would be a godsend for cheaters, but still requires a human to jump in and rewrite whole sections. No, if you want to REALLY want to cheat using AI, you should most likely utilize either 1. Abstractive Summarizers (e.g. Pegasus) or 2. paraphrasing tools (e.g. like at https://quillbot.com/ https://quillbot.com/). I believe that Quilbot is primarily powered by MLMs like BERT rather than CLMs like GPT-2 (but someone who works there can enlighten me more). Copy and paste a text that you want rewritten in your own words (e.g. the ideas of a really smart individual), and then it rewrites it using totally different language but preserving the same meaning. (old) Plagerism detection tools don't work and hell, it's not hard to fool the never ones. You can try tools for detecting if something is AI written by a particular model and weights (e.g. to prove if they used GPT2-Medium), but if I fine-tuned those same weights, than proving it was plagiarism will become exceedingly difficult. Welcome to the brave new world of cheating. Also, techniques like this are coming to a CS department near you (in the form of source code generation powered by NLP models).
- charcircuit 6y ago
  GPT-3 is just as much of a cheat as using a thesaurus. New writing tools shouldn't be banned just because old people didn't have those tools.
- vsareto 6y ago
  I feel like we're overestimating the value of generating text and selectively picking out things that sound good. This is up for debate, but you'd probably do better to read actual authors and pick things up from them and maybe making your own changes. The time spent parsing the output to see if it's good could be spent on thinking about something to write? Would be neat to try and publish books with a percentage of AI generated text and see how well they do. Maybe there's a sweet spot for productivity.
vmception 6y ago
Wolfram Alpha has been solving calculus problems for 12 years and it is barely a footnote in how the college experienced has changed So I would say this likely will just be there. It just is. Wont change anything, universities will acknowledge it, a headline or two will occur when its use was discovered in a paper that a student didnt even skim to make less obvious, and most papers will fly under the radar. Other kinds of assessments will still do their job.
pedalpete 6y ago
A friend started a AI to improve writing (https://outwrite.com https://outwrite.com) and when the initially started, they had a detect plagiarism feature that teachers could use, I think they stopped developing that eventually. If I recall correctly, the way it worked was to build up a model of this persons writing, and how it compared to to other people, and then would measure the likelihood that sentences and paragraphs matched the rest of the writing. I suspect something similar could be done with GPT-x

grogenaut 6y ago

Have you read much gpt3 stuff? While it's coherent in a sentence it is very rambling over paragraphs to pages. It could probably do fine for a grade school or bad highschool paper. I think if you turned it in for college you'd get an f. On an unrelated note my fake daughter is now a TA and the professor lead off saying "we are in a golden age of cheating". They're going for way more short assignments as it's a lot more work to cheat on those than one make or break test.

SubiculumCode 6y ago
Have you read college freshman essays? While it's coherent in a sentence it is very rambling over paragraphs to pages.
- grogenaut 6y ago
  yes I've read them, would those pass an English composition class in college? This comment generated by Gpt3
  - whimsicalism 6y ago
    I think GPT essays could definitely pass a freshman expository writing class. I went to a pretty good university and when we did peer review I was pretty surprised at (what I considered) the low average quality of the writing.
    - bluetwo 6y ago
      Examples?
      - whimsicalism 6y ago
        Looking back, I think freshman me was perhaps a bit harsh in my assessment of my peers. Here are two excerpts, one my own writing and one from a peer. Rereading them, I am not sure GPT3 could recreate either of these, but you can judge: Peer: > The Gaza Conflict Gave Hamas what they needed to build an even deeper anti US narrative and anti-israeli narrative. The reasons that Israel was able to act the way it did so during these conflicts were:The civil war/Russian invasion of Ukraine, especially after the July 17 downing of Malaysian Airlines Flight 17; President Assad’s brutal tactics in the Syrian civil war, which seemed to be paying off at the time; and advance of the Islamic State in northwest Iraq and the horrific videos of their executions. As a way to show 0 tolerance for Islamic Radicalism President Obama and his administration gave Israel full support at the start of the Gaza operation. Me: > Eleven years later, peace in Israel and Palestine seems no closer. Despite being ousted by Fatah from the West Bank, continually targeted by economic sanctions, and subject to military action, Hamas seems no closer to disappearing than when it was first founded (Milton-Edwards 212) and continues to be a major political force, particularly in the Gaza strip. What has changed, however, is that Hamas has grown desperate. In “A rare opportunity to influence Hamas,” Daniel Nisman argues that the increased isolation experienced by Hamas due to the international community's pressure is in fact counterproductive.
        nl 6y ago
        Here's some samples using the small GPT-2 (emphasis: small GPT-2!) in AiTextGen, seeded using "The Gaza Conflict Gave Hamas what they needed " (1st 3 samples, not cherry picked): > The Gaza Conflict Gave Hamas what they needed to survive and that's not something Israel can afford. > The Gaza Conflict Gave Hamas what they needed to take control of Gaza. In the past few days I have had a few conversations with people who have been in Gaza who are very critical of the IDF. These people are often very, very angry. They are also very angry about the way the IDF is doing what it is supposed to be doing and what it is supposed to be doing. So in my mind, the issue of Hamas being in control of Gaza is not something that should go away. It is something that needs to be addressed. Q: What have you been hearing about since the war started? A: The IDF has been saying that it is going to allow Hamas to take over Gaza. Q: What has been particularly troubling to you lately? A: I have been hearing about this as well. Q: What has been more troubling to you recently? A: My friends have been saying that they want to call it Operation Protective Edge. Q: And, you know, what do they mean by that? A: They mean that they want to use it to make sure that they have enough force to take control of Gaza. Q: And that means that they want to make sure that they have enough > The Gaza Conflict Gave Hamas what they needed?" The UN is now considering whether to send military advisers to Gaza to assist the country's military forces. The Security Council is considering whether to send the equipment, along with the training, to a military operation in the region. The US is also considering sending a team of US special forces to assist the Palestinian armed forces in the conflict. The United Nations is now considering whether to send military advisers to Gaza to assist the country's military forces. Kerry's comments come as the US has been in touch with the Palestinians to offer support in exchange for a full ceasefire, and as the US continues to support the PA and Hamas, the two groups have been engaged in a long-running conflict with Israel in the Gaza Strip. In January, Kerry condemned Israel's "continued offensive against Gaza," saying the blockade was the "worst violation of international law on the part of the Israeli government and the civilian population of Gaza." The US is now considering whether to send military advisers to Gaza to assist the country's military forces. According to Reuters, the US Secretary of State John Kerry said this week that "there is no guarantee" that the US will send special forces "to the Gaza Strip So yeah - not fantastic, but interestingly not terrible either. The non-factual but coherent nature of it is very troubling.
  - gwern 6y ago
    Well... https://www.eduref.net/features/what-grades-can-ai-get-in-college/ https://www.eduref.net/features/what-grades-can-ai-get-in-co... https://arxiv.org/abs/2009.03300 https://arxiv.org/abs/2009.03300

rjzzleep 6y ago

Even long before GPT-3 a friend of mine did his thesis with generated text, in an engineering university and received a B. This is 6 years ago. I have my own beefs with thesis' in general already, since 2 thirds of it seems to be filled with redundant text to prove that you went to university. I guess it's a little bit different, since back then he had to actually work to generate it and now it's a lot easier

whimsicalism 6y ago
> his thesis with generated text 6 years ago = probably LSTM. He wrote an entire thesis with this and got a B? That seems implausible to me, but maybe I'm used to higher grading standards. Did he just use it to fill in parts of it? Also, the plural of thesis is theses, not thesis' which implies the possessive.
kwhitefoot 6y ago
How does that work? Don't you have to defend a thesis?

b0rsuk 6y ago

Corporate speeches, sermons, motivational talks, poetry, and political speeches. They are either not required to make sense or no one dares to interrupt.

Der_Einzige 6y ago

Anecdotally, but I know a number of people who do university assignments for money. Many of the clients for the folks I know are at the university level are folks with poorer then average English language skills and are usually in intro writing courses. I'd be terrified if I were one of them right now. GPT-3 would be a godsend for cheaters, but still requires a human to jump in and rewrite whole sections. No, if you want to REALLY want to cheat using AI, you should most likely utilize either 1. Abstractive Summarizers (e.g. Pegasus) or 2. paraphrasing tools (e.g. like at https://quillbot.com/ https://quillbot.com/). I believe that Quilbot is primarily powered by MLMs like BERT rather than CLMs like GPT-2 (but someone who works there can enlighten me more). Copy and paste a text that you want rewritten in your own words (e.g. the ideas of a really smart individual), and then it rewrites it using totally different language but preserving the same meaning. (old) Plagerism detection tools don't work and hell, it's not hard to fool the never ones. You can try tools for detecting if something is AI written by a particular model and weights (e.g. to prove if they used GPT2-Medium), but if I fine-tuned those same weights, than proving it was plagiarism will become exceedingly difficult. Welcome to the brave new world of cheating. Also, techniques like this are coming to a CS department near you (in the form of source code generation powered by NLP models).

charcircuit 6y ago
GPT-3 is just as much of a cheat as using a thesaurus. New writing tools shouldn't be banned just because old people didn't have those tools.
vsareto 6y ago
I feel like we're overestimating the value of generating text and selectively picking out things that sound good. This is up for debate, but you'd probably do better to read actual authors and pick things up from them and maybe making your own changes. The time spent parsing the output to see if it's good could be spent on thinking about something to write? Would be neat to try and publish books with a percentage of AI generated text and see how well they do. Maybe there's a sweet spot for productivity.

vmception 6y ago

Wolfram Alpha has been solving calculus problems for 12 years and it is barely a footnote in how the college experienced has changed So I would say this likely will just be there. It just is. Wont change anything, universities will acknowledge it, a headline or two will occur when its use was discovered in a paper that a student didnt even skim to make less obvious, and most papers will fly under the radar. Other kinds of assessments will still do their job.

pedalpete 6y ago

A friend started a AI to improve writing (https://outwrite.com https://outwrite.com) and when the initially started, they had a detect plagiarism feature that teachers could use, I think they stopped developing that eventually. If I recall correctly, the way it worked was to build up a model of this persons writing, and how it compared to to other people, and then would measure the likelihood that sentences and paragraphs matched the rest of the writing. I suspect something similar could be done with GPT-x