6 ms·

This is anecdotal evidence at best, but it is worth considering. I know of several individuals who were able to complete their entire Master's thesis utilizing

by gzer0 5y ago

This is anecdotal evidence at best, but it is worth considering. I know of several individuals who were able to complete their entire Master's thesis utilizing a combination of AI generated content (GPT-3) and a paraphrasing tool. The generated text was well over 50 pages, completely bypassed all known content/plagiarism checks and was even included in the Universities "exemplary examples". To this day, it is still there. This is of significant concern as some of these GPT-3 based tools are now integrated within MS Word itself. Word 2021 allows for "add-ons", out of which I have noticed several third party content generation and paraphrasing tools.

phkahler 5y ago
Sounds like automated review of automatically generated papers. And people pay money for that...
13415 5y ago
Please include a link to these theses, because as it stands this anecdote sounds extremely implausible. I don't know what university you were, but I've been at a few in Europe and at every one of them Master theses were evaluated from the start to the end by several humans. GPT-3 is unable to produce even two pages of coherent text, let alone 50 pages good enough to be accepted as a Master thesis in any discipline at any university I could think of (even the worst ones). I can imagine that plagiators use paraphrasing software quite extensively, though, and that it is a problem.
- gzer0 5y ago
  Let me clarify: It was not all automated, there was a fair bit of manual intervention needed. I understand your concerns and they are valid and this is why I preface my statement with "anecdotal evidence". What I write is most certainly not the entire story and a fair bit of detail is left out. It should be known that this is widespread across multiple industries and this will only become more of an issue in the future. This is a US-based institution, fully accredited.
throwawaygh 5y ago
> Master's thesis I don't doubt this at all, and I have no doubt that GPT-3 with a bit of human editing can spit out something better than the lower third of masters students at corn row colleges. Masters degrees are cash cows, which is why no one in unregulated industries cares about them. People in regulated/unionized industries also don't actually care; even educators, who at least nominally see intrinsic value in education, go to borderline diploma mills to get that union-mandated raise at minimal effort.
- derefr 5y ago
  How do you feel about doctorates?
  - throwawaygh 5y ago
    Depends. University of Phoenix awards doctorates that take 3-4 years (HUGE red flag -- the best and brightest phd students might get out in 4 years if everything goes perfectly; an "expected time to graduation" of anything less than 5 years is almost certainly a worthless degree). Those doctorates don't require much more than taking some coursework and paying a boatload in tuition. Basically an expensive and length online masters program. Not worth the paper they're printed on, unless you're employed by the government or in a union job that mandates raises for education attainment. As a general rule of thumb, PhDs from R01 universities that are paid for by the university through research assistantships or teaching assistantships are generally a good signal of at least minimal training in research skills. Another good general rule of thumb is that paying for a PhD -- beyond perhaps some MD/PhDs or maybe nursing phds, stuff like that -- is always a good sign of someone who has both a meaningless degree and also poor reasoning/research skills. But anyways, real doctorates outside of a few fields (e.g., pure math) usually come with a non-trivial publication record that speaks for itself. You don't even need to know that the person has a doctorate; you can just read their papers and a rec letter from an advisor describing the student's role in each paper. (I'm excluding discussion of professional degrees like JDs, PharmDs, etc. which are technically doctorates but sort of their own class.)
    - 5y ago
deleted 5y ago
[deleted]
MengerSponge 5y ago
Does Poe's Law cover parody becoming real? Because BBSpot called this nearly 18 years ago: "Word 2004 to Pioneer AutoUnsummarize Feature" https://www.bbspot.com/News/2003/12/autounsummarize.html https://www.bbspot.com/News/2003/12/autounsummarize.html
bjourne 5y ago
I really doubt you can computer generate a Master's thesis. Completing a Master's thesis at an accredited institution is a heck of a lot of work and even a cursory reading of a thesis by an examiner, supervisor, opponent, or other interested party would give the generated content away. Maybe if you get your degree from a diploma mill you could get away with it, but then your degree wouldn't be worth toilet paper anyway. I've heard similar stories about generated phd theses and it is even more implausible. The reason is that writing a thesis is much more than just producing a hundred pages or so of prose. Any university student can poop that out in a few weeks. The main job of a thesis is coming up with a research question, conducting an experiment or a study, and describe the results and how it fits in whatever niche of the scientific world you are working in.
- hdjjhhvvhga 5y ago
  I agree that in most cases it would be very difficult to do. But I can imagine some specific circumstances where it could be pulled off, possibly with some manual modifications: soft sciences like sociology (you can't imagine the amount of bs I've read during my college years), the subject matter being very different from the area your supervising prof specializes in, the topic that allows for arbitrary speculation, an underfunded university branch with profs having a more lax attitude.
  - andai 5y ago
    https://xkcd.com/451/ https://xkcd.com/451/
  - deleted 5y ago
    [deleted]
- dash2 5y ago
  Oh my sweet summer child. I regularly get dissertations with any or all of: barely readable English, useless empirics, half-baked research questions.
  - pottertheotter 5y ago
lostlogin 5y ago
> third party content generation and paraphrasing tools. Presumably this is an arms race against things like https://www.turnitin.com/ https://www.turnitin.com/ Empower students ‘to do their best, original work’ and this is what you get. Though what the alternative is, I have no idea.
laurent92 5y ago
> I know of several individuals who were able to complete their Master’s thesis utilizing… Doesn’t it stay published forever? Might be a shame for the someone during their career. On the other hand, even a chapter of Mein Kampf was accepted in 20 journals, after replacing the old word with newer versions. Human reviews are hard. Maybe we should put computers in charge of reviewing papers, they’d recognize the work of AI quicker? https://www.foxnews.com/us/academic-journal-accepts-feminist-mein-kampf https://www.foxnews.com/us/academic-journal-accepts-feminist...
lelanthran 5y ago
> I know of several individuals who were able to complete their entire Master's thesis utilizing a combination of AI generated content (GPT-3) and a paraphrasing tool. That's nice, how did it do the defense?

by gzer0 5y ago

phkahler 5y ago

Sounds like automated review of automatically generated papers. And people pay money for that...

13415 5y ago

Please include a link to these theses, because as it stands this anecdote sounds extremely implausible. I don't know what university you were, but I've been at a few in Europe and at every one of them Master theses were evaluated from the start to the end by several humans. GPT-3 is unable to produce even two pages of coherent text, let alone 50 pages good enough to be accepted as a Master thesis in any discipline at any university I could think of (even the worst ones). I can imagine that plagiators use paraphrasing software quite extensively, though, and that it is a problem.

gzer0 5y ago
Let me clarify: It was not all automated, there was a fair bit of manual intervention needed. I understand your concerns and they are valid and this is why I preface my statement with "anecdotal evidence". What I write is most certainly not the entire story and a fair bit of detail is left out. It should be known that this is widespread across multiple industries and this will only become more of an issue in the future. This is a US-based institution, fully accredited.

throwawaygh 5y ago

> Master's thesis I don't doubt this at all, and I have no doubt that GPT-3 with a bit of human editing can spit out something better than the lower third of masters students at corn row colleges. Masters degrees are cash cows, which is why no one in unregulated industries cares about them. People in regulated/unionized industries also don't actually care; even educators, who at least nominally see intrinsic value in education, go to borderline diploma mills to get that union-mandated raise at minimal effort.

derefr 5y ago
How do you feel about doctorates?
- throwawaygh 5y ago
  Depends. University of Phoenix awards doctorates that take 3-4 years (HUGE red flag -- the best and brightest phd students might get out in 4 years if everything goes perfectly; an "expected time to graduation" of anything less than 5 years is almost certainly a worthless degree). Those doctorates don't require much more than taking some coursework and paying a boatload in tuition. Basically an expensive and length online masters program. Not worth the paper they're printed on, unless you're employed by the government or in a union job that mandates raises for education attainment. As a general rule of thumb, PhDs from R01 universities that are paid for by the university through research assistantships or teaching assistantships are generally a good signal of at least minimal training in research skills. Another good general rule of thumb is that paying for a PhD -- beyond perhaps some MD/PhDs or maybe nursing phds, stuff like that -- is always a good sign of someone who has both a meaningless degree and also poor reasoning/research skills. But anyways, real doctorates outside of a few fields (e.g., pure math) usually come with a non-trivial publication record that speaks for itself. You don't even need to know that the person has a doctorate; you can just read their papers and a rec letter from an advisor describing the student's role in each paper. (I'm excluding discussion of professional degrees like JDs, PharmDs, etc. which are technically doctorates but sort of their own class.)
  - 5y ago

deleted 5y ago

[deleted]

MengerSponge 5y ago

Does Poe's Law cover parody becoming real? Because BBSpot called this nearly 18 years ago: "Word 2004 to Pioneer AutoUnsummarize Feature" https://www.bbspot.com/News/2003/12/autounsummarize.html https://www.bbspot.com/News/2003/12/autounsummarize.html

bjourne 5y ago

I really doubt you can computer generate a Master's thesis. Completing a Master's thesis at an accredited institution is a heck of a lot of work and even a cursory reading of a thesis by an examiner, supervisor, opponent, or other interested party would give the generated content away. Maybe if you get your degree from a diploma mill you could get away with it, but then your degree wouldn't be worth toilet paper anyway. I've heard similar stories about generated phd theses and it is even more implausible. The reason is that writing a thesis is much more than just producing a hundred pages or so of prose. Any university student can poop that out in a few weeks. The main job of a thesis is coming up with a research question, conducting an experiment or a study, and describe the results and how it fits in whatever niche of the scientific world you are working in.

hdjjhhvvhga 5y ago
I agree that in most cases it would be very difficult to do. But I can imagine some specific circumstances where it could be pulled off, possibly with some manual modifications: soft sciences like sociology (you can't imagine the amount of bs I've read during my college years), the subject matter being very different from the area your supervising prof specializes in, the topic that allows for arbitrary speculation, an underfunded university branch with profs having a more lax attitude.
- andai 5y ago
  https://xkcd.com/451/ https://xkcd.com/451/
- deleted 5y ago
  [deleted]
dash2 5y ago
Oh my sweet summer child. I regularly get dissertations with any or all of: barely readable English, useless empirics, half-baked research questions.
- pottertheotter 5y ago

lostlogin 5y ago

> third party content generation and paraphrasing tools. Presumably this is an arms race against things like https://www.turnitin.com/ https://www.turnitin.com/ Empower students ‘to do their best, original work’ and this is what you get. Though what the alternative is, I have no idea.

laurent92 5y ago

> I know of several individuals who were able to complete their Master’s thesis utilizing… Doesn’t it stay published forever? Might be a shame for the someone during their career. On the other hand, even a chapter of Mein Kampf was accepted in 20 journals, after replacing the old word with newer versions. Human reviews are hard. Maybe we should put computers in charge of reviewing papers, they’d recognize the work of AI quicker? https://www.foxnews.com/us/academic-journal-accepts-feminist-mein-kampf https://www.foxnews.com/us/academic-journal-accepts-feminist...

lelanthran 5y ago

> I know of several individuals who were able to complete their entire Master's thesis utilizing a combination of AI generated content (GPT-3) and a paraphrasing tool. That's nice, how did it do the defense?