7 ms·

In other words, it disrupts classroom assignments where the student is being asked to produce bullshit but where historically the easiest way to produce that bu

• titzer 4y ago
  > This will certainly require a lot of reworking of these traditional curriculums that consist heavily of asking the student to generate bullshit, but maybe that wasn't the best way to educate students this whole time. Just curious which discipline you have a grudge against here. Because presumably disciplines are actually disciplines where someone working in the field for their entire career can spot BS.
  • SpaghettiX 4y ago
    GP did not mention disciplines, and I don't think individual disciplines are to blame. The approach of evaluating students based on "text generation" is very boring to study for, easy to fool (a parent/guardian can do it, last year's students can pass you an A grade answer, ChatGPT can generate it) and doesn't prepare students for reality (making new things, solving new problems).
    • picture 4y ago
      Not all of us can be making new things and solving new problems, however..
    • xoac 4y ago
      What if the thing you are supposed to be making is written communication in the form of a document, or a book, or a paragraph explaining why production was down? What if the goal of these things was to make a student literate?
    • juve1996 4y ago
      How exactly do you teach "Making new things, solving new problems."? How can you solve a math problem if you can't do basic algebra, even if you can run an algebraic statement through wolfram and get a result? You learn problem solving largely through solving already solved problems in life. They're new problems to the student, not the teacher.
  • g_p 4y ago
    Not the parent poster, but I suspect this may be pointing towards humanities-based subjects, where I've seen regular assignments in the form of essays on fairly subjective topics. I do think there will be a challenge in handling use of GPT in some kinds of essay questions, even in objective and "right/wrong" type disciplines - I asked a friend to grade some answers to an essay style exam question, and they felt the GPT output was considerably better than many students in the class - ChatGPT was producing markedly better output than weak students. This wasn't a properly blinded or double-blinded test, but the output from GPT was so much better than weaker students that there was little need. You could tell from glancing at the text that it was better. ChatGPT had far better grasp of the English language, and the language model meant what it wrote was more coherent, structured, and better flowing than a weaker student would write. It had a significantly better vocabulary than many students, and the GPT output appeared to always use words correctly. Sentences were fully-formed and coherent. They had proper grammatical structure, made proper use of punctuation, and advanced a cohesive idea (as the essay asked for). This wasn't true of weaker (human) students' attempts. The content that ChatGPT produced was not always factually perfect, but when compared with anyone who wasn't "top of the class", there were fewer factual errors in the Chat GPT output, since it was mostly regurgitating fairly simple text. A paraphrase of part of the Wikipedia article would probably have beaten many students though. Where GPT output falls apart is references - even when asked to produce references, they are generally non-existent (plausible paper name, plausible author, plausible journal name, just that paper doesn't exist and that author didn't publish anything in that journal, etc.), and do not actually support the statements which cite them - this still seems to be a fairly robust way of detecting non-human output at present. It would be interesting to see this kind of experiment carried out at scale in a controlled environment, where the examiners are unaware that GPT outputs are interspersed with real students' work, in order to make it a fair assessment.
• deleted 4y ago
  [deleted]
• Erik816 4y ago
  If it can generate working code, does that mean that asking students to produce working code was bullshit? Or does it just mean that AI can now do a lot of things we used to asked students to do (for probably solid educational reasons) at an above average level?
  • tshaddox 4y ago
    If it's reliably generating working code then that isn't bullshit! (Ignoring, of course, other things about the code that might be relevant to the assignment, like coding style or efficiency.) What I'm saying is that if you are looking at the AI's output and judging that it's bullshit, and if you can't distinguish that output from your students' satisfactory essays, then that by definition means that the assignment was to produce bullshit.
    • xoac 4y ago
      This is a pretty dumb take and it's repeated in this thread. The goal of the assignment is not a result. I mean the professor can probably write a better essay than some kid who just learned that this subject exist. The point of this exercise is to have the student learn how to do the work, so they can do it when it's not a simulated exercise.
      • brookst 4y ago
        You’re missing the point. Even if you disagree with the point, it’s important to understand it. If the goal is “to have the student learn how to do the work”, and there is a tool they can use to do so, then using the tool is doing the work. Your position only makes sense if you define “the work” to also include exactly the process you personally learned. No fewer tools (did you learn on a word processor?), no more tools (is spellcheck OK?).
        • lm28469 4y ago
          I think you're missing the elephant in the room: we learn _to_ learn and be able to adapt, not to "do the work".
• lukev 4y ago
  I agree with some of this (particularly the conclusion that better education methods are required) but lets be a bit generous for a second. The ability to write well is (or was) an important skill. Being able to use correct grammar, to structure even a simple argument, to incorporate sources to justify one's statements, etc. Even if we're just talking about the level of what GPT 3.5 is capable of, that still corresponds to, let's say, a college freshman level of writing. Now, perhaps with the advent of LLMs, that's no longer true. Perhaps in the near future, the ability to generate coherent prose "by hand" will be thought of in the same way we think of someone who can do long division in their head: a neat party trick, but not applicable to any real-world use. It isn't at all clear to me though that we're yet a the point where this tech is good enough that we're ready (as a society) to deprecate writing as a skill. And "writing bullshit" may in fact be a necessary element of practice for writing well. So it isn't self-evident that computers being able to write bullshit means that we shouldn't also expect humans of being able to write bullshit (at a minimum, hopefully they can go well beyond that.)
  • tshaddox 4y ago
    The teacher (or developer of the curriculum) needs to decide both 1) what they consider to be cheating and 2) how important it is to establish a certain level of confidence that students aren't cheating. If writing grammatically without any external help is deemed to be extremely important, and it's far too easy for students to cheat on take-home writing assignments, then suck it up and use some of your in-class time for writing exams. Your in-class time is finite anyway, so deciding how to use it is already critically important. If electronic earpieces become commonly used by students to cheat on in-class exams, and that's important enough to the teacher, I guess they better figure out a way to screen for those devices. If eventually nearly everyone has some sort of brain link that can whisper grammatical phrases directly into their brain and cannot feasibly be detected in the classroom, well, then either your entire curriculum is on the honor system, or maybe you'll need to update your thoughts about what tasks are important for your students to be able to perform without "external" help.
    • lukev 4y ago
      Oh, I agree. Optimistically, the existence of LLMs will be a forcing function for education to focus on what actually matters. (IMO, this is the ability to critically assess and generate arguments, something GPT isn't particularly good at (yet)). But your example is actually rather striking. In most highschool and college classes for algebra and calculus, we're asking students to solve/prove problems that computers have been easily able to solve/prove, and have been for decades. But the educational consensus is that being able to do algebra or differentiate/integrate by hand is valuable (up to a certain complexity.) Which is why calculators are not allowed (at certain levels) and why "show your work" is an important part of grading at every level. Perhaps this whole discussion is a nothingburger once we figure out what the language arts equivalent to "show your work" is for text generation.
• fardo 4y ago
  > This will certainly require a lot of reworking of these traditional curriculums that consist heavily of asking the student to generate bullshit, but maybe that wasn't the best way to educate students this whole time. That process was invaluable for making normal bullshitters into bullshit artists -- how will we train our elite human bullshitters in the modern age of language models?
• gmd63 4y ago
  The point of school work isn't to generate something of value externally, it's to generate understanding in the student. Much like lifting weights or running are "bullshit" activities. You haven't seen bullshit until you see whatever is produced by a society with a voracious appetite that runs on "gimme what I want" buttons and a dismal foundation of understanding. The students bullshitting their way through course work are like the lifters using bad form to make the mechanical goal of moving the weight easier in the short term. They completely miss the point.
  • p1necone 4y ago
    > The students bullshitting their way through course work are like the lifters using bad form to make the mechanical goal of moving the weight easier in the short term. They completely miss the point. Eh, when your future career prospects hinge on what grades you get it's perfectly rational to optimize for getting those grades over learning "properly". Especially in subjects you know you're already competent enough in or that aren't relevant to your future goals. It's stupid in weightlifting because nobody is going to pay you a higher salary if you can do 15 reps instead of 10. (Although there's a related and opposite problem where people massively overestimate how important their highschool grades are and get unnecessarily stressed out over it.)
    • Avicebron 4y ago
      If you've written off yourself as "competent enough" or things "aren't relevant to future goals" you might want to think about the phrase "you don't know what you don't know"
      • p1necone 4y ago
        I mean stuff like bullshitting your way through an english assignment when you're going to study computer science at uni. You can be fairly confident that understanding the intricacies of Catcher in the Rye won't be important in any compsci paper or dev job, but the writing skills from the bullshitting are still useful.
    • 4y ago
• anvuong 4y ago
  Bullshitting well is an important skill in life. Sometimes the content doesn't matter, it's how you say it that matters.
• pazimzadeh 4y ago
  Yeah but it would be nice if that was the floor instead of a ceiling - personally I've never seen a ChatGPT-generated text that couldn't have been generated by asking a high schooler to generate the same thing.
• deleted 4y ago
  [deleted]
• g9yuayon 4y ago
  > > where the student is being asked to produce bullshit It depends on which subject? I have only good things to say about my college days. They are life changing. The rigorous training in computer science and maths has been paying dividend all these years. The professor in the writing class did not only teach me how to write, but also how to understand and appreciate literature. Even classes like Canadian Culture and History taught me so much about arts, literature, politics and history of Canada. I have a hard time fathoming in what classes will we write bullshit (yeah, I have some ideas, but I guess I was lucky enough that my school taught me how to love and appreciate the wonder of nature and civilization, instead of teaching me hatred to towards you know what).
• Al-Khwarizmi 4y ago
  You make it sound bad, but there's nothing inherently wrong with learner outcomes being worthless, or even worse than worthless. When people learn a new language at a very basic level, they generate dull and simple sentences, and butcher the pronunciation. When they learn martial arts, they kick the air or a punching ball. When they learn to play the violin, they play hellish sounds that can make your ears hurt. But all that is still needed for learning.
• Spooky23 4y ago
  Part of education is learning how to write. Some of that may be bullshitty, but useful nonetheless.
• siwatanejo 4y ago
  Hahaha that's the most insightful comment I've read in HN thus far.
• ern 4y ago
  The education system is moribund. During Covid, I thought we'd see the flipped classroom shine, and a revolution in education. Instead we got children chained to Zoom or Teams and attempts to replicate traditional classrooms remotely. I wouldn't be surprised if my grandchildren are still doing the same bullshit in 20 years.
• lm28469 4y ago
  Why would you learn math when you have calculators ? Why would you learn to write when you have grammarly ? Why would you read books when you have summaries ? Why would you watch movies when you have reviews ? Sometimes, a lot of times actually, it's not about the result but the process. Otherwise the most optimal path at life is suicide or matrix style coma pods. Everything we do before a PhD is by definition bs, it's just regurgitating already known facts.