9 ms·
AI Checkers Forcing Kids to Write Like a Robot to Avoid Being Called a Robot
- SteveGerencser 2y agoI submitted several scientific/research papers my wife wrote or co-authored in the late 90s early 2000s and they were all flagged as AI written by multiple tools. It was good for a laugh.
- fakedang 2y agoThat's just more evidence that the Kaylonian robots have already infiltrated our social and familial ranks. /s
- oerdier 2y agoAs a teacher, my answer to publicly available AI, regarding grading, has been to only let work by students I have seen been done myself (or staff, trusted implicitly) count towards passing/failing/diplomas. This means graded longer tasks such as writing or programming, which in the past were done outside of class and then handed in, are instead done in person (like most exams are, traditionally). To acknowledge the longer process, the time alotted for such a test is much longer (up to 3 hours instead of 1) and I can expect less of work created in 3 hours than work created over 6 weeks. Students still have the opportunity to hand in work that was done outside of such an in-person test, which I will grade, but that grade isn't on the record is serves only as an indication. I wish I could trust students, but plagiarism has become so easy now (added to a zeitgeist of personal gain above all) that a critical mass fails to resist. I think diploma's are very useful. They get their usefulness from the trustworthiness of the party handing out the diploma. Only recording grades for work that has been done in an environment controlled by that party has downsides, but in the current and foreseeable situation I don't see a better way of maintaining a base level of trustworthiness. I love teaching without grades or diplomas involved so much more. Then it's all about doing what's best for the students.
- candiddevmike 2y agoAre you seeing a shift in class duration to accomplish this? For my kids, I don't see how their classes can squeeze both instruction time + work time in the allocated slots. Is instruction time being sacrificed or "replacing" homework (watching videos/reading text at home)?
- oerdier 2y agoIn principle nothing about the structure of the course has to change. You can keep an existing course with instruction during class and and working on a project outside of class. Even handing in the project and grading could stay the same, although the grade for the project wouldn't be recorded and would serve only as an indication. You then add a longer test at the very end, for a grade. How I set up my course is different from traditional education. Most instruction happens from self study (including videos and software). Face to face time is for questions. All elements, material, staff, and students, have to be tuned for this setup to work though and is hard to pivot to in existing setups because education (in the Netherlands at least) is very entrenched. This self-study centered learning approach, like the in-person tests instead of projects, comes from necessity. Great (and I expect ever increasing) teacher shortages.
- orwin 2y agoA friend of mine seems to have switched to a "exercise/exam in school, learn at home" methodology, where the next lesson is given in sheet format at the end of the hour, and "school" is mostly exercises he uses to see how much each student understand. It seems to be quite happy with this, and told me good students are quite happy with this (lessons are learned at their own pace), and poor ones now have more time with him helping them during the exercises. Only issue is that he now have troubles not talking to/helping students during exams and now really hate those (always disliked them as well as grading, but now it seems worse). He also heavily encourage student to learn with each other during "perm" hours (basically in highschool, hours where student have to stay in school but have nothing to do and can work on their own, go to the library or play/chill/whatever), but that was before and because he fondly remember the days where all 26 of us, the full class, all teamed together against homework (yeah, i had great classmates)
- samarthr1 2y agoOh god, this really does capture my problem. My university demands 40 page reports that are never read by anyone, so my seniors used ChatGPT to generate the needed drivel needed, so now we have the ai detectors, and are told that the % of AI content as flagged by a tool (turnitin) must be below 10%. Unfortunately, the way I write, (like any well educated Indian) is a rather verbose, formal and rather florid in it's manner, and with no fewer than 3 conjuctives (when I don't want the reader to understand what I just wrote) per sentence. This style is for good or bad understood by turnitin as being AI generated. So, I have had to unfortunately simplify and write in a more stilted manner, just to be able to submit my assignment.
- Frenchgeek 2y agoTrain ChatGPT to avoid being flagged by turnitin? (Then write a paper on it in a way designed to trip said AI detector)
- samarthr1 2y agoBeen there, tried that. It only sorta worked, but failed basis m basic "formalism" tests. So could not be directly used. The writing style and vocabulary expected by Indian universities is a style of English that has not been used in England for 80 or so years. It is uniquely frozen in time, florid, and all together horrid. (I had to make it rhyme, sorry)
- greiskul 2y ago> when I don't want the reader to understand what I just wrote Unless you are writing poetry or art (where ocasionally this makes sense, to avoid censorship for instance, or to make a point), that's just bad writing. Should you get penalized for plagiarism? No. Should you get bad grades for writing like this? Probably yes.
- samarthr1 2y agoIt only matters when someone actually reads it no? If a tree falls and nobody heard it, did it actually fall? Also relevant, this is for my final year engineering project, not a writing course. The report that I write is not even read by those who grade my project, rather, the page count is checked, and a small amount of sanity-check-reading to make sure I have not just filled the pages with cinema songs is done. My grade is decided by my relationship with the panel, nothing else, not the quality of work done, not the idea, not the deliverables shown. I am absolutely sick and tired of this. I just wanted to do a good job, and write good code, but no, that does not move the needle one bit with our panel of reviewers. (Pardon my rant, but I really did want to do a great job with my project, but my university did not want to reward it) At any rate, I can get into a whole rant about incompetent lecturers at many Indian universities, but that is way off topic.
- madrox 2y agoWriting teachers have a lot to learn from Math teachers, who have had to fight against the crutch of calculators for decades. The future here is obvious. More in-class tests to evaluate writing ability instead take home is what will happen. For take home, require "showing your work" and the research that went into writing. Show the steps. Yes, you could still use AI for any take home work, but I think what forces students to AI isn't the lack of will to do the work. It's the "I have a 5 page paper due tomorrow and I haven't started" cliff. I mean, writing education is messed up in so many ways, but if I'm being realistic this is the path forward.
- ok_dad 2y agoSeriously, the answer is to have the students turn essays in using stages. First stage is to write on paper by hand, even if there’s not a lot of research or facts yet. Lay out the argument first using what they remember from the initial research, then look it over with the kid and have them explain it a bit. Then, they do the next stage which is to do closer reading and research, and fill in the gaps. Finally, they polish the essay and turn in the final. It’s hard to use AI to do this, and if a student did use AI effectively like this, they’ll still learn a lot about the topics, because they’re forced to think about it. Imagine, rather, that you have these kids actively use AI to help them write an essay, but make them provide the transcripts so you can so how they think and use the tool. Perhaps even sit with them and use the AI tools and show how they can be used as a guide, as long as you don’t take the responses as facts like in an encyclopedia, but rather use AI like a parent who is getting but doesn’t quite remember the details. You’d not trust it implicitly but just use as a guide. Perhaps someone should make an AI that isn’t so sure of itself, and suggests ways to research things yourself but points you towards the closest cardinal direction to your question?
- logicchains 2y agoThis is an expected consequence of using ML to detect AI. LLMs are more well-read than the average person, so they'll have a bigger vocabulary than the average person, so the ML model will learn to associate a bigger vocabulary with a higher probability of being AI. Which unfairly penalises well-read humans, who also have a bigger vocabulary than average. Personally I think the whole endeavour is a waste of energy. Unless there's a massive civilisational collapse, humans will always have LLMs as writing aides, so trying to teach them to write without one is teaching them a skill they'll never need to use. If you want to teach them to reason on the spot, then go back to requiring them to orally put forth and defend an idea.
- rootusrootus 2y ago> If you want to teach them to reason on the spot, then go back to requiring them to orally put forth and defend an idea. I have been thinking for the last couple years that we may end up in that spot. It is more labor intensive, and perhaps a little more susceptible to bias, but pretty hard to fake. At the end of the term, each student has half an hour with the professor (or a TA) where they discuss the content of the class. It should be quickly apparent who knows the material well enough and who skipped too many lectures.
- actionfromafar 2y agoUbicomps colonial impulse, back for round 2.
- jedberg 2y agoIf you put Shakespeare or the Constitution through one of those, they will say 90%+ AI. Any student who gets dinged by one of those things just needs to show this result until the administration caves and bans the stupid checkers. They may have to go to the local press or a board meeting to get that result though.
- thih9 2y agoDo you have a source on Shakespeare being flagged? I just tested first few lines of Hamlet with a random AI detector[1] and got a result of 100% human. I saw a post saying that the opposite happens, i.e. AI output, when prompted to write like Shakespeare, can beat AI detectors[2]. [1]: https://quillbot.com/ai-content-detector https://quillbot.com/ai-content-detector and https://www.zerogpt.com https://www.zerogpt.com [2]: https://old.reddit.com/r/freelanceWriters/comments/1amngnv/ive_beaten_originalitydotai_well_shakespeare_did/ https://old.reddit.com/r/freelanceWriters/comments/1amngnv/i...
- jedberg 2y agoI don't have a direct link. It came from a reddit post by a professor who was trying to convince his department to stop using them by putting famous works through to show how bad it is. But my guess is that it works both ways depending on the cheating detector and the prompt used.
- dghlsakjg 2y agoThe most effective way to get them to stop is to put the professor's work through...
- mensetmanusman 2y agoSo that’s how Shakespeare did it…
- ronsor 2y agoAI checkers need to be sued out of existence.
- Animats 2y agoThere is a certain style to LLM-created writing. It tends to float in the air, lightly tethered to real world examples. Unfortunately, that's also what passes for academic style in some quarters. I mentioned that a few days ago here.[1] LLMs have grammatically correct blithering nailed. One amusing result is that Derrida and his followers now sound like an LLM with the temperature medium high. Maybe we should be teaching kids to not write like that. Mandatory XKCD.[2] [1] https://news.ycombinator.com/item?id=41419645 https://news.ycombinator.com/item?id=41419645 [2] https://xkcd.com/451/ https://xkcd.com/451/
- arjie 2y agoInteresting that my childhood evaluation systems in India are resistant to this new development. 100% scores from in-person exams, no loss of dynamic range through grading (i.e. 97% score > 96% score), zero score to homework. I imagine that the primary constraint in the US is labour cost since you need what you call a proctor and what we call an invigilator, and perhaps the substantial student power (a student and their parents can overrule teachers). I like the relative mobility in the Indian system since anyone can swot to the test, but clearly the US system is much more enjoyable for a wider variety of students and they have a wider range of skills. I think I'd like for my children to have a composite of the two. I dislike neverending homework and love tests because spiky activity allows for lots of self-directed exploration and the tests allow for evaluation. Though it's like that this is because I, personally, am quite capable with tests: I operate very well under time pressure and have an exceptional memory.
- samarthr1 2y agoIs the relative mobility a good thing though? (Not strictly on topic, so apologies) I would argue that for the exams that actually matter (the JEE, say) PCM scores are very poor indicators of success in the chosen branch, and are an arbitrary selection criteria, and not very removed from a lottery. A better system may include psychometric pairing for branch of engineering opted? My argument is essentially that if I like (say) Civil engineering more, then I would have demonstrated some additional knowledge in parts of the desired branch, so testing that should in theory give better civil engineering graduates, instead of people who joined it for the tag of the institute and not for the branch? But again, I am a _bit_ salty about the 12th standard exam treadmill in india.
- arjie 2y agoYes, it's practically a lottery with a skill floor. And I think that's a pretty good way of doing things. The purpose of these technical institutes is to reliably train people into a bunch of skills, not to get people who have the skills already and stamp them. The purpose of an entrance exam is to have a method that: - demonstrates a floor of intelligence and skill - is demonstrably fair - has high dynamic range - can be trained against by the conscientious I think the JEE behaves well against these criteria. In addition, affirmative action is explicitly handled by quotas. This gives a predictable system with known constraints. When participating, you know your targets. Personally, if I know my constraints ahead of time, and I can optimize to them by studying, and the studying is directionally useful to education, that is sufficient to me. If I fail, it's on me and I can accept that.
- hobs 2y agoHow can we claim any high ground when we are handing off the need to check for plagiarism to a tool that in effect is the same as the student using it to create plagiarized works. It doesn't even work, its non deterministic, and most importantly - if the teacher is familiar with the student's writing it should be pretty obvious if this is a fake or not based on something they can be observed doing.
- rafram 2y agoInteresting piece. I really wonder why these tools even show detection percentages when they’re meaninglessly low. A 17% result from a classifier model is totally meaningless. You could show a well-trained animal classifier the first Google Images result for “cat” and it would still say there’s a 17% chance it’s a cow. Showing low-likelihood predictions will do nothing but freak people out and confuse them.
- dheera 2y agoOK so back then using SAT words was good, and now using SAT words is bad. Got it.
- deleted 2y ago[deleted]
- from-nibly 2y agoTests should not be a tool for authenticating learning. They were NEVER good tools for this and now its clearer why. Tests should be used for a teacher to help a student figure out where they need to focus. It should be like getting a blood test from a doctor unless they are testing for drugs why would you try to cheat with someone elses blood? We don't need to figure out how to sidestep the fundamental flaws with our institutional education we need to find and fix the root cause. Being "autheticated" as "knowing stuff" (your degree and GPA) doesn't translate into companies being able to tell if you can do a job. It didnt before AI and it doesn't now. Everyone should stop and think about how dumb it is that children and adults are trying to cheat at learning. You can't cheat learning. So why do people try and succeed? Because we arent actually asking for learning we are asking for something else.
- RachealMuldoon 2y ago[dead]