12 ms·
It Wasn't AI
- kunley 3y agoInteresting why this site is down...
- tikkun 3y agoThis is a great idea. Huge problem. Most of these AI detectors have poor accuracy, but it seems like education institutions treat them as truth.
- morkalork 3y agoHow many essays are written and graded every year? Even with a 0.0001% FPR, how many students would be facing the severe punishments for plagiarism like auto-failing a course or even expulsion? Ironically enough, using AI to make such decisions seems like one of those unethical use cases.
- minimaxir 3y agoContext for the website creation: https://www.reddit.com/r/ChatGPT/comments/13hi5y6/i_fed_gptzero_a_letter_from_1963_that_went_viral/ https://www.reddit.com/r/ChatGPT/comments/13hi5y6/i_fed_gptz...
- krossitalk 3y agoIronic that I had to pass Cloudflare "Are you human" verification.
- nixcraft 3y agoYes, that is true. ChatGPT refuses to work with VPN and regularly checks for your IP using Cloudflare. Most of these new AI-driven startups are using Cloudflare. It is funny that startups collect/scrap data from all over the web but don't want to scrap their chatbot response without paid API. I guess it is life...
- jszymborski 3y agoI literally close the tab when that happens.
- simonw 3y agoOn the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil. Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable. The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the detector, then fiddle with it (by manually tweaking it or by prompting the AI to "reword this to be less perfect") until it passes. Which means these tools need to not be openly available... which makes them much harder to honestly test and evaluate, making it even easier to sell something that doesn't actually work. But... I don't think this site is particularly convincing right now. It has spelling mistakes (which at least help demonstrate AI probably didn't write it) and the key "How AI Detection Software Works" page has a "Coming Soon" notice. The "examples" page is pretty unconvincing right now too - and that's the page I expect to get the most attention: https://itwasntai.com/examples https://itwasntai.com/examples It looks to me like this is still very much under development, and is not yet ready for wider distribution.
- JohnFen 3y ago> AI detection software will inevitably be mostly snake oil. Probably so. The problem, of course, is that the inability to detect AI authorship leads to the increase of general distrust of everything in society.
- deleted 3y ago[deleted]
- ShamelessC 3y agoThat ship sailed with social media unfortunately.
- cookieperson 3y agoWait... Are you saying that I shouldn't be eating tide pods?
- 3y ago
- jvanderbot 3y agoAny sufficiently good AI detector will soon become part of the training process.
- jandrese 3y agoI am concerned about the false positive rate on AI detectors. But if they are being used to flag submissions for further review only then I can go with it. Finding out if a student cheated shouldn't be too hard, just quiz them on what they wrote.
- notahacker 3y agoAnd even before quizzing them, their work can be cross referenced against their other writing. If they've gone from writing with poor understanding of subject matter basics and poor written English to something like an exemplar essay in a matter of weeks, it's pretty clearly they've had some sort of help whether that's AI or not.
- danenania 3y agoThat's true, but it's also straightforward to train detectors with massive quantities of human and AI-generated text. So AIs can be trained by the latest detectors, but detectors can also trained by the latest AIs. It will probably come down to which side has more resources to devote to training.
- yieldcrv 3y agooooh I like that, the student can sue for copyright infringement because the teacher uploaded their work and proved that they uploaded it? sounds like a simple sublicensing clause imposed on the student will fix that, but the next few semesters a few examples can be made of the teachers and institutions will pay off that tuition
- petesergeant 3y agoProbably not; schools have been using Turnitin for a while, and presumably had students sign waivers
- Animats 3y agoThat's was litigated years ago, for TurnItIn.[1] Uploading student papers for plagiarism checking purposes was considered fair use by a US federal court. [1] https://macleans.ca/education/uniandcollege/judge-rules-anti-plagiarism-program-does-not-violate-copyright https://macleans.ca/education/uniandcollege/judge-rules-anti...
- okdood64 3y agoIf a professor fails you because they thought your final essay was written by an AI and it wasn't, do you have legal grounds to start a lawsuit against the school?
- jacobsenscott 3y agoCertainly anyone can start a lawsuit. Maybe there's eventually a chance for some kind of class action prevail, but I don't see how an individual student could make a dent against the resources of a university.
- aripickar 3y agoIMO one of the way that most schools are going to end up being able to detect plagiarism is going to be a custom word processor (or something similar) that can track all of the edits made into a document. Basically, have the students type an essay where all of the keystrokes are recorded by the program, and so it can be detected by the program whether someone is copy and pasting whole essays, or if someone is actually typing and revising the essay until it is submitted. Essays that are just turned in in general are probably going to be a thing of the past.
- bryzaguy 3y agoThis is a fantastic idea
- chrismorgan 3y agoThis is a horrible idea.
- gitfan86 3y agoYou would need to ensure that Chrome extensions and keyboards with macros were disabled somehow
- blacksmith_tb 3y agoOr they could move to a more British style, with in-person essays, proctored by human observers (not that there aren't old-fashioned ways to cheat on those too, but they're well-known).
- greiskul 3y agoYes, when I had to take university entry exams in Brazil, all parts of the exam were in person, including writing the essay, with a mandatory topic only disclosed when the exam starts. Preventing ai cheating might become more difficult for educational projects that are long form, like writing a dissertation, or big coding challenges. Although, for coding, one thing that I have consistenly seen work, is to just ask students to do a walk through of the code. People that just copy someone elses work are generally lazy, and don't really study what they copied, and it becomes easy to see who put in the work.
- moffkalast 3y agoHas anyone tried using an LLM as an LLM detector yet?
- minimaxir 3y agoThat is the implementation that many of these approaches are doing. It is a very, very hard problem.
- jacobsenscott 3y agoIt is an impossible problem. An llm can produce any combination of words. A human can produce any combination of words.
- flatiron 3y agoIt works…ish. GPT4 is pretty good at detecting what it wrote but I was able to get a false positive with the United States constitution. Or maybe we can go deeper and say maybe it was AI generated?
- jandrese 3y agoSo if you submitted the US Constitution word for word for your assignment you wouldn't expect the cheating detector to flag it? How isn't this flagrant plagiarism?
- function_seven 3y agoI would expect a plagiarism detector to flag it. I would not expect an "AI wrote this" to flag it, because it was not written by AI.
- jandrese 3y agoAn AI detector is basically a plagiarism detector trained on AI datasets. Fundamentally it's about discovering writing that is too similar to existing writing, which is why I'm concerned about false positives.
- inconceivable 3y agothis entire line of reasoning (using AI to detect AI, with disastrous results) is ripe for a giant lawsuit. a sufficiently wealthy school is bound to accuse a sufficiently wealthy student at some point.
- zzzeek 3y agoWith the advent of new technology, so must entire practices and industries spring up to counteract the inherent harm this technology will cause and is already causing. is it a given that technological progress will often necessitate societal harm? Is such technological progress actually progress for humanity? there seems to be this universal notion that "things that can be built will be built and are inevitable". It is for example argument #1 anytime anyone suggests we should be manufacturing and selling fewer guns - that this is not possible, since guns are "inevitable". You can 3-d print them after all! Therefore, everyone must be armed and we must live in an armed society with regular mass shootings, because what can we do? It's also a ubiquitous slogan used around AI - that AI is "inevitable". It's already out there, Google internal docs are betting that OSS AI will become the norm, and that's that. AI will be everywhere used for everything, making it's fairly unreliable decisions about things like who broke into your house last night, who's likely to be shoplifting, is that a bike in the crosswalk or just nothing at all, etc., and that's now the world we live in. Are humans as a species perhaps in need of better ways to not build things, since right now every possible thing that is imagined and becomes possible therefore "must" be built, en masse, and humanity's occupation becomes mitigating the species against all the harms brought about by all this "progress". anyway that's the low blood sugar version. I'll likely have not much to say after lunch
- kunley 3y agoVery well put. Why not to build nukes massively? Nukes are inevitable, because in the human nature there is inherent hunger for power (ask any psychologist nearby), and nukes are the ultimate power. With nukes at reasonable prices for wealthy families, the notion of an "atomic family" would really get a new exciting meaning! I am ofc sarcastic, but the reasoning is IMO the same like with the AI.. The real question here is: why to refrain from building nukes and why to refrain from uncontrollably developing AI.
- zzzeek 3y agoif you could build a nuke in your garage, the world would have ended long ago. the extreme difficulty, expense, and industrial-grade complexity in handling and enriching uranium and then forming it into a viable weapon is the only thing that kept that from happening
- Spivak 3y agoThis is an "everything sucks all around situation" because since real things are tied to academic performance you have to weed out dishonesty for fairness but also the power disparity between student/teacher and the black box nature of the detection makes it impossible to actually prove your innocence. I wish more than anything that the availability of AI will at some point force schools to restructure how classes work to make cheating like this a non-issue. Higher education is actually unbelievably horrible at actually educating. I only realized that once I graduated and on a whim wanted to learn about something that requires university level expertise. If you're not there for the credential it's a monumental waste of time. If classes were designed for students who wanted to be there and the grades were only for your benefit and not used as a target for anything you might actually have engaged learners.
- thih9 3y ago> Web server is down Error code 521 I’m seeing an error message from cloudflare. Is the website working for anyone else? Is there an archive / mirror?
- ehsankia 3y agoLooks like it is indeed down.
- dbrereton 3y agoSeems like the site might have crashed, here's an archive link: https://web.archive.org/web/20230515030802/https://itwasntai.com/ https://web.archive.org/web/20230515030802/https://itwasntai.... But tl;dr many students have been accused of using AI by teachers who think that AI detection software works, when it really doesn't. So the goal of this site is to communicate to teachers that AI detection software isn't reliable. I originally discovered this in a reddit comment which you can see here: https://www.reddit.com/r/ChatGPT/comments/13hi5y6/comment/jk5ndkq/ https://www.reddit.com/r/ChatGPT/comments/13hi5y6/comment/jk...
- jacobsenscott 3y ago"AI Detectors" are the new anti-virus - useless but once required by bureaucratic policy both unstoppable and profitable.
- andai 3y agoI had to prove I wasn't AI to load the page.
- jfghi 3y agoIronically, the best detector for plagiarism would be a 15 minute conversation asking the student about their research and opinions on the topics written, kind of like interviewing someone who claims redis expertise on their resume.
- r_hoods_ghost 3y agoThis is essentially the Oxbridge tutorial system where you have an hour(ish) meeting once(ish) a week with one or two other students and your tutor and talk over what you've learned, are set an assignment or three and have to answer any questions about the last week's assignments. A slightly more scalable version is the seminar system where you have up to 10 students and a tutor and you do roughly the same thing. It only works if participation is mandatory, missing seminars / tutorials is penalised and tutors are given leeway to adjust grades based on semina performance or flag students who do really well on their essays but appear incapable of explaining or defending what they've written.
- guy98238710 3y agoIt's not just university students anymore. My 11yo kid got accused of being a robot on a physics competition where the only reward is a (paid) summer camp full of extra physics lessons. All that was needed to trigger the accusation was a bit less fluent explanation of the solution, something you would expect from a student struggling with a difficult task. People are growing unreasonably paranoid.
- sidewndr46 3y agoThis seems kind of pointless to me. Long before I entered the academic system, Turnitin had pioneered the industry of accusing students of plagiarism while simultaneously claiming unlimited license to their works. They also built a parallel industry selling to services to students on how to avoid being considered as plagiarism. In the real world, that is known as organized. But in academia, it is business as usual.
- dahwolf 3y agoTo prove authenticity, be politically incorrect in a way AI never would. You'll be dismissed, canceled and fired, but at least authentically so.
- richbell 3y agoCAPTCHA v11: > What was the name of H.P. Lovecraft's cat?
- zirgs 3y agoI tried running gpt4chan locally. It's pretty good at generating politically incorrect, racist, misogynist and antisemitic stuff.
- refulgentis 3y agoWhat is up with these snowflake neues, posting bitter over-the-top political comments? They stick out like a sore thumb in spring 2023. It flew under the radar more when everyone was short-tempered, say winter 2021. The people who are still stuck on it and in their own heads seem to have a __negative__ herding effect. There's a seed of irrationality that drives some, and as there's fewer, they stick out more, making it more grating and driving more people away.
- dahwolf 3y agoWhat's up with catastrophizing an obvious joke? See, if you'd intellectually engage in good faith, you'd understand the point that one of few (if not the only) reliable ways to tell apart AI from humans in the future would be human shortcomings. A misguided offense extrapolated into pseudo-intellectual tone policing is a beautiful example of that, so you pass.
- deleted 3y ago[deleted]
- refulgentis 3y ago“Catastrophizing” lol. I wish you all the best, and hope you feel better soon my friend.
- deleted 3y ago[deleted]
- benhoyt 3y agoI've been reviewing answers to questionnaires we send out to potential software engineering candidates. Sometimes candidates seem to write 90% of the submission themselves, and then use ChatGPT for the last couple of questions (which are more general, like "Outline your thoughts on documentation in software projects"). I joked to a colleague that I'd come up with a fool-proof ChatGPT detector in one line of Python: is_chatgpt = paragraphs[-1].startswith(('In conclusion', 'Finally'))
- btilly 3y agoMy favorite example of it wasn't AI is that software to detect GPT flagged the US Constitution as almost certainly being AI. https://stealthoptional.com/news/us-constitution-flagged-as-ai-generated-content-by-chatgpt-detector/ https://stealthoptional.com/news/us-constitution-flagged-as-... As long as that kind of egregious mistake is possible, we should look at such tools with suspicion.
- cwkoss 3y agoTeachers need to ask students to write things that are hard for AI to cheat on: if a bunch of humans end up writing very similar essays to the prompt - that's a prompt problem!
- Balgair 3y agoI love this AI cheating detection stuff. Mostly because it really gets at the root of the issues in education. Like, fine, you have made some system where cheating is impossible. Great. But have your students learned anything? If educators put in even a iota of effort into learning their students, then they know who is cheating and who isn't. But if they put that same amount back into teaching, then everyone wins. Education is not a contest with winner and losers. (Yes, ok, you went to a bad school where it was a contest for your pre-med degree. Look where that has gotten US healthcare.)
- TRiG_Ireland 3y agoI like to think that when I was in college I wrote with enough flair and personality that no one could mistake me for an AI. Perhaps I'm overestimating myself.
- dweinus 3y agoWhat if we just let cheaters cheat? If they don't have the knowledge, they won't last long in a job that requires it. As the saying goes "You're only cheating yourself"
- ptdn 3y agoI am compelled to point out that in one of the info pages, the site includes screenshots of a conversation with ChatGPT where the author claims to trick AI detection by generating text with a lower temperature. But asking ChatGPT, through the LLM interface, to lower the temperature doesn't lower the temperature. There's no mechanism for it to do so. It may have some (nevertheless real) placebo effect, because the LLM thinks it should behave different and assigns some vague "meaning" to "temperature" -- but this isn't a technical change to the model operation.