4 ms·
On the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil. Academia and education desperately wants this software to wo
by simonw 4y ago
On the one hand, this is sorely needed: AI detection software will inevitably be mostly snake oil.
Academia and education desperately wants this software to work! As a result, selling them something that doesn't work is going to be very profitable.
The most obvious problem with this class of software is how easy it would be to defeat if the students could access it themselves: generate some text, run it through the detector, then fiddle with it (by manually tweaking it or by prompting the AI to "reword this to be less perfect") until it passes.
Which means these tools need to not be openly available... which makes them much harder to honestly test and evaluate, making it even easier to sell something that doesn't actually work.
But... I don't think this site is particularly convincing right now. It has spelling mistakes (which at least help demonstrate AI probably didn't write it) and the key "How AI Detection Software Works" page has a "Coming Soon" notice.
The "examples" page is pretty unconvincing right now too - and that's the page I expect to get the most attention: https://itwasntai.com/examples https://itwasntai.com/examples
It looks to me like this is still very much under development, and is not yet ready for wider distribution.
- JohnFen 4y ago> AI detection software will inevitably be mostly snake oil. Probably so. The problem, of course, is that the inability to detect AI authorship leads to the increase of general distrust of everything in society.
- deleted 4y ago[deleted]
- ShamelessC 4y agoThat ship sailed with social media unfortunately.
- cookieperson 4y agoWait... Are you saying that I shouldn't be eating tide pods?
- rorroe53 4y agoBut why should educational instutions care? Education is a business, students are the customers (or in countries with state-funded education, the government). If AI helps people graduate faster, that's more money to the institutions, less effort to the students, and nice statistics for the governments. At least in my country most degrees aren't worth much anyway, they just open you doors to internships where you really learn stuff. AI isn't going to make the situation any worse.
- simonw 4y agoThey care because this is massively disruptive to the way they teach at the moment. They have decades of practices in place for how they evaluate students which don't work if students can have AI do the work for them. They can chose to reinvent everything about how they operate, or they can pay money to a company that promises to make that problem go away for them. It's not surprising that many of them are trying the latter option first.
- visarga 4y agoPeople getting into AI today follow courses like: LLM University - https://docs.cohere.com/docs/llmu https://docs.cohere.com/docs/llmu and never get to learn about linear regression, bias and variance, cost function and gradient descent, regularisation and optimisation - all the good things taught by Andrew Ng in the amazing course he run 12 years ago just before creating Coursera. Is that a good thing?
- simonw 4y agoLearning about those things isn't particularly relevant to learning how to use LLMs for NLP tasks. Not saying they're not worth learning, but I think it's reasonable for them not to be included in the syllabus for that particular course. Kind of like how learning memory management in C doesn't need to be a pre-requisite for a course on Python. Note that learning about those things will likely make you a better LLM+NLP practitioner, in the same way that having a good grasp of memory management in C will help you be more effective at working with Python - but it's OK to leave them out of introductory courses.
- 4y ago
- visarga 4y ago> It has spelling mistakes (which at least help demonstrate AI probably didn't write it) Nowadays if you want to be convincing you got to maek some spelling misakes. Something that looks like predictive keyboard errors, or typing errors.
- shortj 4y agoFunny enough, one fo the first things I did with ChatGPT was teach it how to write with charcater keyboard transpositions and subtle typos so people wouldn't think varioius text was AI generated. Works pretty well. (Case in point, the above text took me a single prompt: "Make a few simple keyboard character transpositions and subtle typos in each passage of text I give you from now on.")
- wizofaus 4y agoStill, I doubt an LLM would ever start a sentence with "Funny enough" (the normal usage is "Funnily enough" but even that doesn't seem like something the current crop of AIs would use without explicit prompting).
- shortj 4y agoNice catch and very good point. The model actually tried to fix this in a couple cases, but not all, when I was playing with it to see how else it would typo.
- paulddraper 4y agoObligatory XKCD: https://xkcd.com/810/ https://xkcd.com/810/
- swyx 4y agothis is exactly why i was stunned that someone apparently gave 3m to GPT-zero despite all its known flaws https://www.forbes.com/sites/rashishrivastava/2023/05/09/with-seed-funding-secured-ai-detection-tool-gptzero-launches-new-browser-plugin/ https://www.forbes.com/sites/rashishrivastava/2023/05/09/wit... its too easy to be negative about things in hype cycles and retroactively look back and go "see! i was right! this was a terrible idea!" but.. this is a terrible idea to ai detection fans: show us on an information theory basis how you will smuggle in enough bits, avoiding user obfuscation, please. i will change my mind and support you the moment you prove this can be done, otherwise i am default extremely skeptical
- ShamelessC 4y agoSuch a bad choice of name too. Ripe for litigation.