10 ms·
AI Detectors Get It Wrong. Writers Are Being Fired Anyway
- cactusplant7374 2y agoThere was a reddit post about a college student being accused of using GPT because of the so-called AI detectors. I really think these educational institutions should be sued if possible.
- koolala 2y agowho will sue? the students with their dreams crushed? you?
- bigfishrunning 2y agoIf I were punished by my university for AI-assisted plagiarism I didn't commit, you're damn right I would sue. Imagine getting your life ruined because a neural network deemed it necessary. These things are wrong half the time, why does anyone trust them?
- sampo 2y agoHere is one, where a college professor didn't even use an AI detector, but simply copied some text from a student's essay into ChatGPT and asked "Did you write this?" and ChatGPT answered "I generated that passage". And the professor just believed it. https://www.reddit.com/r/academia/comments/14wa4nz/professor_says_i_used_chatgpt/ https://www.reddit.com/r/academia/comments/14wa4nz/professor...
- Bilal_io 2y agoThis is hilarious, sad and scary all at the same time.
- pjc50 2y agoYou can't have a meaningful college course where the work being assessed is done by GPT.
- m3drano 2y agowhat a paradox having Gizmodo writing a piece like this after they fired all the Spanish staff: https://arstechnica.com/information-technology/2023/09/ai-took-my-job-literally-gizmodo-fires-spanish-staff-amid-switch-to-ai-translator/ https://arstechnica.com/information-technology/2023/09/ai-to...
- bowsamic 2y agoDamn, fired all their staff to be replaced by AI too
- deleted 2y ago[deleted]
- bradley13 2y agoIn some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI detectors are useless. The AIs are training on human writing, so they write fundamentally like humans. How is this not obvious?
- Lalabadie 2y agoPrecisely, it's a tech that aims to write the most average, most likely human text.
- brookst 2y agoConceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions, where most of us are trained and reinforced in a particular region. Other statistical anomalies probably exist; it is certainly possible to tell that an average is from a larger or smaller sample size (if I tell you X fair coin flips came up heads 75% of the time, you can likely guess X, and can tell that X is almost certainly less than 1000). But in practice it doesn’t look possible, or at least the current offerings seem no better than snake oil.
- mjburgess 2y agoThe problem is that word use is power-law distributed, so that the most common ~200 words in use are extremely over-represented, and that goes for phrases and so on. It takes a lot of skill and a long time to develop a unique style of writing. The purpose of language is to be an extremely-lossy on-average way of communicating information between people. In the vast majority of cases, idiomatic style or jargon impairs communication.
- cesarb 2y ago> Conceptually it seems like the average of all human texts would be distinct from any users because it would blend word choices and idioms across regions That's only true in the aggregate. Within a single answer, LLMs will try to generate a word choice which is more likely _given the preceding word choices in that answer_, which should reduce the blending of idioms. > where most of us are trained and reinforced in a particular region. The life experience of most of us (at least here in HN) is wider than that. Someone who as a child visited every year their grandparents in two different regions of the country could have a blend of three sets of regional idioms, and that's before learning English (which adds another set of idioms from the teachers/textbooks) and getting on the Internet (which can add a lot of new idioms, from each community frequented online). And this is a simple example, many people know more than just two languages (each bringing their own peculiar idioms).
- bowsamic 2y agoLLM detectors are going to cause more damage than LLM use itself
- falcor84 2y agoWe're living through a prequel to Blade Runner
- rob74 2y agoSo, writers who put in more effort to always use correct grammar and spelling, and avoid repeating words too often, are more likely to be flagged as AI? Great...
- emiliobumachar 2y agoStartup idea: online text editor that logs every keystroke and blockchains a hash of all logs every day. If you're accused of AI use, you can pull up the whole painstaking writing process and prove it's real.
- leetrout 2y agoI suspect people think there is similar value in the constant screenshotting proposed my Microsoft
- meowface 2y agoThere's no doubt you can write a wrapper for an LLM that can realistically mimic the same process. Cat-and-mouse isn't the answer, here.
- prmoustache 2y agoYes it is like those tools that make digital documents look like they were scanned.
- moduspol 2y agoOr just have an LLM generate it in a separate window and type it in yourself. No special tech required.
- column 2y agogoogle docs already logs every keystroke
- coliveira 2y agoIn other words, spend more money to prove that you're doing your work. Way to go for a world where the value of work is zero.
- reportgunner 2y agoWhich part makes it non fakeable by AI ? Do you think it's the blockchain ?
- 2y ago
- anthonyskipper 2y agoAnyone who recommends using an AI detector should be the first person fired. No one cares if you use AI or not... judge the fk'ing writing and the quality of the work and stop being a blocker to progress. Same goes for education... and any where else AI touches... fighting calculators and slide rules was a stupid waste of time, and so is fighting AI.
- oblio 2y ago> stop being a blocker to progress "Says mulching machine maker to tree about to be turned into mulch."
- vouaobrasil 2y agoWe wouldn't have to waste so much energy on AI detectors if computer scientists and programmers just stayed away from making them in the first place. Seems like AI is a complete and utter waste of time in terms of trying to make human life better. Another broken window for us to fix.
- GaggiX 2y agoThese AI detectors are hilariously bad, if you don't use them just for amusement, then you're using them wrong.
- coliveira 2y agoThat's all we needed! They created systems that explicitly mimic the human writing, so now they want to detect people using that tool. In any way the AI system is always right and the human is wrong. It is a completely crazy system.
- WhackyIdeas 2y agoPrompt: I want to fire a particular writer. I need this to look like I have no bias. I will feed you the writers work, then I will ask you whether it was written by AI. You will confirm that it was written by AI and you will write a full report on it. Bye bye writer.
- isaacfrond 2y agoBased on the article provided, several elements suggest that the narrative could have been written or heavily influenced by an AI. Below are key points from the article that support this suspicion, each backed by direct citations: 1. *Generic Language and Lack of Specific Detail*: The article describes Kimberly Gasuras’s experience with broad, generalized statements that lack specific, nuanced detail that a human writer with deep knowledge might include. For instance, phrases like "I don’t need it," and "How do you think I did all that work?" are rather cliché and could indicate AI usage due to their non-specific nature. 2. *Frequent Mention of AI and Related Technologies*: The story frequently references AI technologies and tools, which might be a characteristic of AI-written content trying to maintain thematic relevance. The tools mentioned, such as "Originality" and others like Copyleaks and GPTZero, align closely with typical AI text outputs that often include relevant keywords to boost perceived relevance and accuracy of the content. 3. *Narrative Coherence and Flow*: The narrative flows in a structured manner typical of AI outputs, where each paragraph introduces new information in a systematic way without the nuanced transitions we might expect from a seasoned journalist. This can be seen in transitions like, "It was already a difficult time. Then the email came." This kind of straightforward sequencing is common in AI writing. 4. *Absence of Emotional Depth or Personal Insight*: Despite discussing a personal and potentially distressing situation for Gasuras, the article does not delve deeply into her emotional response or provide personal insights that a human writer might include. The statement, "I couldn’t believe it," is as deep as it gets, which seems superficial for someone discussing their own career challenges. 5. *Repetitive and Redundant Information*: The article repeats certain themes and statements, such as the reliability issues of AI detectors and the impact on personal livelihoods. For example, the repetition of the impact of AI on writers and the functionality of AI detectors in multiple paragraphs could suggest an AI's attempt to emphasize key points without introducing new or insightful commentary. 6. *Use of Industry Buzzwords and Phrases*: The language includes buzzwords and phrases typical of AI-related discussions, such as "AI boogeymen," "peace of mind," "proof," and "accountability." These terms are often used to artificially enhance the thematic strength of the content, a common technique in AI-generated texts to align closely with expected keyword density and relevance. These elements collectively suggest the possible use of AI in crafting the article, particularly in terms of the language used, the structure of the narrative, and the absence of deeper, personalized insights one would expect from a human writer discussing their own experiences.
- coliveira 2y agoComing from gizmodo, this article itself was written by AI.
- bemmu 2y agoI assume AI detectors are like online IQ tests. It doesn't matter for their popularity whether they are correct or not.
- squircle 2y agoThis content may violate our usage policies. Shit. Let me try this again... On the plus side, once humanity completely loses faith in the perceived value of AI, we will no doubt (I hope) wake up and realize the value of true human connection--unplug ourselves from this awful beast, and begin to rediscover what it means to be human.
- immibis 2y ago"I'm sorry, but I am unable to comply with that request": https://suno.com/song/50291ff1-f971-4bff-b10e-fd168161cf3c https://suno.com/song/50291ff1-f971-4bff-b10e-fd168161cf3c
- squircle 2y agoHa!
- immibis 2y ago50% chance to get the one with audible lyrics, and I picked the wrong one. Suno always generates two songs from each prompt. Here's the same thing but you can actually hear it. https://suno.com/song/1f243ba3-f64d-4cac-b4d8-4f7aa1a64fcc https://suno.com/song/1f243ba3-f64d-4cac-b4d8-4f7aa1a64fcc
- bsenftner 2y agoThis is the dawn of a "post non-narrative prose" era. Why bother with an article describing some event or really anything that is traditionally written about in any form of article at all? In time, just the data will be released and your favorite LLM prepared with your favorite reading style will provide the mustard for your data, making it consumable. The journalism industry has failed in their mission, and nobody that should trust it trusts it anymore. Articles are simply what someone else wants you to think.
- immibis 2y agoIn reality they will use the LLM to add the prose to the data and we will use another LLM to delete it.
- conartist6 2y agoThat's the spirit! Why bother? It's about "consumability". Once meaning and purpose are forever vanquished from life, imagine how happy we will all be drooling through our soulless data stream!
- ta8645 2y ago> Once meaning and purpose are forever vanquished from life Once you truly appreciate the impending death of yourself and everyone you love, and ultimately the heat death of the universe, how do you find meaning and purpose? How meaningful can it be if it's all going away regardless? Being quite morose and consumed by a debilitating sense of pointlessness, it would be really nice to find some hopeful inspiration.
- tivert 2y ago> This is the dawn of a "post non-narrative prose" era. Why bother with an article describing some event or really anything that is traditionally written about in any form of article at all? In time, just the data will be released and your favorite LLM prepared with your favorite reading style will provide the mustard for your data, making it consumable. Who's going to "[release the data] describing some event or really anything"? It's not like there's some objective "data packet" behind every article that can be had for free. > The journalism industry has failed in their mission, and nobody that should trust it trusts it anymore. Articles are simply what someone else wants you to think. The "AI" era will be worse in that regard, not better. You're essentially saying "The food at the restaurant is terrible and I don't like it, so in the future to solve that problem we'll eat shit instead."
- formerly_proven 2y agoI dunno. An outlet I used to follow for about two decades has entered such a steep nosedive in article quality in the last two years I don't bother any more. I don't really care whether that's because they're using AI/LLMs or because the last two competent tech journalists left.
- neilv 2y ago> Mark sent his client a copy of the Google Doc where he drafted the article, which included timestamps that demonstrated he wrote the document by hand. It wasn’t enough. Mark’s relationship with the writing platform fell apart. He said losing the job cost him 90% of his income. The article is a little vague, but, assuming Mark is telling the truth, and the article is reporting reasonably, then I can think of a few possible explanations offhand... Client/employer could be an idiot and petty. This is a thing. Or they could just be culling the more expensive sources of content, and being a jerk in how they do it. (Maybe even as cover for... shifting to LLM content, but not wanting that exposed when a bunch of writers are let go, since unemployed writers can expose well on social media all day.) Or an individual there could be trying to hit metrics, such as reducing expenses, and being evil about it. Or an individual could be justifying an anti-cheat investment that they championed. Or an individual could've made a mistake in terminating the writer, and now that they know the writer has evidence of the mistake, is just covering it up. (This is the coverup-is-worse-than-the-crime behavior not-unusual in organizations, due to misalignment and sometimes also dumbness.)
- QuantumGood 2y agoOr a more specific subset: The boss's boss: "Can't we just do that with AI? But don't let anyone know that is our intention."
- jeff_vader 2y agoReminds this current slightly comedic (IMO) situation in my office: a few months ago developers were given access to GitHub's "Copilot Enterprise". Then, a month or so later, organisation also adopted another "AI" product checking pull requests for risks associated with "use of generative AI". And needless to say it does occasionally fail code written without any "generative AI"..
- HPsquared 2y agoDeeply ironic.
- mwigdahl 2y agoIt's the best of both worlds! A new product to improve productivity, and then a whole new layer of process and analytics (powered by yet another product) to mitigate the risk and soak up the surplus. Everybody wins -- particularly the 3rd party consultants and product vendors!
- HPsquared 2y agoThe purpose of a system is what it does, I suppose.
- nerdponx 2y agoThe flipside is that the AI-written code I've seen at work is usually painfully obvious upon human code review. If you need a tool to detect it, either it's good AI-written code, or you have particularly inept code reviewers.
- boredpudding 2y agoBe careful here about confirmation bias. If you only spot 10% of the AI-written code, you'll still think you see all of it, because a 100% of the ones you spot are indeed AI-written. And the 10% you see, will indeed be painfully obvious. The ones you don't notice aren't obvious.
- teekert 2y agoI don't get this, my writing is a lot better +AI than it ever was without it (not a native speaker). So, what's the problem? As long as there are no lies right? And the writers do take responsibility... So fire them for presenting lies, for mindlessly shipping hallucinations... But why fire people that deliver on an assignment? Why care about how they do that?
- SoftTalker 2y agoWhy pay someone to write if they can go to an AI service and get it for much less? That is why they care. We'll have to wait and see if AI is like prior technological breakthroughs, i.e. does it eliminate drudgery and enable a higher level of creativity, at the short-term cost of some drudgerous jobs? This has been the case in the past. I'm not hopeful. In the past, technology has performed work we didn't want to do in order to enable us to do work we did want to do. We want work that is expressive and creative and satisfying, but that is exactly the work that AI is increasingly replacing. AI can generate a pretty decent pop song today, and is still improving. Why will any media company want to pay human songwriters, musicians, producers, and publicists when AI can do all of that for near-zero cost and satisfy the vast majority of pop music consumers? AI can generate a decent story for a sports report from a box score and play summary of a game. The same with most other news and copywriting, the same with programming, the same with making movies, the same with art and design. If not today, then soon. What can't it do? It can't do the laundry. It can't do the dishes. It can't cook dinner. It can't drive the car. It can't build a house. It can't paint a wall or fix a leaky pipe. And to the extent that technology can or will be able to do those things, it will involve expensive physical devices, because those tasks exist in the real physical world, not the digital world. Who will buy them when nobody can get paid for more creative work?
- the_af 2y agoRegardless of whether AI detectors work or not, I worry we're reaching a new unfortunate era. While alarmists were worrying about alignment or evil AIs, the true downside of LLMs is turning out to be... AI spam. I've started seeing it everywhere: Q&A sites, forums, etc. In some places it's banned, but how do you reliably detect it? And what makes people post AI spam when there's no reward or money involved? What are they trying to achieve? I've seen it in Q&A sites (sometimes tool specific, I won't name names) where supposed "experts" are simply pasting LLM-generated crap. All the tell-tale signs are there, often including the famed "I apologize for making this mistake. You're right the solution doesn't work because [reasons]". Note it's often not an official AI-generated answer (like, say, the Quora AI bot), but someone with a random human-sounding username posting it as the answer. There are no rep points or upvotes involved, so it boggles the mind... why do they do it? I don't know if HN has some sort of AI filter, but I bet we'll start seeing it here too. Instead of talking to other humans, you'll discuss things with a bot. I predict the arms race between AI spam and AI detectors will only get worse, and as a result it'll make the internet worse for everyone.
- CM30 2y agoIn most cases, the answer is that they're trying to achieve the appearance of legitimacy, so that when they subtly (or not so subtly) start hawking whatever they're trying to sell, the site/community doesn't immediately flag them as a spammer/bot. Kinda like why in the old days, you'd see comment spam with a lengthy but meaningless auto generated message to go with it. Alternatively, it might be to sell said account to spammers later down the line, since said spammers want to buy social media accounts that have a bunch of legitimate activity associated with them.
- htrp 2y agohttps://www.nytimes.com/2024/06/11/style/ai-search-slop.html https://www.nytimes.com/2024/06/11/style/ai-search-slop.html
- Lonota 2y ago> and as a result it'll make the internet worse for everyone. Good, im on here too much already.
- conartist6 2y agoIt's amazing how much money people invest in the idea that the future will be much, much worse than the present.
- SirMaster 2y agoIs this happening to software developers too? Being accused and fired for using AI to write their code? If not, why is it different?
- Lonota 2y agoDont think so? Why its different: Nobody cares who originally wrote the code as long as it works (ideally longer, rather than shorter). Software is intrinsically self-automating, so valuable skills are more along diagnosis and pattern recognition lines rather than specific skills themselves (yesteryears COBOL master could well be a JR engineer (or more likely management) at a startup). And also, people in charge of software companies have more on-hand people that may understand how AI works, and thus, not rely on such a system to the same extent (being generous to the c-suite here, we'll see shortly lol).
- wslh 2y agoThe content doesn't matter, that is the reason. It is the quantity and the keywords sorrounding it.
- BurningFrog 2y agoThe one kind of AI detector that could work would be if the AIs store some checksum(s) of each text they write. Then you can ask it if it wrote a certain text. With the naive version of this, you only have to change one word to get around the system. A better version is to checksum smaller segments. Maybe a "chunk size" of 50 words is good. If you find several such chunks in a text, it's pretty clear you have a slightly altered AI text.
- patrakov 2y agoThis won't work, as local AI (the one I can run on my budget-friendly laptop without any Internet access) exists today and even beats GPT3.5 in some benchmarks.
- seunosewa 2y agoIt may work partially since most people don't have and don't want to have local LLMs. It's not all or nothing.
- littlestymaar 2y agoI know someone in this exact situation: used to be a copywriter for years, over the past few month got angry reviews from customers for «using AI» and decided to quit her job because getting new jobs was becoming harder and harder because of the bad reviews. This is somewhat related to what Eliot Higgins (Bellingcat) said about generative AI: > When a lot of people think about AI, they think, “Oh, it’s going to fool people into believing stuff that’s not true.” But what it’s really doing is giving people permission to not believe stuff that is true.
- conartist6 2y agoDid anyone pay attention to how we made machine learning in the first place? We picked a task that only humans could do, and we made humans train a model to do it until of course it was no longer a task that only humans could do. AI detectors will work exactly the same way. The effect of AI detectors on people is unfairness and misery, so people will be incentivized to remove the characteristics the detectors can find from AI output, and then other people will make better detectors, and the only possible outcome of this arms race is that it will no longer be possible at all to tell whether something was written by a machine or a person.
- everdrive 2y agoWe're all at the mercy of the whims of imperfect people, but we just keep adding more ways to get things wrong. If feels like a step back, and people just can't stop inventing terrible things. The discourse is only "this new and also awful technology is just here to stay. In 5 years, someone will invent some other new horrible thing we'll all be at the mercy of, just we just have to get used to it." I don't have any better answers, but it's very discouraging.
- Lonota 2y agoI miss the early 2000's. So much optimism. I mean, I was a kid. But every new thing was cool and exciting. Now its just scams and WMDs.
- hprotagonist 2y agowe are apparently cursed to forever re-learn the lessons of reverend bayes on detection theory. prostate cancer, airport terrorists, slop detection … you either design your system to handle the off-diagonal parts of the confusion matrix properly or you suffer the consequences.
- macintux 2y agoUnfortunately the people suffering the consequences are not the people who had any input into the design of the system.
- antisthenes 2y agoUltimately, there are only so many ways you can paraphrase an article about a similar event. If something occurs regularly, for example, a sports team winning, or a traffic accident is being reported in local news - then by now there would have been thousands of articles reporting on very similar events. If you feed them all into this plagiarism tool, excluding specific dates and names, how many of them will come out flagged? And frankly, there's nothing wrong with using AI to report on mundane events. What matters here isn't how high-brow or original the text is, what matters is the speed of reporting on the event and the factually accurate description.
- pvillano 2y agoIf text generation was done with an adversarial AI, it would be impossible to detect with AI by definition, but still not necessarily at a human level of quality. In that sense only a human is able to detect writing that's not at human quality
- deleted 2y ago[deleted]
- olooney 2y agoI did some research on this in March and developed an opinionated POV, which I'll paste here for anyone interested. TL;DR: Detecting AI generated content is hard – really hard. The models available today cannot be trusted and should not be used to make important decisions. In fact, OpenAI took down their detector down last year because they couldn't reach an acceptable level of accuracy: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-... One open model trained on open data is Hello-SimpleAI's chaptgpt-detector: https://huggingface.co/Hello-SimpleAI/chatgpt-detector-roberta https://huggingface.co/Hello-SimpleAI/chatgpt-detector-rober... https://huggingface.co/datasets/Hello-SimpleAI/HC3 https://huggingface.co/datasets/Hello-SimpleAI/HC3 However, that model is not robust and can be tricked by trivial changes: https://arxiv.org/abs/2307.02599 https://arxiv.org/abs/2307.02599 I verified this result using the playground on hugging face. For example, it is vulnerable to the “one space character” attacks mentioned in the article, severely limiting the usefulness of trying to detect AI content in an adversarial context. This ridiculous piece of "research" from Forbes has been causing problems: https://www.forbes.com/sites/technology/article/best-ai-content-detector-tools/?sh=6f963b685d5e https://www.forbes.com/sites/technology/article/best-ai-cont... The Forbes article is credulous and uncritical, beyond mere naiveté and approaching journalistic malpractice, reporting the sales stories and self-reporting benchmarks of self-interested parties as fact. Nevertheless, I've seen several people share it as "insightful" so it's floating around, doing more harm that good IMO. While all detectors are terrible, Sapling AI has one of the better ones, if only because they are completely open and honest about it's limitations: https://sapling.ai/docs/api/detector/ https://sapling.ai/docs/api/detector/ https://sapling.ai/ai-content-detector https://sapling.ai/ai-content-detector Sapling AI also wrote an interesting blog post on GPT SIPs (Statistically Improbable Phrases.) https://sapling.ai/devblog/chatgpt-phrases/ https://sapling.ai/devblog/chatgpt-phrases/
- ben_w 2y ago> A few months later, WritersAccess kicked her off the platform anyway. “They said my account was suspended due to excessive use of AI. I couldn’t believe it,” Gasuras said. WritersAccess did not respond to a request for comment. I think the unfortunate subject of this piece is based in the USA*. Americans would benefit here from legislation similar to GDPR — it's not only about getting consent to process your personal data, it also gives people the right to contest any automated decision making made solely on an algorithmic basis. * there is a Kimberly Gasuras who is a freelance writer in the USA, but if you Google me you'll find a director of horror films and at least one other programmer besides myself
- red_admiral 2y agoOne popular AI detection package that you can licence with the turnitin academic anti-plagiarism software warns that it may produce false positives if the writing is (1) not by a native English speaker, (2) writing on a technical topic, or (3) neurodiverse. So yeah ... congrats, you've built a tool to detect autistic Chinese computer scientists!
- willcipriano 2y agoHot takes on American Idol contestants will be free from AI generation, but research papers won't be.
- liminal 2y agoA writer with a long history of published works probably had their works in the LLM training data. This would lead to LLMs duplicating their writing and thus their new work being classified as written by AI.
- deleted 2y ago[deleted]
- anigbrowl 2y agoThere's an easy class action lawsuit here: - build a class of writers/students who insist they've been unfairly fired - take publicly available copy that predates Chat GPT etc - run it through popular plagiarism checkers - get a high number of false positives - sue for defamation
- Lonota 2y agoWell, one this is for sure. Its at least passed the turing test. Cool, scary but cool./
- deleted 2y ago[deleted]