5 ms·
OpenAI won't watermark ChatGPT text because its users could get caught
- latexr 2y ago> On one hand, it seems like the responsible thing to do; on the other, it could hurt its bottom line. Remember that next time OpenAI and Sam Altman throw sand in your eyes with “our goal is to empower and serve humanity” or whatever they try to sell you. If they believed what they preach, the choice would’ve been obvious.
- cqqxo4zV46cp 2y agoYeah. Okay. Sure. How do you feel about community-developed / open-source models that’ll certainly do the same thing citing “freedom” and “choice” as the justification. If people are just salty about OpenAI and want to buy into the Sam Bad culture war then so be it, but trying to make sense of this contradictory backwards-engineered justification is tiring.
- latexr 2y ago> How do you feel about community-developed / open-source models that’ll certainly do the same thing citing “freedom” and “choice” as the justification. Bullshit rationalisations are bullshit no matter who they come from. Being open-source doesn’t excuse you from it. > If people are just salty about OpenAI and want to buy into the Sam Bad culture war then so be it Your mistake is assuming to know why someone you’ve never met has an opinion about something and immediately thinking the worst of it, instead of understanding individuals can think and form opinions for themselves. I was criticising Sam Altman for his Worldcoin crypto scam way before OpenAI was a worldwide phenomenon. https://www.technologyreview.com/2022/04/06/1048981/worldcoin-cryptocurrency-biometrics-web3/ https://www.technologyreview.com/2022/04/06/1048981/worldcoi... https://www.buzzfeednews.com/article/richardnieva/worldcoin-crypto-eyeball-scanning-orb-problems https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
- xcdzvyn 2y ago[dead]
- deleted 2y ago[deleted]
- methyl 2y agoI think it's just too easy to fool it, as article says > But it says techniques like rewording with another model make it “trivial to circumvention by bad actors.”
- falcor84 2y agoAm I right that "bad actors" here refers to the actual paying users, i.e. us and our children?
- lewhoo 2y agoIf our children use it to cheat in school then I suppose yes, those who do so are bad actors.
- creesch 2y agoThat and potentially asking it specifically to not follow certain patterns. I also wonder if the false positive rate in a college/school setting will be higher. Because to me, it seems that these models are often trained on college papers. Because I see a lot of the patterns you already see on those. Specifically, paragraphs that start with conjunctive adverbs and phrases. Things like; however, furthermore, moreover, in summary, in conclusion, etc. Besides, smart students that are not utterly lazy will be able to work around it anyway. They'll let chatGPT (or whatever LLM) turn out an entire paper for the contents, then rewrite it themselves. So a tool like this will only catch the most obvious cases. Meaning that in the end it only battles a symptom by effectively sweeping it under the carpet.
- wongarsu 2y agoRemember when GPT was "too dangerous" to release into the world and unless you were twitter-famous you had to apply and wait for months to even get access? Times sure have changed
- matusp 2y agoI remember when GPT-2 was "too dangerous" to release. I am confused why people still take these clown claims seriously.
- FeepingCreature 2y agoGPT-2 was never too dangerous to release, that's made up. OpenAI were trying to set a precedent to delay releases in anticipation of more dangerous models. This was and remains a good thing.
- notarobot123 2y agoMaybe "too dangerous" is mainly a PR strategy to hype up the power of the tool. It's either PR or religious ideology (or both if you drink your own Kool-Aid)
- winternewt 2y agoIt is now, I don't think it was then. OpenAI's moral compass has taken a sharp turn.
- llamaenters 2y agoNot sure about remains. The Llama is out of the bag with that one! OpenAI models aren't the best all-round anymore - there are plenty of faster models to choose from. Claude seems faster. Groq is stupid fast in serving Llama. gpt4 still has the smarts edge, but it isn't much of an edge any longer. OpenAI is iPhone, it is 2019 and people are realizing the Androids are better value and just as good
- 2y ago
- carbondating 2y agoKind of pointless when there are many open-source models that are approaching ChatGPT-level people could use instead.
- creesch 2y agoWhile the models might be there, the hardware to actually run them on is much widely spread, making it per definition less of an issue. The people that have the knowledge to actually run them is even smaller. The latter might not be much of a hurdle given tools like Ollama and jan, for now it still is a bit of a hurdle.
- j-bos 2y agoOn a personal level, I'm happy about this. Having all personal info trackable is a tiring aspect of modern life? Remember when social media didn't zero out the exif on photos and anyone could easily grab a map to your family's home? On a society level? I'm stil ok with this. Watermarks are trivially removed by the motivated and unpleasant.
- romanovcode 2y agoI'm not sure what you are saying here. Do you really believe that internally OpenAI does not track and save important info based on your inputs creating a "persona"? This is any intelligence agencies wet dream. The functionality that was introduced that would save info about you as a person to "improve" responses cross-chats basically made a whole profile out of you as a person. Oh, and what a surprise - OpenAI directors since this year are ex-CIA or have very close connections to the agency.
- j-bos 2y agoNo I believe and understand that. Doesn't mean I want more.
- tgv 2y agoIt doesn't seem trackable. And it may not be perfect, but that's no reason to discard it. At the moment, it's being used at a large scale to avoid homework by lazy students, which is nearly everyone. How much further do you want tech to erode education?
- falcor84 2y agoNot the parent, but I personally would want tech to erode education exactly to the level where students aren't asked any more to spend time on things that machines can perform trivially for us. Education should prepare us for the real world of today, rather than some make-belief role play version of a bureaucratic office from the late 19th century, where we don't have computers and the only way you have of affecting the world is by writing memos with a pencil.
- inktrail 2y agoWe solved this with a simple but novel solution that is doesn't break with rewording/paraphrasing. The problem we've run into is that teachers don't want to have the hard conversation when they get the evidence of cheating. And school leadership don't want to fix the issue, they want to put a chatbot on their resume. Sadly, I don't think OpenAI releasing this would have any effect fixing the problem. It's a problem with the people in education.
- CharlieDigital 2y agoIt's a problem with the people in education. Sounds absurd to reduce all of the thousands of school districts; millions of educators, K-12 teachers, professors, and administrators; and multitudes of viewpoints into one "ignorant" block of hapless Luddites.
- inktrail 2y agoWe've had hundreds of conversations with different schools and teachers. It's definitely a generalization, but it's pretty disheartening how common these are the views.
- CharlieDigital 2y agoWe've had hundreds of conversations with different schools and teachers Show me the paper. Let's see what the actual data looks like.
- xena 2y agoTo be fair, the Luddites weren't anti technology. They were standing against factories pricing them out of labor.
- pona-a 2y agoWhat we're seeing is a new profitable industry, with a tendency towards natural monopoly, refusing to act in the public good because its incentives conflict with it. At stake is the entire corpus of 21st century media, at risk of being drowned out in a tsunami of statistically unidentifiable spam, which has already begun and will only get worse, until nobody will even acknowledge the web post 2022 as worth reading or archiving. The solution is very, very simple: just regulate it. Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value. OpenAI's concern is that users will switch away to other vendors if they add watermarks? Well, if nobody can provide that service, OpenAI still has the lead. A portion of the market may indeed cease to exist, but in the same vein, if we had always prioritized markets over ethics, nobody would be opposed to having a blooming hitman industry. Open weights models do exist, but they require much greater investment, which many of the abusers aren't willing to make. Large models already require enough GPU power to be barely competitive with underpaid human labor, and smaller ones seem to already fall into semi-predictable patterns that may not even need watermarking. Ready-made inference APIs can too include some light watermarking, while with general purpose notebooks/VMs, the question may still be open. Still, it's all about effort to effect ratio. Sometimes inconvenience is enough to approximate practical impossibility for 80% of the users.
- xienze 2y ago> Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value. How do you watermark plain text?
- throwaway2037 2y agoWhy not post the original article that The Verge has basically summarised? https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-writing-135b755a https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-wri...
- dagw 2y agoBecause most people cannot read that article due to the paywall.
- kristjank 2y agoI do not think adding covert metadata (which this seems to be, however well-encoded) to output is a good addition to any program, regardless if it's a language model or something even more sinister. I am not so naive to believe OpenAI refuses to do it because of the goodness of their heart, but it's still a welcome benefit to me.
- briandear 2y agoPerhaps all documents should be watermarked “grammar improved by Microsoft Word” or “Animation effects provided by Keynote” or have films watermarked with “Automatic Color Correction provided by DaVinci.” It Takes a Village by Hillary Clinton: “Ghostwritten by Barbara Feinman.” If we want to watermark GPT, fine — then let’s watermark absolutely everything not directly and personally created by the claimed creator. But we’re getting into interesting legal territory here — work for hire agreements would be in jeopardy because authorship of something under work for hire is owned by then company, not the contractor. There’s also a First Amendment issue — requiring companies and individuals to watermark creative works or disclose uncredited authors amounts to compelled speech that doesn’t serve a public interest high enough to provide an exception to First Amendment protections. The unintended (or perhaps subversively intended) consequences of requiring watermarks can be astounding. Journalists could potentially be compelled to reveal sources for instance because the content they’ve created was partially provided by someone else. It’s a stretch, but then again, the gymnastics courts and prosecutors routinely employ make such scenarios plausible (albeit unlikely.)
- tbruckner 2y ago...or maybe the education of what we deem as useful work needs to change. Imagine if digital cameras were watermarking their photos because art classes refused to consider photography as a form of art.
- miga 2y ago...until they are legally required, just like producers of printers are.
- lofaszvanitt 2y agoBird of paradise courtship spectacle (dance) for the masses.
- nope1000 2y agoIf I understand the regulation correctly, they will have to add it in order to comply with the AI act: > In addition, providers will have to design systems in a way that synthetic audio, video, text and images content is marked in a machine-readable format, and detectable as artificially generated or manipulated. https://ec.europa.eu/commission/presscorner/detail/en/ip_24_4123 https://ec.europa.eu/commission/presscorner/detail/en/ip_24_...
- dsign 2y agoToday, a colleague committed ten lines of code outputted by ChatGPT, without testing it, and the systems broke. Any chance watermarking would help with this? Just kidding (about the watermarking). The AI cat is already out of the bag. I'm for regulation when it comes to AI direst threats, but students using ChatGPT to cheat is a problem that can be solved with live, supervised exams, the kind I had at school in the nineties...we couldn't even use a pocket calculator. If anything, I'm torn that today's AI systems aren't good enough to do the really serious words. Mark my words, we will have self-aware murder robots before we have AI systems able to write quantum-simulation software.
- thunky 2y ago> Today, a colleague committed ten lines of code outputted by ChatGPT, without testing it, and the systems broke Do you work at Crowdstrike? But seriously, I can only imagine how bad their code would have been without ChatGPT.
- geor9e 2y agoCurious about the idea of watermarking text. How would one go about embedding a watermark in plain text without altering its readability? Are there specific techniques or tools designed for this purpose? To me it seems like a challenging task. Given the simplicity of text what methods could ensure the watermark remains intact? Perhaps there’s a way to subtly adjust character frequencies or patterns? That said I'd love from good sources to delve.
- Elizabeth0147 2y ago[dead]
- Elizabeth0147 2y ago[dead]
- Andrewmargaret 2y ago[dead]