4 ms·
What we're seeing is a new profitable industry, with a tendency towards natural monopoly, refusing to act in the public good because its incentives conflict wit
by pona-a 2y ago
What we're seeing is a new profitable industry, with a tendency towards natural monopoly, refusing to act in the public good because its incentives conflict with it. At stake is the entire corpus of 21st century media, at risk of being drowned out in a tsunami of statistically unidentifiable spam, which has already begun and will only get worse, until nobody will even acknowledge the web post 2022 as worth reading or archiving.
The solution is very, very simple: just regulate it. Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value. OpenAI's concern is that users will switch away to other vendors if they add watermarks? Well, if nobody can provide that service, OpenAI still has the lead. A portion of the market may indeed cease to exist, but in the same vein, if we had always prioritized markets over ethics, nobody would be opposed to having a blooming hitman industry.
Open weights models do exist, but they require much greater investment, which many of the abusers aren't willing to make. Large models already require enough GPU power to be barely competitive with underpaid human labor, and smaller ones seem to already fall into semi-predictable patterns that may not even need watermarking. Ready-made inference APIs can too include some light watermarking, while with general purpose notebooks/VMs, the question may still be open.
Still, it's all about effort to effect ratio. Sometimes inconvenience is enough to approximate practical impossibility for 80% of the users.
- xienze 2y ago> Force all companies training LLMs to add some method of watermarking with a mean error rate below a set value. How do you watermark plain text?
- exe34 2y ago"I seem to do fine for a stretch, but at the of the sentence I say the wrong cranberry."
- xienze 2y agoSo deliberately bork the output in such a way that users lose all confidence in the product?
- lores 2y agoYou should read the original article... OpenAI say it doesn't downgrade quality.
- exe34 2y agono, you modify the output probability so that you sample in a deterministic pseudo-probabilistic way - i.e. save the seed, and insert low SNR bias into the sampling. you can recover the bias afterwards and prove you generated the sequence. my example was just a reference to Jarvis in the avengers.
- slfnflctd 2y agoThis is a fairly obvious initial question which I assume nearly everyone who doesn't already have a rough answer in mind would ask, so I'm happy to report that it's fortunately quite clearly addressed in TFA, and in fact makes up a significant part of the (not very long) piece.
- deleted 2y ago[deleted]
- preciz 2y agoYou are DEEPLY WRONG on all issues you mentioned. Open weight models do not require great investment. In fact I can run them on my 400 EUR computer. Also why you want to regulate text output from machines in the name of "public good"? That's insanity.
- lewhoo 2y agoWhy exactly is it insane ? To reliably differentiate (let's assume it's possible for the sake of argument) between "you made this" and "you didn't make this" or at least "a human made this" seems to carry mostly (if not only) benefits.
- fragmede 2y agothe problem is your parenthetical - it's not possible, so attempting to do so isn't actually really possible. what's worse than a watermark? one that doesn't actually work.
- lores 2y agoOpen AI literally said they have a semi-resilient method with 99.9% accuracy. It will become full-resilient for practical purposes if all LLMs implement something similar.
- jeanlucas 2y ago> Open AI literally said they have a semi-resilient method with 99.9% accuracy. They also said many other things that never happened. And they never showed it. I bet $100 they do not have a semi-resilient method with 99.9% accuracy, especially with all the evolving issues around "human vs computer" made content. I bet you also the `semi-` in the beginning leaves a lot of room for interpretation and they are not releasing this for more reasons than "our model is too good".
- lores 2y ago
- sva_ 2y ago> Open weights models do exist, but they require much greater investment, which many of the abusers aren't willing to make. I think the biggest concern are state actors who have no problem spending money on some gpus. I don't think it is feasible to watermark open weights LLM outputs.
- raxxorraxor 2y agoI don't think this is a good idea. Also, how would you watermark text exactly? Hash all generated texts that were ever put out? Not feasible I believe and easily circumvented. I can run a specialized AI trained to a certain domain on my laptop that reaches GPT levels and sometimes even beyond that. The work needed here is exactly what malicious actors would also invest and they would have competitive advantage. I believe we should watch the developments, but I don't believe regulation is warranted yet.