3 ms·

Anthropic shares details about how Claude's new watermarks will work

9 points by sbulaev 2mo ago

treetalker 2mo ago
It seems to me that a model provider who can watermark generated text (unbeknownst to the users) could also specifically watermark generated text prompted by particular users, thus enabling tracking and, consequently, harassment and control of those users and populations as a whole. Bad for anonymity and for free speech. I hope to be corrected by someone who knows the tech much better than I do.
- arlcode 2mo ago
  That is a concerning possibility but I wonder what the threat model is. They have all that information already and could simply classify and pass it on without the need to encode it with a probability.
  - xscott 2mo ago
    One possibility is that you have Claude write something that would not get you in trouble and concatenate that with something you wrote by hand that would. Your signature from the safe material is now associated with your content on the dangerous part, and then the brownshirts come knocking on your door.
n4pw01f 2mo ago
If you look at how this works under the hood, “when two words that don’t necessarily matter to the context, it will pick one word over the other and a pattern emerges” And that pattern will be reverse engineered in weeks by a startup And a downstream agent produced by said startup will be built to roll those dice again and defeat the watermarking And a the LLM vendors will combat this And a cottage industry shall be born anew
- cyanydeez 2mo ago
  To be fair, the circle jerk in virtual space is limitless. Its just unbounded reach arounds, turtle cum all the way down, human detritus as far as the eye can see. The same VC capital will chase it because god forbid we fail at real challenges; no, we must build the torment nexus.
WalterGR 2mo ago
From yesterday, 68 comments on the original source: https://news.ycombinator.com/item?id=49303350 https://news.ycombinator.com/item?id=49303350
jannw 2mo ago
you can remove AI company invisible text watermarks at www.demarkify.com - and also perform other tonal text changes like removing AI lexical tells

9 points by sbulaev 2mo ago

It seems to me that a model provider who can watermark generated text (unbeknownst to the users) could also specifically watermark generated text prompted by particular users, thus enabling tracking and, consequently, harassment and control of those users and populations as a whole. Bad for anonymity and for free speech. I hope to be corrected by someone who knows the tech much better than I do.

arlcode 2mo ago
That is a concerning possibility but I wonder what the threat model is. They have all that information already and could simply classify and pass it on without the need to encode it with a probability.
- xscott 2mo ago
  One possibility is that you have Claude write something that would not get you in trouble and concatenate that with something you wrote by hand that would. Your signature from the safe material is now associated with your content on the dangerous part, and then the brownshirts come knocking on your door.

n4pw01f 2mo ago

If you look at how this works under the hood, “when two words that don’t necessarily matter to the context, it will pick one word over the other and a pattern emerges” And that pattern will be reverse engineered in weeks by a startup And a downstream agent produced by said startup will be built to roll those dice again and defeat the watermarking And a the LLM vendors will combat this And a cottage industry shall be born anew

cyanydeez 2mo ago
To be fair, the circle jerk in virtual space is limitless. Its just unbounded reach arounds, turtle cum all the way down, human detritus as far as the eye can see. The same VC capital will chase it because god forbid we fail at real challenges; no, we must build the torment nexus.

WalterGR 2mo ago

From yesterday, 68 comments on the original source: https://news.ycombinator.com/item?id=49303350 https://news.ycombinator.com/item?id=49303350

jannw 2mo ago

you can remove AI company invisible text watermarks at www.demarkify.com - and also perform other tonal text changes like removing AI lexical tells