3 ms·
this would be a very heavy watermark application. there are many simpler methods, for example you can have a tiny windowed transformer operating on the output
by teravor 2mo ago
this would be a very heavy watermark application.
there are many simpler methods, for example you can have a tiny windowed transformer operating on the output text and all you do is alter certain words (that don't change meanings) to maximize its surprise. the tiny language model will have a special training regime to build up a somewhat unique view of the language.
we are talking about a 0.5 bit watermark here (existence). I would have zero confidence in being able to reliably remove such a watermark from pretty much any medium.
- wrsh07 2mo agoThat's actually much worse because it fundamentally changes the output, whereas this doesn't change the output, it just changed the prng
- cma 2mo agoWhen you tell the AI: copy this function to here, a small window rewriter would mean it just corrupts and changes it instead of moving it. And even it's own tool use would have some small window dumb model changing the tool calls based on what it thinks are synonyms? The Aaronson approach is much better than this, it's essentially like changing out the random seed for the sampling parts that were already random. For an operation like recall of previous text, the tight logits that result still keep it doing that close to deterministically. For something it creates itself, with more spread out probability mass, it gets watermarked.