3 ms·
I care about optimal word choice when generating LLM texts. Because my use case is almost exclusively reading the generated text not posting it. I use LLMs to s
by throwawayffffas 2mo ago
I care about optimal word choice when generating LLM texts. Because my use case is almost exclusively reading the generated text not posting it. I use LLMs to summarize, translate and review other texts. When using LLMs in that way, as a research tool watermarking is a pointless and should not get in the way of "optimal" results.
- wpietri 2mo agoAgain, given the limits of LLMs (stochastic, rapidly changing, everything's a hallucination, widely known prose issues) I am skeptical that you really care that much about optimal prose. I could believe it's one of the things that you care about, but at a pretty low priority level. Taking you at your word, though, I'd be interested to see what you think of the watermarking technology in a blind A/B test.
- throwawayffffas 2mo agoIt's very important on translations at least. Watermarking will result in poorer results. Do they also do it with code? Do you think deliberately picking tokens that are not the highest probability in code is acceptable for the consumer?
- wpietri 2mo agoWhat's your evidence that it will result in worse translations? I'm skeptical that such a thing as a universally optimal translation exists in cases beyond the trivial. But if it does, I see no reason to think LLMs are anywhere close to it, so I think nobody will be able to tell the difference with watermarking. That's certainly true for code. LLM code is at best mediocre. There is oceans of room to subtly watermark generated code without practical impact.