4 ms·

It didn’t take, apparently.

• dofm 2mo ago
  It’s fully possible I didn’t explain it very well in the first place, but he is making a wider point. The point I made (quite briefly) is that watermarking is only feasible because for good writing it is necessary to use T>0, or the writing will never explore a more creative choice, and that at T=0 you don’t even need a watermark to spot LLM-generated text. The point he is making is consistent with this, isn’t it? Either you allow temperature to drive creativity, consistently in a way that can be influenced and analysed, or you adulterate that process for the purposes of meeting a corporate/legal directive, in a way that is proprietary and obscure. These are ethically distinct approaches, and since he disagrees with the EU objective he comes down on one side I guess. Me, I don’t care about the hypothetical enough. Not least because I think Claude writes depressingly badly and I doubt any steganographic change will enrage me less.
  • inigyou 2mo ago
    What does he think of all the other adulterations of LLMs that already happen?
  • beering 2mo ago
    No, you are jumping to conclusions about how watermarking works. This is some audiophile thinking that because your RNG is “pure”, you get text with an expansive soundstage or whatever. Intuitively this may be true or false depending on your personal prior but you’d need to show it mathematically. The overall token distribution shouldn’t change and the frequency at which you see the word “load-bearing” will remain the same.
    • dofm 2mo ago
      > This is some audiophile thinking that because your RNG is “pure”, you get text with an expansive soundstage or whatever. You are projecting that onto me, and I cannot tell you how comically poorly aimed it is.
  • wasabi991011 2mo ago
    The point he is making is not consistent with understanding how temperature influences LLM text generation, no. He repeatedly states that choosing "the best word" is the most important thing to him. I don't know how you reconcile that with creativity itself, let alone probabilistic sampling.
    • dofm 2mo ago
      I mean creativity there in the LLM sense (temperature driving more creative solutions), not the human sense, and in the context of its consistency, configurability and being amenable to analysis. The watermarking approach makes that non-reproducible, yes? Because Anthropic can and will change it as they see fit.
      • baobabKoodaa 2mo ago
        Your response seems to be missing the point completely. Gruber thinks "best" writing is produced by choosing the "best" word (highest scoring token) at each step. This is very clear from his writing.
    • eru 2mo ago
      Yes, naively understood in his sense 'best' word means that you pick the word with the maximum score, instead of sampling from the distribution. That doesn't actually give you the 'best' text in any human sense of the word. Just like playing the 'best' move in Poker without sampling leads you to lose a lot of money.
  • Imnimo 2mo ago
    >Either you allow temperature to drive creativity, consistently in a way that can be influenced and analysed, or you adulterate that process for the purposes of meeting a corporate/legal directive, in a way that is proprietary and obscure This framing does not make sense to me. What do you mean by "influenced and analysed"? How have you or anyone been influencing or analyzing the randomness behind the sampling process to create better writing? What makes the unadulterated randomness "driving creativity" but a different random choice uncreative?
    • dofm 2mo ago
      > How have you or anyone been influencing or analyzing the randomness behind the sampling process to create better writing? You're mischaracterising or misunderstanding my point, or I mangled it. I mean it is possible to analyse, control, monitor, study the impact of changing temperature on the writing, yes? The point about watermarking is that this relationship — change the temperature, see the effect — is now being adjusted by an unstated, secret process you explicitly can't control. (I gather Anthropic have recently taken away this setting anyway; that was news to me.)
      • Imnimo 2mo ago
        I don't think watermarking breaks this relationship. Watermarked text is still being sampled from the model's output distribution, and adjusting the temperature still has the same affect on that output distribution. I think a good intuition here is that watermarking is sort of like picking a specific PRNG seed. It's not changing or interfering with the temperature - we're still sampling from the model's probability distribution. But we're making it so the analog of the PRNG seed is coupled to the previous context.
  • solid_fuel 2mo ago
    He’s not making a wider point, he’s crashing out because the EU is involved. I don’t really think it is any more complicated than that - there are no technical merits to the criticism.