5 ms·

If they only collect the data for analysis I guess this is fine (they already get way more sensitive data from users anyways, so if privacy is your concern you'

If they only collect the data for analysis I guess this is fine (they already get way more sensitive data from users anyways, so if privacy is your concern you've made the mistake many steps ago). The much more interesting question is if they directly act on this data in their API. For example by rate-limiting, compute-limiting or rerouting to weaker models. That might even be legally questionable. I would really like to see this as a follow-up analysis, but I guess it is way more difficult and will also cost quite a bit in tokens.

bakugo 3mo ago
I've heard that it was possible to trigger really obvious output poisoning on Fable with something as basic as asking the model to think outside of its built-in hidden thinking delimiters. This watermark may trigger a similar mechanism.
SubiculumCode 3mo ago
Would it be legally questionable, or actually complying with U.S. export law?
- sigmoid10 3mo ago
  I'm thinking more of EULAs. Even if Anthropic somehow wedges this into their TOS, it might still be illegal. For example, in many US states this could potentially be classified as consumer fraud. You can't just sell one thing and then secretly and intransparently turn it into something else before shipping it. And in the EU it might violate GDPR too.
krupan 3mo ago
"If they only collect the data for analysis I guess this is fine" I think you missed the memo on how foolish this attitude is. It came out around the time Edward Snowden made his discoveries at the NSA public. I suggest you look into it
- sigmoid10 3mo ago
  As I said above, if you are worried about privacy while hooking up Claude Code, you need to reevaluate your understanding of this technology.
  - krupan 3mo ago
    True. A lot of us missed the memo, or forgot it because of our amazement of LLMs
    - sigmoid10 3mo ago
      This is not some fundamental problem. There are amazing options that don't suffer this downside. Most people are just too lazy and want something that works out of the box, so they accept or ignore the inherent risks of tools like Claude Code or Codex. But you just have to invest a little bit of research into what you need and which open model and harness suits your requirements. Then you can have your cake and eat it too. Long term this will be the standard anyways, it's only difficult while the frontier is moving so fast and aimed at ridiculous IPOs.

bakugo 3mo ago

I've heard that it was possible to trigger really obvious output poisoning on Fable with something as basic as asking the model to think outside of its built-in hidden thinking delimiters. This watermark may trigger a similar mechanism.

SubiculumCode 3mo ago

Would it be legally questionable, or actually complying with U.S. export law?

sigmoid10 3mo ago
I'm thinking more of EULAs. Even if Anthropic somehow wedges this into their TOS, it might still be illegal. For example, in many US states this could potentially be classified as consumer fraud. You can't just sell one thing and then secretly and intransparently turn it into something else before shipping it. And in the EU it might violate GDPR too.

krupan 3mo ago

"If they only collect the data for analysis I guess this is fine" I think you missed the memo on how foolish this attitude is. It came out around the time Edward Snowden made his discoveries at the NSA public. I suggest you look into it

sigmoid10 3mo ago
As I said above, if you are worried about privacy while hooking up Claude Code, you need to reevaluate your understanding of this technology.
- krupan 3mo ago
  True. A lot of us missed the memo, or forgot it because of our amazement of LLMs
  - sigmoid10 3mo ago
    This is not some fundamental problem. There are amazing options that don't suffer this downside. Most people are just too lazy and want something that works out of the box, so they accept or ignore the inherent risks of tools like Claude Code or Codex. But you just have to invest a little bit of research into what you need and which open model and harness suits your requirements. Then you can have your cake and eat it too. Long term this will be the standard anyways, it's only difficult while the frontier is moving so fast and aimed at ridiculous IPOs.