4 ms·

I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it. Currently I can recognize AI text becaus

I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it. Currently I can recognize AI text because I read thousands of ai generated text. I know that 110% of yahoo finance news is generated. I don't want to read an AI generated personal blog, but if I do what's the problem really? Other than the companies distinguishing AI text for getting better training data, how do people benefit from watermarked text exactly?

tcdent 2mo ago
This feature is born out of the same government that created the cookie laws, so it makes sense to me that it would follow similar politically-valuable but practically-questionable set of beliefs. Really, Anthropic (and the other labs that follow) are just trying to satisfy the requirements of the law so they can continue to serve the EU. Wether it's actually effective is something else entirely.
- Lorkki 2mo ago
  The cookie laws and GDPR are hampered by malicious compliance, sure, but also outright intentional non-compliance. What's needed is more efficient oversight and enforcement.
  - theoreticalmal 2mo ago
    How would more efficient compliance help the constant annoying pop ups?
    - setopt 2mo ago
      According to the rules, accepting or refusing cookies should be equally difficult. Making them both difficult makes people not use your site, so the incentive should be to make both options easy if you follow the rules. Malicious compliance is to make cookie acceptance much easier than refusal. Lack of oversight doesn’t punish this. Now, the incentive is to make life hard for people who decline cookies. That results in more annoying pop-ups.
      - ffsm8 2mo ago
        also, session cookies for eg login and error tracing etc dont actually need a banner. you only need to use it if youre participating in the internet stalking behaviour by integrating eg google analytics and similar integrations
      - kevincox 2mo ago
        > Malicious compliance is to make cookie acceptance much easier than refusal. That isn't malicious compliance, it is just non-compliance. If the equally difficult rule wasn't there then it would be malicious compliance.
    - ForHackernews 2mo ago
      You are not required to present an annoying banner to the user. You could simply honor user-agents' DO NOT TRACK header and silently avoid setting tracking cookies and that would comply with the law.
avaer 2mo ago
None of this is science or evidence based, it's political theater. The labs do this because the government has made it clear it will intervene if they don't say the line. I suspect most of the people at the labs think AI watermarking is a retarded approach. The people that make the laws and the people complaining about AI don't know or care about the reality of the situation nearly as much as they care about being re-elected and feeling good about their social posture.
- mschuster91 2mo ago
  > None of this is science or evidence based, it's political theater. The labs do this because the government has made it clear it will intervene if they don't say the line. I suspect most of the people at the labs think AI watermarking is a retarded approach. Well, the negative effects of AI are already visible, and I'm not talking about job losses here. I'm talking about AI hallucination, about mass generated spam, about AI-assisted scamming (apparently Indian scammers are shifting to use AI systems [1]) and about political manipulation. Governments usually are slow to react, but eventually they do react, when a technology poses more issues than it is worth. [1] https://timesofindia.indiatimes.com/gadgets-news/new-ai-scam-is-targeting-indians-all-the-details/articleshow/99914367.cms https://timesofindia.indiatimes.com/gadgets-news/new-ai-scam...
9dev 2mo ago
I for one value the transparency of being able to tell human-written content apart, especially when moving into a time where we’re suddenly questioning an AI agent's motivations. If that is ingrained so deeply into these systems they cannot rip it out without us noticing, there’s at least one more safeguard. I know how ludicrous this sounds, but you can’t deny AI development is speeding up to scary levels. Additionally, think of cases like paying a lawyer or an expert for an extensive report or opinion on something. Wouldn’t you want to know if that is actually their carefully assembled professional assessment rather than the output of an LLM prompt?
- josephg 2mo ago
  Me too! And a detection system that works 50%-90% of the time is better in a lot of cases than no automated detection at all. For example, universities need to make cheating with chatgpt risky for students. This article says any AI watermarking can be defeated trivially, by running the text through a tool which strips weird Unicode characters (fine) and then runs it through an LLM which replaces all words with similar words (whaaattt?). Running text through another - probably much lower quality llm which scrambles the words you use would make slop even sloppier. And it’s another whole step you have to know to do. A great many people who use LLMs to avoid doing work will not know about these extra steps, or not make use of these sort of tools.
  - sib 2mo ago
    >> And a detection system that works 50%-90% of the time is better in a lot of cases than no automated detection at all. For example, universities need to make cheating with chatgpt risky for students. I would say that's not necessarily the case unless there are zero false positives. In fact, your university situation is exactly where a detection system that works 50-90% of the time would be a nightmare if a meaningful share of the 10-50% errors were false positives.
    - josephg 2mo ago
      A watermarking system like this should have an incredibly low false positive rate. Orders of magnitude lower than the current crop of AI detector tools.
nonethewiser 2mo ago
>I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it. Does it sound good? Either anthropic does not tell anyone the signal and only they can verify the text. Or they share the signal and everyone can verify it, which means everyone can bypass it and its just an inconvenience.
- throwaway27448 2mo ago
  Are they not offering an API to perform this check? Would be pretty simple to use any open model, even a weak local one, to basically tweak the text until it doesn't flag.
- nradov 2mo ago
  There are limits to what you can hide in text. If there is a signal then obviously independent cryptographers will be able to find it.
deleted 2mo ago
[deleted]
Boltgolt 2mo ago
> I feel like this is going to end up being like cookie laws Companies will implement it in the worst way possible? Nowhere in the cookie laws does it say you need to add a banner, you just can't spy on users without their consent.
- LoganDark 2mo ago
  The point of the banner has always been to malicious-compliance the users into being annoyed by the laws. Same as when one of the food delivery apps responded to a fair wages regulation by adding a special line item to every single order with help text that directed the user to appeal the regulation, it's just cookie banners usually don't say it explicitly
  - Joeri 2mo ago
    The web’s ad markets bake in a loss of privacy by design, and so every website that wants to participate in those ad markets has to dupe as many users as possible into handing away their privacy. Couple that with declining web ad revenues because of the move to mobile apps, and you get a web filled with malicious compliance cookie banners. Had google offered a privacy-preserving ad model (basing the ads off the content of the page instead of who the user is) things would be very different.
ifh-hn 2mo ago
I would income education institutions might benefit from being able to distinguish between AI and human generated text. The people who benefit, in the long run, are the students.
UltraSane 2mo ago
You can't think of the benefit of having LLM output watermarked? So many lazy students are going to get caught by this.
- razakel 2mo ago
  Cheating was obviously going to happen when education became a business.
theshrike79 2mo ago
There are stories of students literally copy-pasting LLM answers to school assignments with prompts and all. This will most likely bring a sift end to at least the low hanging fruit.
- conartist6 2mo ago
  Should open up some jobs currently held by unscrupulous people as well ; )
- mFixman 2mo ago
  Removing AI watermarks is still easier than doing a school assignment. The only way to prevent AI cheating is to make the assignments in class in pen and paper. To be honest, I don't understand why schools moved away from that in the first place.
  - graemep 2mo ago
    > I don't understand why schools moved away from that in the first place. Cost and convenience. Being able to work on computers does help people with bad handwriting, which includes some disabilities.
    - mFixman 2mo ago
      And now instead of teaching disabled students how to improve their handwriting to improve their lives regardless of their circumstances, we lowered the bar and made all students have bad handwriting.
      - graemep 2mo ago
        I am talking about disabilities which affect things like motor control which means that they cannot improve their handwriting very much. How does good handwriting improve people's live significantly? I know some people who prefer handwritten notes and I think there is research that says they can be more effective when learning, but most adults very rarely write more than a few lines by hand.

tcdent 2mo ago

This feature is born out of the same government that created the cookie laws, so it makes sense to me that it would follow similar politically-valuable but practically-questionable set of beliefs. Really, Anthropic (and the other labs that follow) are just trying to satisfy the requirements of the law so they can continue to serve the EU. Wether it's actually effective is something else entirely.

Lorkki 2mo ago
The cookie laws and GDPR are hampered by malicious compliance, sure, but also outright intentional non-compliance. What's needed is more efficient oversight and enforcement.
- theoreticalmal 2mo ago
  How would more efficient compliance help the constant annoying pop ups?
  - setopt 2mo ago
    According to the rules, accepting or refusing cookies should be equally difficult. Making them both difficult makes people not use your site, so the incentive should be to make both options easy if you follow the rules. Malicious compliance is to make cookie acceptance much easier than refusal. Lack of oversight doesn’t punish this. Now, the incentive is to make life hard for people who decline cookies. That results in more annoying pop-ups.
    - ffsm8 2mo ago
      also, session cookies for eg login and error tracing etc dont actually need a banner. you only need to use it if youre participating in the internet stalking behaviour by integrating eg google analytics and similar integrations
    - kevincox 2mo ago
      > Malicious compliance is to make cookie acceptance much easier than refusal. That isn't malicious compliance, it is just non-compliance. If the equally difficult rule wasn't there then it would be malicious compliance.
  - ForHackernews 2mo ago
    You are not required to present an annoying banner to the user. You could simply honor user-agents' DO NOT TRACK header and silently avoid setting tracking cookies and that would comply with the law.

avaer 2mo ago

None of this is science or evidence based, it's political theater. The labs do this because the government has made it clear it will intervene if they don't say the line. I suspect most of the people at the labs think AI watermarking is a retarded approach. The people that make the laws and the people complaining about AI don't know or care about the reality of the situation nearly as much as they care about being re-elected and feeling good about their social posture.

mschuster91 2mo ago
> None of this is science or evidence based, it's political theater. The labs do this because the government has made it clear it will intervene if they don't say the line. I suspect most of the people at the labs think AI watermarking is a retarded approach. Well, the negative effects of AI are already visible, and I'm not talking about job losses here. I'm talking about AI hallucination, about mass generated spam, about AI-assisted scamming (apparently Indian scammers are shifting to use AI systems [1]) and about political manipulation. Governments usually are slow to react, but eventually they do react, when a technology poses more issues than it is worth. [1] https://timesofindia.indiatimes.com/gadgets-news/new-ai-scam-is-targeting-indians-all-the-details/articleshow/99914367.cms https://timesofindia.indiatimes.com/gadgets-news/new-ai-scam...

9dev 2mo ago

I for one value the transparency of being able to tell human-written content apart, especially when moving into a time where we’re suddenly questioning an AI agent's motivations. If that is ingrained so deeply into these systems they cannot rip it out without us noticing, there’s at least one more safeguard. I know how ludicrous this sounds, but you can’t deny AI development is speeding up to scary levels. Additionally, think of cases like paying a lawyer or an expert for an extensive report or opinion on something. Wouldn’t you want to know if that is actually their carefully assembled professional assessment rather than the output of an LLM prompt?

josephg 2mo ago
Me too! And a detection system that works 50%-90% of the time is better in a lot of cases than no automated detection at all. For example, universities need to make cheating with chatgpt risky for students. This article says any AI watermarking can be defeated trivially, by running the text through a tool which strips weird Unicode characters (fine) and then runs it through an LLM which replaces all words with similar words (whaaattt?). Running text through another - probably much lower quality llm which scrambles the words you use would make slop even sloppier. And it’s another whole step you have to know to do. A great many people who use LLMs to avoid doing work will not know about these extra steps, or not make use of these sort of tools.
- sib 2mo ago
  >> And a detection system that works 50%-90% of the time is better in a lot of cases than no automated detection at all. For example, universities need to make cheating with chatgpt risky for students. I would say that's not necessarily the case unless there are zero false positives. In fact, your university situation is exactly where a detection system that works 50-90% of the time would be a nightmare if a meaningful share of the 10-50% errors were false positives.
  - josephg 2mo ago
    A watermarking system like this should have an incredibly low false positive rate. Orders of magnitude lower than the current crop of AI detector tools.

nonethewiser 2mo ago

>I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it. Does it sound good? Either anthropic does not tell anyone the signal and only they can verify the text. Or they share the signal and everyone can verify it, which means everyone can bypass it and its just an inconvenience.

throwaway27448 2mo ago
Are they not offering an API to perform this check? Would be pretty simple to use any open model, even a weak local one, to basically tweak the text until it doesn't flag.
nradov 2mo ago
There are limits to what you can hide in text. If there is a signal then obviously independent cryptographers will be able to find it.

deleted 2mo ago

[deleted]

Boltgolt 2mo ago

> I feel like this is going to end up being like cookie laws Companies will implement it in the worst way possible? Nowhere in the cookie laws does it say you need to add a banner, you just can't spy on users without their consent.

LoganDark 2mo ago
The point of the banner has always been to malicious-compliance the users into being annoyed by the laws. Same as when one of the food delivery apps responded to a fair wages regulation by adding a special line item to every single order with help text that directed the user to appeal the regulation, it's just cookie banners usually don't say it explicitly
- Joeri 2mo ago
  The web’s ad markets bake in a loss of privacy by design, and so every website that wants to participate in those ad markets has to dupe as many users as possible into handing away their privacy. Couple that with declining web ad revenues because of the move to mobile apps, and you get a web filled with malicious compliance cookie banners. Had google offered a privacy-preserving ad model (basing the ads off the content of the page instead of who the user is) things would be very different.

ifh-hn 2mo ago

I would income education institutions might benefit from being able to distinguish between AI and human generated text. The people who benefit, in the long run, are the students.

UltraSane 2mo ago

You can't think of the benefit of having LLM output watermarked? So many lazy students are going to get caught by this.

razakel 2mo ago
Cheating was obviously going to happen when education became a business.

theshrike79 2mo ago

There are stories of students literally copy-pasting LLM answers to school assignments with prompts and all. This will most likely bring a sift end to at least the low hanging fruit.

conartist6 2mo ago
Should open up some jobs currently held by unscrupulous people as well ; )
mFixman 2mo ago
Removing AI watermarks is still easier than doing a school assignment. The only way to prevent AI cheating is to make the assignments in class in pen and paper. To be honest, I don't understand why schools moved away from that in the first place.
- graemep 2mo ago
  > I don't understand why schools moved away from that in the first place. Cost and convenience. Being able to work on computers does help people with bad handwriting, which includes some disabilities.
  - mFixman 2mo ago
    And now instead of teaching disabled students how to improve their handwriting to improve their lives regardless of their circumstances, we lowered the bar and made all students have bad handwriting.
    - graemep 2mo ago
      I am talking about disabilities which affect things like motor control which means that they cannot improve their handwriting very much. How does good handwriting improve people's live significantly? I know some people who prefer handwritten notes and I think there is research that says they can be more effective when learning, but most adults very rarely write more than a few lines by hand.