3 ms·

EU regulations are going to force it, friend. The tsunami is coming and cannot be stopped and Anthropic has jack to do with it.

HarHarVeryFunny 2mo ago
I wonder if that is entirely true. It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on it ("Habsburg AI"). In the case of Anthropic it would also be entirely unsurprising if they've been lobbying the government to force everyone to do something in their (Anthropic's) own best interest.
- kergonath 2mo ago
  > It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on it It’s also in their interest to demonstrate that they can be trusted and to show that they at least pay lip service to limit the obvious downsides of the tools they are selling. The use cases they sell to mainstream audiences are not affected by detection tools. The point of having a LLM do the work for you is that the work is done, and reliably. It does not matter if it is done by a LLM, and most of the time it is obvious anyway.
  - 2_foos_in_a_bar 2mo ago
    The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases. The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese. So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way. In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe. The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table. This is all conjecture at this point though.
    - kergonath 2mo ago
      I’m sorry, this is going to be a bit long but you made good points. > The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases. I am not sure how practical that would be. The cat’s already out of the bag and they won’t prevent companies in the whole world from releasing open weight models. Playing catch up by distilling flagship models is also relatively cheap; we’d see smaller companies setting up shop in friendly regimes. And I don’t see any appetite to go full child porn and criminalise the possession of a LLM. So we’d end up with a similar situation as with illegal downloads, i.e., everyone will do it and nobody will care. > The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese. They could forbid using LLM for hacking, but hacking is already illegal. They could make it a factor when determining punishment, but I don’t think that would work terribly well. Most of the dangerous stuff we can do with LLMs is already illegal, or should become so. Things like propaganda, identity theft, harassment, scams. We need enforcement with teeth on these, not pointless feel-good legislation. Again, there are parallels with cryptocurrencies and torrenting software. These things have illegal uses, but it’s also really difficult to make them illegal, at least in semi-functioning democracies. > So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way. I don’t know that mainstream audiences are really against LLMs. They are mostly against AI in a nebulous sense, but even non-technical people use ChatGPT or equivalent. I think that the critical mass is already there and the tools are convenient enough that they couldn’t outlaw them without a massive uproar. Detection tools don’t seem all that relevant to mainstream audiences’ use of LLMs. AI companies will sell this as a safeguard against misuse, and everyone will be happy about it. The politicians will say they accomplished something, the AI companies will slowly turn public opinion, and the public will have shiny toys. > The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table. I don’t know. They can open possibilities that we don’t necessarily consider. One example I have is a friend who is getting his house refurbished. He’s not an engineer or a material scientist. He does not have enough free time to read thoroughly on the many subjects involved. With a decent LLM, he could untangle the technical documents sent by the architect and the contractors to really understand what was going on and be involved, rather than passively follow the architect’s advice. For starters, the LLM was very useful in finding issues in the quotes he received when he was looking for an architect. Those were long, technical documents, with no really standardised structure and full of jargon. I don’t think that person is going to want to stop using LLMs now. Many people are having this sort of moments right now. > In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe. Maybe. But i don’t believe the EU is well equipped for that. Labour laws are largely local and different in each member state. The EU regulations are basically the common denominator, ore or less, and it is easy to see why: for regulations to get adopted, they need a strong enough majority in the Commission, in the Parliament, and in the Council. It is very difficult to get anything controversial that affect the sovereignty of member states passed. The angle of the current AI regulations is that they set the rules for the single market, which is where the EU is the most legitimate. It is difficult to see market angle for the effect of AI in labour, and I think enough member states would be keen to kill the project. Also, there are many influences at play, but the EU is fundamentally an economically liberal institution. It very rarely goes in the direction that reduces economic activity. Look at how clumsy it is at fighting against cheap Chinese imports. I don’t think the institutions themselves would really want to make AI illegal. Companies have too much to lose.

HarHarVeryFunny 2mo ago

I wonder if that is entirely true. It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on it ("Habsburg AI"). In the case of Anthropic it would also be entirely unsurprising if they've been lobbying the government to force everyone to do something in their (Anthropic's) own best interest.

kergonath 2mo ago
> It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on it It’s also in their interest to demonstrate that they can be trusted and to show that they at least pay lip service to limit the obvious downsides of the tools they are selling. The use cases they sell to mainstream audiences are not affected by detection tools. The point of having a LLM do the work for you is that the work is done, and reliably. It does not matter if it is done by a LLM, and most of the time it is obvious anyway.
- 2_foos_in_a_bar 2mo ago
  The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases. The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese. So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way. In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe. The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table. This is all conjecture at this point though.
  - kergonath 2mo ago
    I’m sorry, this is going to be a bit long but you made good points. > The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases. I am not sure how practical that would be. The cat’s already out of the bag and they won’t prevent companies in the whole world from releasing open weight models. Playing catch up by distilling flagship models is also relatively cheap; we’d see smaller companies setting up shop in friendly regimes. And I don’t see any appetite to go full child porn and criminalise the possession of a LLM. So we’d end up with a similar situation as with illegal downloads, i.e., everyone will do it and nobody will care. > The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese. They could forbid using LLM for hacking, but hacking is already illegal. They could make it a factor when determining punishment, but I don’t think that would work terribly well. Most of the dangerous stuff we can do with LLMs is already illegal, or should become so. Things like propaganda, identity theft, harassment, scams. We need enforcement with teeth on these, not pointless feel-good legislation. Again, there are parallels with cryptocurrencies and torrenting software. These things have illegal uses, but it’s also really difficult to make them illegal, at least in semi-functioning democracies. > So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way. I don’t know that mainstream audiences are really against LLMs. They are mostly against AI in a nebulous sense, but even non-technical people use ChatGPT or equivalent. I think that the critical mass is already there and the tools are convenient enough that they couldn’t outlaw them without a massive uproar. Detection tools don’t seem all that relevant to mainstream audiences’ use of LLMs. AI companies will sell this as a safeguard against misuse, and everyone will be happy about it. The politicians will say they accomplished something, the AI companies will slowly turn public opinion, and the public will have shiny toys. > The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table. I don’t know. They can open possibilities that we don’t necessarily consider. One example I have is a friend who is getting his house refurbished. He’s not an engineer or a material scientist. He does not have enough free time to read thoroughly on the many subjects involved. With a decent LLM, he could untangle the technical documents sent by the architect and the contractors to really understand what was going on and be involved, rather than passively follow the architect’s advice. For starters, the LLM was very useful in finding issues in the quotes he received when he was looking for an architect. Those were long, technical documents, with no really standardised structure and full of jargon. I don’t think that person is going to want to stop using LLMs now. Many people are having this sort of moments right now. > In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe. Maybe. But i don’t believe the EU is well equipped for that. Labour laws are largely local and different in each member state. The EU regulations are basically the common denominator, ore or less, and it is easy to see why: for regulations to get adopted, they need a strong enough majority in the Commission, in the Parliament, and in the Council. It is very difficult to get anything controversial that affect the sovereignty of member states passed. The angle of the current AI regulations is that they set the rules for the single market, which is where the EU is the most legitimate. It is difficult to see market angle for the effect of AI in labour, and I think enough member states would be keen to kill the project. Also, there are many influences at play, but the EU is fundamentally an economically liberal institution. It very rarely goes in the direction that reduces economic activity. Look at how clumsy it is at fighting against cheap Chinese imports. I don’t think the institutions themselves would really want to make AI illegal. Companies have too much to lose.