5 ms·

The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothi

The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated. Creating models that don't tag their output as such would be banned. So far nothing strange about the law. The obvious problem with the law is that you need to require even screenshotting an image AI and reposting it online without the made-with-ai metadata to be outlawed. And that would be an absolute mess to enforce, at least for images. But most importantly: whatever we do in this space has to be made on the assumption that we can't really influence what "bad actors" do. Yes being responsible means leaving money on the table. So money has to be left on the table, for - erm - less responsible nations to pick up. That's just a fact.

worldsayshi 2y ago
Any law that tries to categorize non-trustworthy content seems doomed to fail. We need to find better ways to communicate trustworthiness, not the other way around. (And I'm not sure adding more laws can help here.)
- alkonaut 2y ago
  No I don't think technical means will work fully either. But the thing about these regulations is that you can basically cover the 99% case by just thinking about the 5 largest players in the field, be it regulation for social media, AI or whatever. It doesn't matter that the law has loopholes or that some players aren't affected at all. Regulation that helps somewhat in a large majority of cases is massive.
  - dangitman 2y ago
    [dead]
arder 2y ago
I think the most acheivable way of having some verification of AI images is simply for the AI generators to store finger prints of every image they generate. That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you". There's already technology for that sort of thing in the world (content ID from youtube, CSAM detection etc.). It's obviously not perfect, but could help and doesn't have the enormous side effects of trying to lock down all image generation.
- Someone 2y ago
  > That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you". Firstly, if you want to know an image isn’t generated, you’d have to go to every ‘whoever’ in the world, including companies that no longer exist. Secondly, if you ask evil.com that question, you would have to trust them to answer honestly for both all images they generated and images they didn’t generate (claiming real pictures were generated by you can probably be career-ending for a politician) This is worse than https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF: “Program testing can be used to show the presence of bugs, but never to show their absence!”. You can neither show an image is real nor that it is fake.
- kortex 2y ago
  What's to stop someone from downloading an open source model, running it themselves, and either just not sharing the hashes, subtly corrupting the hash algo so that it gives a false negative, etc? Also you need perceptual hashing (since one bitflip of the generated media alters the whole hash) which is squishy and not perfectly reliable to begin with.
  - alkonaut 2y ago
    Nothing. But that’s not the point. The point is that, to a rounding error, all output is made by a small number of models from a small number of easily regulated companies. It’s never going to be possible to ensure all media is reliably tagged somehow. But if just half of media generated is identifiable as such that helps. Also helps avoid it in training new models, which could turn out useful.
anon291 2y ago
> Creating models that don't tag their output as such would be banned. This is just silly. Anyone would be able to disable this tagging in an open model.
- diggan 2y ago
  >> Creating models that don't tag their output as such would be banned. > This is just silly. Anyone would be able to disable this tagging in an open model. And we'd end up with people who thinks that any text that isn't tagged as "#MadeByLLM" as made by a human, which obviously wouldn't be great.
- jerjerjer 2y ago
  > Anyone would be able to disable this tagging in an open model. Metadata (I assume it's file metadata and not a watermark) can be removed from a final product (image, video, text) so open and closed models are equally affected.
drcode 2y ago
turning the earth into paperclips is not gonna happen immediately, so we can safely ignore that risk
woah 2y ago
> The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated. Just make a law that makes it so that AI content has to be tagged if it is being used for fraud
dangitman 2y ago
[dead]

worldsayshi 2y ago

Any law that tries to categorize non-trustworthy content seems doomed to fail. We need to find better ways to communicate trustworthiness, not the other way around. (And I'm not sure adding more laws can help here.)

alkonaut 2y ago
No I don't think technical means will work fully either. But the thing about these regulations is that you can basically cover the 99% case by just thinking about the 5 largest players in the field, be it regulation for social media, AI or whatever. It doesn't matter that the law has loopholes or that some players aren't affected at all. Regulation that helps somewhat in a large majority of cases is massive.
- dangitman 2y ago
  [dead]

arder 2y ago

I think the most acheivable way of having some verification of AI images is simply for the AI generators to store finger prints of every image they generate. That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you". There's already technology for that sort of thing in the world (content ID from youtube, CSAM detection etc.). It's obviously not perfect, but could help and doesn't have the enormous side effects of trying to lock down all image generation.

Someone 2y ago
> That way if you ever want to know you can go back to Meta or whoever and say "Hey, here's this image, do you think it came from you". Firstly, if you want to know an image isn’t generated, you’d have to go to every ‘whoever’ in the world, including companies that no longer exist. Secondly, if you ask evil.com that question, you would have to trust them to answer honestly for both all images they generated and images they didn’t generate (claiming real pictures were generated by you can probably be career-ending for a politician) This is worse than https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF: “Program testing can be used to show the presence of bugs, but never to show their absence!”. You can neither show an image is real nor that it is fake.
kortex 2y ago
What's to stop someone from downloading an open source model, running it themselves, and either just not sharing the hashes, subtly corrupting the hash algo so that it gives a false negative, etc? Also you need perceptual hashing (since one bitflip of the generated media alters the whole hash) which is squishy and not perfectly reliable to begin with.
- alkonaut 2y ago
  Nothing. But that’s not the point. The point is that, to a rounding error, all output is made by a small number of models from a small number of easily regulated companies. It’s never going to be possible to ensure all media is reliably tagged somehow. But if just half of media generated is identifiable as such that helps. Also helps avoid it in training new models, which could turn out useful.

anon291 2y ago

> Creating models that don't tag their output as such would be banned. This is just silly. Anyone would be able to disable this tagging in an open model.

diggan 2y ago
>> Creating models that don't tag their output as such would be banned. > This is just silly. Anyone would be able to disable this tagging in an open model. And we'd end up with people who thinks that any text that isn't tagged as "#MadeByLLM" as made by a human, which obviously wouldn't be great.
jerjerjer 2y ago
> Anyone would be able to disable this tagging in an open model. Metadata (I assume it's file metadata and not a watermark) can be removed from a final product (image, video, text) so open and closed models are equally affected.

drcode 2y ago

turning the earth into paperclips is not gonna happen immediately, so we can safely ignore that risk

woah 2y ago

> The immediate danger of large AI models isn't that they'll turn the earth to paperclips it's that we'll create fraud as a service and have a society where nothing can be trusted. I'd be all for a law (however clumsy) that made image, audio or video content produced by models with over X parameters to be marked with metadata saying it's AI generated. Just make a law that makes it so that AI content has to be tagged if it is being used for fraud

dangitman 2y ago

[dead]