5 ms·

CivitAI Policy Update: Removal of Real-Person Likeness Content

41 points by 50kIters 1y ago

duxup 1y ago
> We are removing models and images depicting real-world individuals So any images that look real, or images that look like specific real people?
- Waterluvian 1y ago
  It’s ambiguous but I have a sense that it’s the latter. The former would be quite the change.
- throwanem 1y ago
  > This applies to any depiction of a real person, regardless of context or rating - including PG and PG-13 content. Whether it’s a public figure, celebrity, influencer, or private individual, if a model or image is based on a real human being, it will be removed. This includes fan-art depictions of characters portrayed by a celebrity - e.g. Indiana Jones (Harrison Ford). It is the second or third paragraph in the link.
  - duxup 1y ago
    The way they word it each time "real person" comes up I'm honestly unsure. I'll assume that means someone in real life. But ... won't that happen accidentally with AI depictions?
    - throwanem 1y ago
      I assume they mean any natural person, and whether it happens accidentally (as opposed to "accidentally") does not appear at issue here. You can probably get Stable Diffusion to violate likeness rights if you try hard enough, but there is a material difference between that and publishing a tool making it trivial for anyone to do so. These are, or I suppose were, such tools: Loras specifically trained to replicate the appearance of specific natural persons. For public figures the rules are slightly different when it comes to one's likeness, but not absent. I can't imagine anyone sensible hosting that stuff for one second, Section 230 or no. But either way, however long this was going on, only a fool would assume it would last forever.
- godelski 1y ago
  Likely the latter. They hosted a lot of LoRAs and embeddings that help generate images of specific people. Users would post samples of those people nude. Honestly, I'm surprised it lasted this long. Like that was clearly egregious. At least pretend to not be a deep fake porn generator...
  - Aeolun 1y ago
    > At least pretend to not be a deep fake porn generator Why? Like, people are going to do it anyway. Whether in their minds or with some tool. Public or behind closed doors. That genie is very firmly out of the bottle, and it’s not going to go back in. At least when it’s on Civit it’s abundantly clear it is fake content. You can run any of these models on a consumer GPU.
    - jazzyjackson 1y ago
      Because we live in a society and so get to have some influence on what behaviors we deem acceptable from our peers, and creating sexual images with the faces of your neighbors, acquaintances and even your foes is considered by some of us unacceptable, degenerate behavior Just because something is possible doesn't mean it has to be allowed to happen
    - throwanem 1y ago
      I'll credit this argument when I see you publish a training set of selfies, or a model finetuned thereon. If you want to argue no one should expect to have anything to hide, you'd better do it from a glass house and in the altogether if you mean to be taken seriously, don't you think? It is a radical departure. The example is therefore necessarily yours to set, else you show yourself a hypocrite.
      - Aeolun 1y ago
        Just because I wouldn’t have an issue with random people generating porn of me, doesn’t mean I want to seek it out, see it, or facilitate it. It’s fundamentally no different from acknowledging that there may be a fanclub on the other side of the country with a stash of aeolun porn. As long as they’re not using it to advertise their services, and don’t tell me what kind of degenerate things they’re doing with it, what does that hurt me? Now that you’ve said it I’m actually mildly curious what would happen if I fine-tune a model purely on myself. But what would I do with it even if I made it? I can’t post it on Civit any more can I?
minimaxir 1y ago
Relatedly, CivitAI has recently had issues with their credit card processors for the reasons you suspect, and today they lost the ability to process credit card payments: https://civitai.com/articles/14945 https://civitai.com/articles/14945 I suspect the company will be operating more conservatively policywise for the foreseeable future.
- beeflet 1y ago
  I think this is a pretty acceptable use-case for cryptocurrency/micropayments. Credit card companies are notoriously restrictive.
  - OsrsNeedsf2P 1y ago
    > acceptable use-case for cryptocurrency Once in a while I think this, and then I remember what a disaster cryptocurrency became
    - protocolture 1y ago
      Crypto is fine its the users that are the problem.
      - okayishdefaults 1y ago
        Is it at this point? When used earnestly, it's regulated, traceable, and slower than other methods of transacting money. History can and has been rewritten, but not when someone is scammed. Seems like bad currency, but maybe you're aware of something meaningful that crypto contributes.
        protocolture 1y ago
        >Regulated Well humans are regulated so I dont know what you are driving at here. >Traceable The core concept was built on traceability. Privacy coins are the aberration. Like crypto was developed by people who want provable auditing of banks online. Actually if you compare Bitcoin to later standards, Bitcoins biggest weakness is that it wants to track coins individually instead of just balances. Literally invented by goldbugs. >Slower Depends on both parties. I can go Crypto -> Crypto -> Fiat in like 15 minutes. Osko can be faster. >History can and has been rewritten Thats a fail state but its done rather less than traditional currency >Seems like bad currency, but maybe you're aware of something meaningful that crypto contributes. My fondest memory was watching a whole bunch of libertarian crypto guys using it to donate to Venezuelans who would pop up in crypto spaces to talk about how hard their lives were and how bad government had screwed up their lives. I liked to think the libertarians were getting scammed but it didnt really matter, because there werent many other onramps into VZ at the time. Really its best feature is that its largely unpreventable. Sure you can police the on and off ramps to an extent. But if I need to evade financial censorship, I can. Mostly I see people against crypto throw up a big smokescreen but at the end of the day they tend to be in favor of the financial censorship that crypto is avoiding at the moment. Be that donations to wikileaks, purchasing services without a credit card or what have you.
userbinator 1y ago
As a saying I once heard goes: Trying to stuff the genie back in the lamp will only ensure it replicates itself into other lamps.
Rohitcss 1y ago
Deepfakes are already becoming a mess to handle.
- Joel_Mckay 1y ago
  Indeed, if businesses will rip off Mark Hamill of all people, than most actors stood zero chance of protecting their image. It is an odd part of US copyright well known to figures who sue already, and it is good normal citizens now have similar rights. Some folks took things way to far already, and unfortunately copyright enforcement on the web has always been ridiculous. =3
  - Rohitcss 1y ago
    Agree!!
sinuhe69 1y ago
Will that also exclude users’ own person? That’ll be to bad. You not only take away the agency of the people, you also limit the creative space severely. I guess open source models for image diffusion will get a huge boost then.
- ZaoLahma 1y ago
  This is an interesting one. Do we own the rights to our own likeness? And if we do, what about doppelgangers - people who look eerily similar to celebrities, or other "unknown" people?
  - AnthonyMouse 1y ago
    > Do we own the rights to our own likeness? And if we do, what about doppelgangers - people who look eerily similar to celebrities, or other "unknown" people? Is this somehow a novel question? We've had the same issue since at least the invention of cameras.
    - throwanem 1y ago
      It's fun for these dorks to fantasize. Knowing things is for nerds.
sureIy 1y ago
What has been the problem exactly? Just generic "fake news" or did the platform generate porn?
- v5o 1y ago
  [dead]
- altairprime 1y ago
  Recent legislative changes (further) increased the risk tier of generative AI when human beings could be reasonably considered to have grounds to sue the business, and the bank likely also sought and received regulatory advice directing them to classify such businesses as higher risk (than their tolerance threshold might have permitted previously). To quote the recent Debanking article: > This particular bank did not, at the time, have a small business practice within its personal banking division. Very many banks do, but this particular bank did not. And thus this bank had not built out the higher degree of policies and procedures [required] https://news.ycombinator.com/item?id=42371476 https://news.ycombinator.com/item?id=42371476 Most likely, the payment processor does not wish to invest operational spending into the necessary banking and processing policies that continuing to transact with CivitAI without restrictions that reduce legal risk would have demanded of them. And then when CAI encouraged everyone to spend money rapidly and fast, it massively spiked their transaction volume, leading the bank to kill services a day early due to the additional risk flag that sort of processing behavior carries on an everyday basis. (I am not your lawyer, this is not legal advice.)
Aeolun 1y ago
What a surprise. Not. All of these companies go the same way once they become popular. Count the number of days until models that are capable of generating real-person likeness content are banned too.

41 points by 50kIters 1y ago

> We are removing models and images depicting real-world individuals So any images that look real, or images that look like specific real people?

Waterluvian 1y ago
It’s ambiguous but I have a sense that it’s the latter. The former would be quite the change.
throwanem 1y ago
> This applies to any depiction of a real person, regardless of context or rating - including PG and PG-13 content. Whether it’s a public figure, celebrity, influencer, or private individual, if a model or image is based on a real human being, it will be removed. This includes fan-art depictions of characters portrayed by a celebrity - e.g. Indiana Jones (Harrison Ford). It is the second or third paragraph in the link.
- duxup 1y ago
  The way they word it each time "real person" comes up I'm honestly unsure. I'll assume that means someone in real life. But ... won't that happen accidentally with AI depictions?
  - throwanem 1y ago
    I assume they mean any natural person, and whether it happens accidentally (as opposed to "accidentally") does not appear at issue here. You can probably get Stable Diffusion to violate likeness rights if you try hard enough, but there is a material difference between that and publishing a tool making it trivial for anyone to do so. These are, or I suppose were, such tools: Loras specifically trained to replicate the appearance of specific natural persons. For public figures the rules are slightly different when it comes to one's likeness, but not absent. I can't imagine anyone sensible hosting that stuff for one second, Section 230 or no. But either way, however long this was going on, only a fool would assume it would last forever.
godelski 1y ago
Likely the latter. They hosted a lot of LoRAs and embeddings that help generate images of specific people. Users would post samples of those people nude. Honestly, I'm surprised it lasted this long. Like that was clearly egregious. At least pretend to not be a deep fake porn generator...
- Aeolun 1y ago
  > At least pretend to not be a deep fake porn generator Why? Like, people are going to do it anyway. Whether in their minds or with some tool. Public or behind closed doors. That genie is very firmly out of the bottle, and it’s not going to go back in. At least when it’s on Civit it’s abundantly clear it is fake content. You can run any of these models on a consumer GPU.
  - jazzyjackson 1y ago
    Because we live in a society and so get to have some influence on what behaviors we deem acceptable from our peers, and creating sexual images with the faces of your neighbors, acquaintances and even your foes is considered by some of us unacceptable, degenerate behavior Just because something is possible doesn't mean it has to be allowed to happen
  - throwanem 1y ago
    I'll credit this argument when I see you publish a training set of selfies, or a model finetuned thereon. If you want to argue no one should expect to have anything to hide, you'd better do it from a glass house and in the altogether if you mean to be taken seriously, don't you think? It is a radical departure. The example is therefore necessarily yours to set, else you show yourself a hypocrite.
    - Aeolun 1y ago
      Just because I wouldn’t have an issue with random people generating porn of me, doesn’t mean I want to seek it out, see it, or facilitate it. It’s fundamentally no different from acknowledging that there may be a fanclub on the other side of the country with a stash of aeolun porn. As long as they’re not using it to advertise their services, and don’t tell me what kind of degenerate things they’re doing with it, what does that hurt me? Now that you’ve said it I’m actually mildly curious what would happen if I fine-tune a model purely on myself. But what would I do with it even if I made it? I can’t post it on Civit any more can I?

minimaxir 1y ago

Relatedly, CivitAI has recently had issues with their credit card processors for the reasons you suspect, and today they lost the ability to process credit card payments: https://civitai.com/articles/14945 https://civitai.com/articles/14945 I suspect the company will be operating more conservatively policywise for the foreseeable future.

beeflet 1y ago
I think this is a pretty acceptable use-case for cryptocurrency/micropayments. Credit card companies are notoriously restrictive.
- OsrsNeedsf2P 1y ago
  > acceptable use-case for cryptocurrency Once in a while I think this, and then I remember what a disaster cryptocurrency became
  - protocolture 1y ago
    Crypto is fine its the users that are the problem.
    - okayishdefaults 1y ago
      Is it at this point? When used earnestly, it's regulated, traceable, and slower than other methods of transacting money. History can and has been rewritten, but not when someone is scammed. Seems like bad currency, but maybe you're aware of something meaningful that crypto contributes.
      - protocolture 1y ago
        >Regulated Well humans are regulated so I dont know what you are driving at here. >Traceable The core concept was built on traceability. Privacy coins are the aberration. Like crypto was developed by people who want provable auditing of banks online. Actually if you compare Bitcoin to later standards, Bitcoins biggest weakness is that it wants to track coins individually instead of just balances. Literally invented by goldbugs. >Slower Depends on both parties. I can go Crypto -> Crypto -> Fiat in like 15 minutes. Osko can be faster. >History can and has been rewritten Thats a fail state but its done rather less than traditional currency >Seems like bad currency, but maybe you're aware of something meaningful that crypto contributes. My fondest memory was watching a whole bunch of libertarian crypto guys using it to donate to Venezuelans who would pop up in crypto spaces to talk about how hard their lives were and how bad government had screwed up their lives. I liked to think the libertarians were getting scammed but it didnt really matter, because there werent many other onramps into VZ at the time. Really its best feature is that its largely unpreventable. Sure you can police the on and off ramps to an extent. But if I need to evade financial censorship, I can. Mostly I see people against crypto throw up a big smokescreen but at the end of the day they tend to be in favor of the financial censorship that crypto is avoiding at the moment. Be that donations to wikileaks, purchasing services without a credit card or what have you.

userbinator 1y ago

As a saying I once heard goes: Trying to stuff the genie back in the lamp will only ensure it replicates itself into other lamps.

Rohitcss 1y ago

Deepfakes are already becoming a mess to handle.

Joel_Mckay 1y ago
Indeed, if businesses will rip off Mark Hamill of all people, than most actors stood zero chance of protecting their image. It is an odd part of US copyright well known to figures who sue already, and it is good normal citizens now have similar rights. Some folks took things way to far already, and unfortunately copyright enforcement on the web has always been ridiculous. =3
- Rohitcss 1y ago
  Agree!!

sinuhe69 1y ago

Will that also exclude users’ own person? That’ll be to bad. You not only take away the agency of the people, you also limit the creative space severely. I guess open source models for image diffusion will get a huge boost then.

ZaoLahma 1y ago
This is an interesting one. Do we own the rights to our own likeness? And if we do, what about doppelgangers - people who look eerily similar to celebrities, or other "unknown" people?
- AnthonyMouse 1y ago
  > Do we own the rights to our own likeness? And if we do, what about doppelgangers - people who look eerily similar to celebrities, or other "unknown" people? Is this somehow a novel question? We've had the same issue since at least the invention of cameras.
  - throwanem 1y ago
    It's fun for these dorks to fantasize. Knowing things is for nerds.

sureIy 1y ago

What has been the problem exactly? Just generic "fake news" or did the platform generate porn?

v5o 1y ago
[dead]
altairprime 1y ago
Recent legislative changes (further) increased the risk tier of generative AI when human beings could be reasonably considered to have grounds to sue the business, and the bank likely also sought and received regulatory advice directing them to classify such businesses as higher risk (than their tolerance threshold might have permitted previously). To quote the recent Debanking article: > This particular bank did not, at the time, have a small business practice within its personal banking division. Very many banks do, but this particular bank did not. And thus this bank had not built out the higher degree of policies and procedures [required] https://news.ycombinator.com/item?id=42371476 https://news.ycombinator.com/item?id=42371476 Most likely, the payment processor does not wish to invest operational spending into the necessary banking and processing policies that continuing to transact with CivitAI without restrictions that reduce legal risk would have demanded of them. And then when CAI encouraged everyone to spend money rapidly and fast, it massively spiked their transaction volume, leading the bank to kill services a day early due to the additional risk flag that sort of processing behavior carries on an everyday basis. (I am not your lawyer, this is not legal advice.)

Aeolun 1y ago

What a surprise. Not. All of these companies go the same way once they become popular. Count the number of days until models that are capable of generating real-person likeness content are banned too.