4 ms·

Discord banned multiple servers that r/deepfakes created and their reasoning was that it's being used for creating involuntary porn and so is against their ToS.

Discord banned multiple servers that r/deepfakes created and their reasoning was that it's being used for creating involuntary porn and so is against their ToS. I'm curious to hear your thoughts on the legal status of deepfakes as well as the moral and ethical questions behind the usage of the tech. A thought experiment : I run a chat room site and will be adding support for user created chat rooms (something like discord for cryptocurrency communities with real time price widgets etc). Let's say I went ahead and got them to use my chat room, what would the consequences be? And what do you think about the moral and ethical questions behind hosting such a community? Am I now enabling the creation of deepfakes? By choosing to avoid hosting such a controversial community, am I imposing my moral and ethical reasoning on others (the free speech argument)? How do think about the deepfakes project, which is mostly being used for creating celebrity porn, but the tech itself is interesting and has applications(targeted ads or in stunt doubles etc)? To be clear, I'm not trying to host a chat room for them, this is just a thought experiment. I don't like what it's being used for and don't want to support it in any way.

ggg9990 9y ago
Legally, in the US, if you own the copyright to the source images, the copyright to the porn, and you’re not selling the output, it’s 100% legal
- zawerf 9y ago
  There was a thread on /r/machinelearning titled "Is it legal to use copyright material as training data?" https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_it_legal_to_use_copyright_material_as_training/ https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_... But the issue here isn't whether training on copyrighted face images is legal. You can totally take copyright-free photos of anybody as long as they appear in public. The more interesting question I think is whether you own the copyright to the 3D capture of your face. For example disney won't let you get away with using models of their characters. Don't I have the right to not let anyone use the "model" that I spent my literal life growing?
  - brazzledazzle 9y ago
    It's even more complicated than your Disney character analogy because if you can own the copyright of your likeness you can sell it. Imagine a young actress signing over her likeness to a predatory agency early in her career before making it big.
  - Larrikin 9y ago
    By default you don't own your likeness, but you can file the paperwork to own it. Michael Jordan famously wasn't in any of the major basketball games during his prime because he owned his own likeness, so there would have to have been separate licensing deals apart from the NBA to include him.
  - PeterisP 9y ago
    It's probably not relevant to bring in copyright here. The restrictions for using your likeness or publicizing deepfake videos with your face on them come from other laws, not copyright. For example, "disney won't let you get away with using models of their characters", but (depending on how you use them) it's just as likely to use trademark restrictions even in situations where copyright law would let them use them. In a similar manner, the fact that you can take copyright-free photos of anybody as long as they appear in public doesn't necessarily mean (for example) that you're allowed to publish those photos like you can your own; or that this has any impact whatsoever on your rights to use copyrighted face images - you could have taken the photo, but you did not, so standard restrictions apply. "If some law prohibits others to do that it does not really equal to "the copyright to the 3D capture of your face" even if some restrictions are similar, it's an apples vs oranges thing, it shouldn't be called that - terminology matters, because it implies different rules. For this discussion, the main question is about restrictions for publicizing fake videos of other people without their consent. There are some restrictions, they're not copyright-based and also are quite different for every jurisdiction, as they aren't harmonized with global treaties like copyright is. Also, these restrictions are different for public people (e.g. the examples with Trump or famous actresses), and the restrictions for distributing videos with your face would be different.
  - rmc 9y ago
    EU Data protection law might kick in as well.
maxerickson 9y ago
Using other people's visage without their permission is gross. It's immoral and unethical. The idea that you can have morals and ethics without sometimes acting on them is also sort of incoherent. If you claim to believe it is immoral to do something and then do business in spite of that, it directly puts the lie to your claim. This is separate from whether you believe that there should be legal consequences or other government action attached to certain acts.
- nlowell 9y ago
  People act against their morals constantly and all the time. Morals often conflict with each other and a person may have moral ideals that they recognize as impractical to actually enact due to friction either with their other morals or reality itself.
- acomar 9y ago
  > The idea that you can have morals and ethics without sometimes acting on them is also sort of incoherent. This depends on your ethical system and your reasons for believing it immoral. For example, if the circumstances change enough to move something from immoral to merely neutral, some ethical systems will then allow you to do it. A utilitarian example: you might generally find something immoral (because it causes harm) but in the presence of somebody who gets great pleasure from it, you find it allowable (actually, it becomes a moral duty to the utilitarian, but you get the gist).
- runeks 9y ago
  > Using other people's visage without their permission is gross. It's immoral and unethical. This seems a bit extreme to me. In my view, the line is crossed when it’s used for fraud, i.e. misrepresenting the truth (for whatever purpose). Face-swapping Daniel Craig’s face onto my body, and showing it to my friends, is neither immoral nor unethical — it’s actually kind of funny.
  - maxerickson 9y ago
    I think publicizing a non celebrity is problematic even if there isn't any misrepresentation. It can be fine in a news context, but there's not a lot of faked videos that would be appropriate in that context. I even thought the faked Fred Astaire ad was in poor taste. http://www.criticalcommons.org/Members/kellimarshall/clips/Astaire_DirtDevil2.mp4/view http://www.criticalcommons.org/Members/kellimarshall/clips/A...
mc32 9y ago
Legal status, i don't think is the issue. It's not illegal to produce likenesses, as long as they aren't published thus allowing people to believe the rendered facsimile is the real person. I think publishing the impersonation is unethical since it allows people to believe it's the actual person, so it's fraudulent from that perspective.
- deleted 9y ago
  [deleted]
jakobegger 9y ago
If you host a party, and a bunch of jerks show up, you better ask them to leave. It’s your home, you can decide who you want to host.
superkuh 9y ago
It also shows why Discord isn't such a good choice for most people. They're obviously susceptible to knee-jerk moves directed by their public relations department. Second, it's not actually a Discord server no matter how much they market it that way. Just like it's not a Facebook server. They just try to muddy the waters vs. real voice chat servers you can run yourself like mumble or teamspeak. People should move away from centralized services where you are the product like Discord and use real voice chat servers. Now there's no ethical issue at all. Cut the gordian knot.
- giancarlostoro 9y ago
  Discord is popular due to easy adoption into it. Mumble takes a bit more effort and I've wasted an hour with a friend trying to make it all work voice and roles. Discord just works. If Mumble could be more user friendly or similarly friendly it would go a long way.
- debt 9y ago
  "It also shows why Discord isn't such a good choice for most people. They're obviously susceptible to knee-jerk moves directed by their public relations department" lol why because they banned people for creating fake revenge porn?
  - superkuh 9y ago
    They also ban people who talk about video game cheats.
debt 9y ago
If Apple's FaceID treats your face like a signature, would it be legal to forge your signature, even it was a really good forgery?
- prepend 9y ago
  It depends on if you commit fraud. It’s completey legal to forge people’s signature on a pad of paper over and over. It’s illegal if you commit fraud with that forgery by signing a check.
emodendroket 9y ago
I mean none of this is a new problem, is it? The means are more sophisticated but the basic problem is not new.

ggg9990 9y ago

Legally, in the US, if you own the copyright to the source images, the copyright to the porn, and you’re not selling the output, it’s 100% legal

zawerf 9y ago
There was a thread on /r/machinelearning titled "Is it legal to use copyright material as training data?" https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_it_legal_to_use_copyright_material_as_training/ https://www.reddit.com/r/MachineLearning/comments/4qrgh8/is_... But the issue here isn't whether training on copyrighted face images is legal. You can totally take copyright-free photos of anybody as long as they appear in public. The more interesting question I think is whether you own the copyright to the 3D capture of your face. For example disney won't let you get away with using models of their characters. Don't I have the right to not let anyone use the "model" that I spent my literal life growing?
- brazzledazzle 9y ago
  It's even more complicated than your Disney character analogy because if you can own the copyright of your likeness you can sell it. Imagine a young actress signing over her likeness to a predatory agency early in her career before making it big.
- Larrikin 9y ago
  By default you don't own your likeness, but you can file the paperwork to own it. Michael Jordan famously wasn't in any of the major basketball games during his prime because he owned his own likeness, so there would have to have been separate licensing deals apart from the NBA to include him.
- PeterisP 9y ago
  It's probably not relevant to bring in copyright here. The restrictions for using your likeness or publicizing deepfake videos with your face on them come from other laws, not copyright. For example, "disney won't let you get away with using models of their characters", but (depending on how you use them) it's just as likely to use trademark restrictions even in situations where copyright law would let them use them. In a similar manner, the fact that you can take copyright-free photos of anybody as long as they appear in public doesn't necessarily mean (for example) that you're allowed to publish those photos like you can your own; or that this has any impact whatsoever on your rights to use copyrighted face images - you could have taken the photo, but you did not, so standard restrictions apply. "If some law prohibits others to do that it does not really equal to "the copyright to the 3D capture of your face" even if some restrictions are similar, it's an apples vs oranges thing, it shouldn't be called that - terminology matters, because it implies different rules. For this discussion, the main question is about restrictions for publicizing fake videos of other people without their consent. There are some restrictions, they're not copyright-based and also are quite different for every jurisdiction, as they aren't harmonized with global treaties like copyright is. Also, these restrictions are different for public people (e.g. the examples with Trump or famous actresses), and the restrictions for distributing videos with your face would be different.
- rmc 9y ago
  EU Data protection law might kick in as well.

maxerickson 9y ago

Using other people's visage without their permission is gross. It's immoral and unethical. The idea that you can have morals and ethics without sometimes acting on them is also sort of incoherent. If you claim to believe it is immoral to do something and then do business in spite of that, it directly puts the lie to your claim. This is separate from whether you believe that there should be legal consequences or other government action attached to certain acts.

nlowell 9y ago
People act against their morals constantly and all the time. Morals often conflict with each other and a person may have moral ideals that they recognize as impractical to actually enact due to friction either with their other morals or reality itself.
acomar 9y ago
> The idea that you can have morals and ethics without sometimes acting on them is also sort of incoherent. This depends on your ethical system and your reasons for believing it immoral. For example, if the circumstances change enough to move something from immoral to merely neutral, some ethical systems will then allow you to do it. A utilitarian example: you might generally find something immoral (because it causes harm) but in the presence of somebody who gets great pleasure from it, you find it allowable (actually, it becomes a moral duty to the utilitarian, but you get the gist).
runeks 9y ago
> Using other people's visage without their permission is gross. It's immoral and unethical. This seems a bit extreme to me. In my view, the line is crossed when it’s used for fraud, i.e. misrepresenting the truth (for whatever purpose). Face-swapping Daniel Craig’s face onto my body, and showing it to my friends, is neither immoral nor unethical — it’s actually kind of funny.
- maxerickson 9y ago
  I think publicizing a non celebrity is problematic even if there isn't any misrepresentation. It can be fine in a news context, but there's not a lot of faked videos that would be appropriate in that context. I even thought the faked Fred Astaire ad was in poor taste. http://www.criticalcommons.org/Members/kellimarshall/clips/Astaire_DirtDevil2.mp4/view http://www.criticalcommons.org/Members/kellimarshall/clips/A...

mc32 9y ago

Legal status, i don't think is the issue. It's not illegal to produce likenesses, as long as they aren't published thus allowing people to believe the rendered facsimile is the real person. I think publishing the impersonation is unethical since it allows people to believe it's the actual person, so it's fraudulent from that perspective.

deleted 9y ago
[deleted]

jakobegger 9y ago

If you host a party, and a bunch of jerks show up, you better ask them to leave. It’s your home, you can decide who you want to host.

superkuh 9y ago

It also shows why Discord isn't such a good choice for most people. They're obviously susceptible to knee-jerk moves directed by their public relations department. Second, it's not actually a Discord server no matter how much they market it that way. Just like it's not a Facebook server. They just try to muddy the waters vs. real voice chat servers you can run yourself like mumble or teamspeak. People should move away from centralized services where you are the product like Discord and use real voice chat servers. Now there's no ethical issue at all. Cut the gordian knot.

giancarlostoro 9y ago
Discord is popular due to easy adoption into it. Mumble takes a bit more effort and I've wasted an hour with a friend trying to make it all work voice and roles. Discord just works. If Mumble could be more user friendly or similarly friendly it would go a long way.
debt 9y ago
"It also shows why Discord isn't such a good choice for most people. They're obviously susceptible to knee-jerk moves directed by their public relations department" lol why because they banned people for creating fake revenge porn?
- superkuh 9y ago
  They also ban people who talk about video game cheats.

debt 9y ago

If Apple's FaceID treats your face like a signature, would it be legal to forge your signature, even it was a really good forgery?

prepend 9y ago
It depends on if you commit fraud. It’s completey legal to forge people’s signature on a pad of paper over and over. It’s illegal if you commit fraud with that forgery by signing a check.

emodendroket 9y ago

I mean none of this is a new problem, is it? The means are more sophisticated but the basic problem is not new.