6 ms·
Creating sexually explicit deepfake images to be made offence in UK
- jjgreen 3y agoMore details on gov.uk [1] but still without a definition of what a 'deepfake' (their quotes) actually is. Reference is made to the law on child pornography, for that no computer (nor child) need be involved. A crude pencil drawing falls within the remit of the law, so I guess something along those lines ... [1] https://www.gov.uk/government/news/government-cracks-down-on-deepfakes-creation https://www.gov.uk/government/news/government-cracks-down-on...
- deleted 3y ago[deleted]
- derefr 3y agoI hope they're very careful with the eventual definition, as at this point you might call Photoshop's AI-driven outpainting feature a "deepfake." If an image is already explicit, and you use AI to de-crop a missing foot back into the picture, have you now created an "explicit deepfake"?
- Y_Y 3y ago> Laura Farris, the minister for victims and safeguarding, said the creation of deepfake sexual images was “unacceptable irrespective of whether the image is shared”. This is an interesting one because it violates the vague principle of "whatever done by consenting adults in the privacy of their own home is cool". Like what if someone pastes my face ona naked body, or imagines me naked, or has a dream about me flying through the window naked and writes it in their private dream diary. I think it's getting into hairy legal territory if the "infringing material" is only seen by the otherwise innocent creator on an airgapped computer.
- Svip 3y ago> Under the legislation, anyone who creates such an image without consent will face a criminal record and an unlimited fine. I feel like "without consent" to be pretty key here.
- oceanplexian 3y agoYou don't have a right to consent to being part of an artwork. For example, even the King of England could not walk into a museum and demand they take a painting down because it's unflattering.
- jncfhnb 3y agoAnd now you do have the right to consent to being in porn.
- randomdata 3y agoUnless the King of England petitioned for law that gave him that right. Which is basically what we're seeing here.
- Gloomily3819 3y agoAnd that would be an authoritarian measure.
- randomdata 3y agoTo petition change isn't authoritarian. That's the foundation of democracy.
- kwhitefoot 3y agoThe petitioning is not authoritarian but the measure is.
- randomdata 3y ago
- dwighttk 3y agoFor what it’s worth that is a bad vague principle
- jt2190 3y ago> Laura Farris, the minister for victims and safeguarding, said… Instead of looking for those answers in the text of Farris’ summary of the law, we should look at the text of the law itself. Edit: https://bills.parliament.uk/bills/3511/publications https://bills.parliament.uk/bills/3511/publications probably has the text we're looking for, somewhere.
- rootusrootus 3y agoThere's precedent for that already, it is not blazing any new territory. Mere possession of CSAM is very illegal.
- Dig1t 3y agoThe existence of a CSAM implies a crime was committed though, since there is no way to legally produce such an image. Possessing a CSAM image is pretty much the same as aiding/abetting the original crime. A private deepfake image is not the same.
- Kranar 3y agoThat's not true, the justification outlawing CSAM being the result of a crime is not the actual legal justification for banning it and it would not at all hold up in court. There are plenty of incredibly disturbing videos and images of some of the most gruesome crimes committed and they are lawfully accessible and obtainable on the open Internet, including some such images on Wikipedia. The documenting of a crime is not and has never been illegal.
- asplake 3y agoPer jjgreen’s comment, in UK law, for CSAM, an actual child need not be involved. Same logic proposed for non-consensual deepfakes.
- rootusrootus 3y agoYou might be right, based on Kranar's comment. I thought that CSAM was illegal to artificially create, as well, which would be similar to a deepfake. On a related note, I assume there is already a lot of deepfake CSAM in certain circles. If that is ruled legal, it will make finding and prosecuting real life CSAM creators much more difficult. At least in the US.
- numpad0 3y agoDepends. Some jurisdictions classify CSAM by end results, others by process. Where it's process oriented it's not CSAM unless a child is not physically abused.
- neom 3y ago1967, then Canadian PM Pierre Trudeau on the government and consenting adults in their bedrooms: https://www.youtube.com/watch?v=6OoqgGgmVmI https://www.youtube.com/watch?v=6OoqgGgmVmI I think the issue here is the between consenting adults part, deepfakes in this instance are probably not between consenting adults. I also think it's important to think about the reality of how this likely plays out. If someone deepfakes me and faps and I never find out and they never tell anyone, it would be quite hard for there to be action. So if someone is deepfaking you and you know about it, there must be a logical path there, and none of them look good for the deepfaker. Can't say I know where I land on the issue, and I've thought about it for days and hours.
- floor2 3y agoExcept the actual human being who resembles the deepfake image isn't actually involved at all. People are acting like a "deepfake of a person" is something different than a drawing, painting, song, poem or any other artistic creation inspired by, based on, or with elements drawn from a real person. "If someone is deepfaking you" is such a weird imaginary problem. Someone might be imagining you. Someone might see you in a coffee shop while writing a screenplay and use the outfit you're wearing to describe a character in their story. Who cares? If someone is committing fraud by trying to pass off a deepfake as a real photo, then we already have laws around fraud, defamation, harassment to cover that. If someone makes a deepfake that looks similar to a real person for their own amusement, that's no different than writing fan-fiction or sketching in a notebook.
- 23B1 3y agoYou should think long and hard about the second- and third-order effects.
- catlover76 3y ago[dead]
- chrisjj 3y agoNo such thing. Distribution of an image is not an effect of its creation.
- bigbillheck 3y ago>"whatever done by consenting adults in the privacy of their own home is cool". For example, cooking meth.
- Grimblewald 3y agoFor sure but for the otherwise innocent creator on an airgapped pc there is nothing to worry about. Who would ever know? I do think the other examples you gave have one significant and fundamental difference - no one would mistake those things as being real. The problem with deepfakes is these have a similar social impact and capcity for harm as real nudes do, only there is no element of consent. You could ruin someones life quite easily with deep fakes, but not with your lewd poetry or sketches. So it seems reasonable to me to ban these with a blanket ban. I would prefer we lived in a world where that wasnt neccesary, but for every technology you must imagine the most sinister and exploitative application and realise that someone will try that and worse. How do you even begin to combat deepfakes and the social problems they bring? how many bodies are considered acceptable until we get it right? Possibly, its best to over reach a little then relax the grip rather than to take it slow, given what is on the line for some.
- sungho_ 3y agoI think the point is about the images which will never be shared No one wonders why they shouldn't spread fake nude photos of real people
- Grimblewald 3y agoand again, if they will never be shared, there is not much to worry about. We do need tools to persecute people who failed on the 'never be shared' front. Be it theft or otherwise, if others get the material from them, it doesn't matter, it should be treated as guilty of producing with intent to distribute. This is for two reasons, the first being this way people cannot hide behind the 'oopsie' defense.Second, if you made compromising material of others without consent, your duty of care is through the roof. If it leaks, you are at fault, no matter how hard you tried to prevent it. You never had to make that material. There was no reason to make it other than a selfish self-centered personal pursuits of a hedonistic nature, now if its victimless I don't care personally but the moment your selfish self-centered behavior hurts others, you are a danger to society.
- 3y ago
- causal 3y agoBecause then people can and will create fine tunes of models trained on specific people and just share those instead. I'm worried we will soon hear about a girl bullied into suicide because of a model she was powerless to stop from spreading.
- bilbo0s 3y agoIt'll be about two seconds before creation of such models is declared an illegal act if a few suicides happen. sigh. I file this whole issue under: "This is why we can't have nice things". People who create models like that ruin everything for everyone.
- randomdata 3y agoThe implication is that only one party has consented when these fakes are created, and therefore not all involved have given their consent. Courts have previously deemed cartoon imagery of children to be equivalent to a real child, so considering the deepfake to be a real person that needs to give consent is not without precedent.
- deleted 3y ago[deleted]
- jncfhnb 3y agoThe basic obvious issue is “can you draw porn of someone”. The implied answer here appears to be “no” and the reason this is deemed necessary is an emergent effect of scale of being able to do this at such volume and quality. Saying you can’t draw porn of people without regard to implied authenticity or medium seems somehow more reasonable to me.
- ryandrake 3y agoThere just seems to be so much gray area here. Where is the burden of proof? Can someone argue that the depiction is of a fictional person who just looks similar to Plaintiff? How similar is too similar? Who's the judge of similarity? What if the training data did not contain any pictures of Plaintiff and the results were just tuned until it looked like him/her? What if Plaintiff has a twin sibling or someone who looks just like him/her and that person consented? What a mess--who's going to litigate all of this?
- jncfhnb 3y agoI don’t feel like anything is implying similarity is sufficient to make an artwork criminal. Although I might call out the opposite problem of making it a crime to request a sfw picture of a specific person and have the model generate something porny
- numpad0 3y agoThere's so much gray area because some people desperately wants porn banned than abuses and harms punished, and clarifying gray areas all the way removes side effects on purely generative porn. Personally I think head-swap porns should be handled with defamation logic than general anti-porn laws. I find it more likely that I would be disgusted by deepfake porn of mine if I were to become such public figure that warrants it, than appreciating it, but it feels weirder that local police would be deciding whether am I to like it or not.
- jncfhnb 3y agoThis is not a blanket ban on generated porn.
- deleted 3y ago[deleted]
- TimedToasts 3y agoUK: Coins the term 'thoughtcrime'. Also UK: Criminalizes 'thoughtcrime'.
- deadbunny 3y agoImagining some naked and producing graphic images of them without their consent is a little different.
- ryandrake 3y agoWhere do you draw the line? What about if someone, without consent, tells a story and describes every last detail of that person, short of drawing a physical picture? What if they write down that story? What if that story is kept in a safe inside your house and nobody else reads it? Why draw the line at a physical image?
- kergonath 3y agoIt’s not really, though, is it? It’s similar to things like death threats: creating a picture of someone getting killed is not illegal, but using it in a coercitive way very much is. It’s not a completely different thing just because it involves boobs. Harm is not done when the file is created, it’s done when it is spread maliciously to bully someone, and that’s where prosecution should be focused. The fact that a representation of a thing itself is illegal is quite backwards and ought to be at least seriously debated more seriously than in the moral panic we have now. Otherwise, we’re back to punishing people for their creations, whether that would be ideas, writings, paintings, or digital files, just because some people do not like it.
- lukan 3y agoYou still may phantasize about whatever thoughts you want to. But creating an image that can be watched in the real world, is more than "thought".
- throwaway74432 3y agoWait until your Neuralink can show what you're visualizing.
- commandlinefan 3y agoWhy this one case? Why isn't creating a deepfake of, say, a political figure taking a bribe also an offense?
- pelagicAustral 3y agoWhat's fake about a politician taking a bribe?
- jquery 3y agoAt least in the USA, our Supreme Court has basically legalized bribery. It's a 9-0 consensus, in their mind bribery is just how democracy works. This is what happens when ethics rules don't apply or are repeatedly ruled unconstitutional (something something fox guarding the henhouse), your term is for life, and you wield silly amounts of unchecked power.
- ceejayoz 3y agoWouldn't that be plain old defamation?
- chrisjj 3y agoNo, because defamation requires distribution.
- causal 3y agoI think you're right that there is a ton of focus on sexual imagery but not enough thought being put into the wider context of how easy it is to fake imagery (and soon audio and video). Plenty of people would still be disturbed to see fake videos of themselves fully clothed.
- TheAceOfHearts 3y agoHow are people thinking about this kind of attack? It was already viable to Photoshop someone's face into existing pornography. New tooling just lowers the barrier to execution. It seems like fighting an impossible battle since the tools are already widely available. I feel bad for people who are harmed by the creation and publication of such content, but it's not obvious to me that it's possible to combat it effectively using the legal system. Then again, maybe such laws don't have to be completely effective, so maybe it's sufficient that sharing such content cannot be done publicly in order to deter the most obvious abuses. Do you think in the very long term we'll remain so attached to protecting our likeness or will continued digitization of reality bring forth a loosening of such cultural norms?
- Eisenstein 3y ago> It seems like fighting an impossible battle since the tools are already widely available. People can do lots of things easily and are prevented by the legal system. I don't agree with the part of the law that makes this a crime if it isn't shared (who is being hurt in that instance?) but using an argument that essentially says 'it is hard to prevent people from doing something so don't even bother criminalizing it' is nonsense.
- deadbunny 3y agoPhotoshop takes time and skill. AI takes a single source image and like 20 seconds. In the time it takes someone to come up with something in Photoshop the AI guy has made thousands of images.
- causal 3y agoAnd getting easier. Producing fully animated, voiced, real time deepfake videos will be trivial within a few years
- bloopernova 3y agoThe barrier to creation is now far, far lower. Just a few face pictures and some idea of their body type, and you've got an instant nude. Children younger than 10 need to be taught the concept of consent and privacy, and how upsetting it is for people whose privacy and consent are violated. It ties into cliques and bullying, too. Teens can be brutal in their bullying and fake nude pictures will be added to the arsenal of abusive actions. Some of this stems from unrestricted usage of computers by minors who don't understand the impact their actions might have. Better parental tools and parents/guardians who take this topic seriously would be beneficial too. (I'm not arguing for some locked-down console instead of standard PCs, I just think that parents need to be able to know what their kids are doing online)
- stronglikedan 3y agoThey'll go after sharing too, since sharing is "creating" a copy. Ripe for abuse.
- Ekaros 3y agoI have fundamentally bigger issue with distribution than creation in this case... Create what ever you want on your private devices... But once you share or distribute without attached consent letter you should be punished.
- Devorlon 3y agoYes, because unconsensually sharing sexually explicit material of people is bad.
- failbuffer 3y agoUK becomes increasingly authoritarian every year. What a tragic and ignoble development for the nation that gave us the Manga Carta.
- hnlmorg 3y agoThis is true the world over. For example women in many states of America are fighting for their right for abortion. When the world is prosperous, people tend to vote for more liberal policies. And when economy is struggling, people tend to vote more conservatively. So we see more authoritarian policies like these.
- randomdata 3y agoIs this instance liberal (accepting of change) or conservative (hesitant of change)? On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy. On the other hand, AI is changing the world and it seems like an effort to try and hold the things around AI in place as to not seem them change. It could go either way. So could the state of the economy right now, for that matter.
- kergonath 3y ago> Is this instance liberal (accepting of change) or conservative (hesitant of change)? I don’t think this framing is very useful. Modern conservatives don’t have problems with change, they just want that change to go their way. They are perfectly able to innovate and push new doctrines or legal theories when it suits them. And on the other hand they do not care about preserving anything besides an idealised vision of “traditional values”, which are not as old as they seem. Looking at the actual policies and arguments, the spectrum is between oligarchy (or dictatorship for the most extremes) and democracy. It makes much more sense that way. > On one hand it seems liberal, wanting to see a change that gives individuals who might be subject of deepfakes more autonomy. It’s also a significant limitation of personal freedom in the case of a victimless crime, which is anything but liberal.
- mtnGoat 3y agoRegulating adult entertainment doesn’t work well, never has. They sites will just move their Operating jurisdiction and move on. The only entity possibly capable would be the card issuers/processors.
- 23B1 3y agoShould be an offense everywhere. "Free speech" is a very nebulous concept to be sure, but this should be a hard limit everywhere.
- Grimblewald 3y agoAgreed, the capacity for real world harm these things have is enormous, while simultabeously the benefit is near non existent. The entitlement of some people to think they should have the right to make photorealistic sexual material of their unconsenting obsession is bewildering.
- 23B1 3y agoIt's an unpopular view here on HN, but their minds will change when it's their sisters or their daughters.
- Grimblewald 3y agoI agree, but I hope we can solve this before their loved ones become collateral.
- chrisjj 3y agoSince they currently do have that right, why would you be bewildered?
- 23B1 3y ago"It's the law" isn't a valid argument in the face of technological change. When laws are changed with careful consideration, for both individual rights and social good – well, that's the very essence of the social contract.
- chrisjj 3y ago> "It's the law" isn't a valid argument in the face of technological change. It is a valid argument where I used it - against your purported bewilderment.
- Devorlon 3y agoWhile there'll be people arguing that you can always draw someone, or cut out their face from a photo and put it on someone else the major wrong that this is attempting to right is the abundance of services that now easily automate this. For those who talk about privacy in your own home, there's plenty of stuff that's done in peoples homes that's illegal and no one cares about. This law's only going to come into effect when these start being shared which I hope we can all agree is a bad thing if done unconsensually.
- chrisjj 3y ago> the major wrong that this is attempting to right is the abundance of services that now easily automate this. Not at all. Else the criminalisation would be of those services.
- rdl 3y agoI'm intrigued by similar state-level laws being passed in the US; a lot of those seem unconstitutional (1A/free speech) in the US, unless incredibly narrowly tailored and banning publishing or other harms from use. More reinforcement that the UK lacks a lot of protections of basic human rights, though.
- kwhitefoot 3y ago> the UK lacks a lot of protections of basic human rights, And there are plenty of people hell bent on removing those that remain.
- ProxCoques 3y agoThe Criminal Justice and Courts Act 2015 makes distributing intimate images without consent a crime in England and Wales. It prohibits sharing, or threatening to share, private sexual images of someone else without their consent and with the intent to cause distress or embarrassment to that person. I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole?
- oceanplexian 3y ago> I get that deepfakes are superimposing a non-consenting face on a (presumably) consenting body, but how does that matter? Is it that "visually lying" is a legal loophole? That's not how deepfakes work, since nothing is superimposed. Models like Stable Diffusion build a training set out of several images of a face (e.g. with something like LORA), and reduce it to a set of algorithms. That algorithm is a sophisticated way of expressing "green shirt, black hair, large chin, etc" It's not actually storing photos of faces. It then unmasks a new face out noise using those blueprints. You can change the blueprints, for example, I could ask SD to give me an image with gigantic rabbit ears. To a layperson it might look the same since the technology is so new, but it's not in any way analogous to a photograph, it's analogous to a painter who saw someone, wrote down some notes on a notepad, went to their studio and painted them. The painting might look realistic but it's not the same as the original photo.
- ProxCoques 3y agoOK but I'm not sure how any of that relates to why separate legislation is needed.
- RcouF1uZ4gsC 3y ago> The minister for victims and safeguarding, Laura Farris, said the bill sent the message that making deepfake images was ‘immoral and often misogynistic’ What is misogynistic is thinking that women's bodies are so shameful that using a computer algorithm to generate visual depictions of them without any interaction with the woman is inherently degrading or inherently harmful. Yes, they can be used for harassment. But so can a text message "XXX is a sl*t". So can a crude pencil sketch. The misogyny is in the intention and use, not any sense of realistically depicting the female body.
- chung8123 3y agoI wonder if look-a-like people can license their images for this or if you can license a deepfake and make it legal? I have a feeling things are going to get murky here. Hopefully we can all be mature and caring people but I have my doubts.
- JohnBooty 3y agoFirst let me say that I fully understand how the leaking of actual intimate photos/videos can be very devastating. No arguments there. But AI porn? There are so many gray areas and loopholes. This just feels so friggin' half-baked. It just feels like something that should be handled on a case by case basis. If somebody creates AI porn of somebody and uses it for clearly harmful purposes (blackmail, humilating somebody to their coworkers, whatever) then that should be punished. But, outlawing them altogether? We tolerate pornographic fanfiction and traditional pornographic illustrations of real adults, even though these might be upsetting to the people depicted therein. Therefore the issue with deepfakes is that not only are these things upsetting, but they might be mistaken for actual photos and videos. - What if I create deepfake porn of $FAMOUS_ACTRESS but it's sort of crappy, and nobody could reasonably mistake it for an actual photo of $FAMOUS_ACTRESS? - What if I create deepfake porn of $FAMOUS_ACTRESS and it's extremely photorealistic, but has clearly fantastical elements? (e.g. she is naked on the surface of the moon, she is having sex with Godzilla, etc) - What if I create deepfake porn of $FAMOUS_ACTRESS and it's realistic, but I've clearly indicated via caption or some other means that it's a synthetic image and not an actual photo of $FAMOUS_ACTRESS? - What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate out her face? - What if I create deepfake porn of $FAMOUS_ACTRESS and censor/pixelate the explicit parts? - How explicit is explicit? What if I've depicted $FAMOUS_ACTRESS in a swimsuit or other revealing outfit that is perfectly legal to wear in public? (reminder: going topless or even nude is legal in a lot of places too) - What if I've generated AI porn of $FAMOUS_ACTRESS that is based on publicly available images? eg: nude scenes from her movies, etc - What if I create deepfake porn and you can't tell who it is? Do we go on a search for the victim? - What if I create AI-generated porn and it accidentally looks like somebody real? - What if I create deepfake porn of $FAMOUS_ACTRESS but I never say it's her? - What if I create deepfake porn of $FAMOUS_ACTRESS but I change a detail or two? Perhaps I depict her with the "wrong" eye color, or add a scar, or something. So is this deepfake porn of "her" or more of a derivative work?
- chrisjj 3y ago"The creation of a deepfake image will be an offence regardless of whether the creator intended to share it" Surely the basis of a lucrative new extortion model. The most dangerous computer virus will be one that creates an image on your PC and alerts the extortionist.
- stonks 3y agoThe criminal offence refers to deepfakes of a real person without their consent. Actual people can be identified in this AI generated content, where the resemblance is close / beyond doubt. ---- The harm: Many people do not seem to understand the harm. Deepfakes are used for sexual pleasure, and used to degrade and dehumanize people resulting in victims getting traumatised. Victims often do not get enough support and the trauma may stay for a lifetime, and some victims may end their lives due to this. These victims get sexually harassed, e.g. one woman is continuously being harassed after someone generated deepfakes of her and people recognised her from those images https://www.reddit.com/r/legaladvice/comments/1c1pzjo/ai_generated_nudes_of_me_all_over_reddit_im_being/ https://www.reddit.com/r/legaladvice/comments/1c1pzjo/ai_gen... In r/singularity thread this is discussed and unfortunately many people think it's okay. https://www.reddit.com/r/singularity/comments/1c5c8pp/uk_criminalizes_creating_sexually_explicit/ https://www.reddit.com/r/singularity/comments/1c5c8pp/uk_cri... ---- Thoughtcrime: It is worth to discuss the 'thoughtcrime' where the generated content is not shared and where the deepfaked person is not harassed. Quote from the article: > The creation of a deepfake image will be an offence regardless of whether the creator intended to share it [..] • Revenge Porn does not make it illegal to store pornographic content as long as there was consent. However, Revenge Porn prohibits sharing no longer consensual content. • Sharing the generated content is not necessary in order to degrade, dehumanize and harass victims, thus still cause harm. On the other hand, if the deepfaked people are not harmed then it is a bit different, which may seem like a "thoughtcrime" and possibly may be. • People cannot guarantee the data they store won't get leaked. Both Revenge Porn and Deepfakes have the same issue. However, in Revenge Porn it is easier to see who created the content and enforce the law, while in deepfakes harder.
- bun_terminator 3y agothis very clearly crosses the line from "bad things are criminal" to "things i don't like are criminal"
- pieresqi 3y ago[dead]