8 ms·

In Spain, girls report AI-generated nude photos of them being shared at school

21 points by agubelu 3y ago

palata 3y ago
I would not be super proud to have contributed to that kind of technology. Raises the point: "is technology always neutral"? At least those working in technology should regularly think about it.
- nik282000 3y ago
  Yes technology is neutral (exceptions being things like the hollow point bullet) but not all applications are neutral. Remember when cameras got small enough to be included in pocket sized devices? They were abused and spent some time at the front of the 24hr news cycle as the subject of a moral panic. Eventually laws were adjusted and some technological adjustment's were made and the same will happen with "AI". In this particular case the issue isn't even the technology it's parenting. The students generating these images are in their teens and are more than capable of understanding that using images of their classmates is not only immoral but also likely illegal. But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. Until 18, parents should be held equally responsible for the actions of their kids.
  - Miner49er 3y ago
    > Yes technology is neutral (exceptions being things like the hollow point bullet) This is a bad example, hollow points have a legitimate use in the prevention of collateral damage, among other things. Unless you mean it's an exception because it's clearly a good technology, but I'd also say that's not true. It falls under neutral.
    - Nevermark 3y ago
      Good point. Given most offensive weapons have a defensive purpose, both active and deterrent, their is a case for them being neutral. A bio weapon, near shore detonated nuclear device, or other doomsday devices that are designed to indiscriminately kill large numbers of civilians strike me as evil tech.
    - zolbrek 3y ago
      How dare you load bullets meant for killing in your gun! Don't you know you're supposed to shoot the bad guy in the leg?
  - palata 3y ago
    > Yes technology is neutral (exceptions being ...) I find it interesting how you contradict yourself in your very first sentence. "Technology is neutral, except when it isn't". To me it means that it is not neutral, doesn't it?
    - diffeomorphism 3y ago
      No, it does not mean that at all. Deserts are dry places except for rare occasions, when they can even be flooded. No contradiction just how language works.
      - palata 3y ago
        But then you missed my point. If I ask "Are deserts really always dry?" (in a context where you can reasonably assume that I understand the concept of a desert) and you answer "Yes they are, except on occasions where they aren't", then what you actually meant was "No, they aren't. Sometimes they can be flooded, but most of the time they are dry". I did not mean to ask "Is technology sometimes neutral?", but rather "Is technology always neutral, and in this case, is this particular technology neutral?". I did not write all that, but I assumed that it would be inferred from the context. And that, too, is how language works. Also note that my above reaction was not aggressive. I genuinely find it interesting that they think "technology is neutral" but start their explanation with a sentence that clearly shows that they think technology is not neutral.
        diffeomorphism 3y ago
        No, I did not miss it but told you that people and language don't work that way. In common language there is no difference between "Yes, almost always, except.." and "No, not 100% always, there are some very rare exceptions". That sounds contradictory at first because one starts with yes and one with no, but both say the same thing and people are perfectly fine with shortening "90% true" to true. The reason for that is that your question is kinda trivial to answer and kills any discussion for no gain of insight. "Is A literally, always B with absolutely no exceptions?". No, unless you talk about mathematics you don't get absolute statements.
  - giantg2 3y ago
    I fail to see the points you are trying to make. Ok, so some laws were adjusted around the use of small cameras. Yet there are still issues today. I mean, you have to actually catch someone using it. How many people are redlight scanning their airbnbs? Many people breaking the law just aren't getting caught. I fail to see how the laws really fixed anything. All that happened is people learned to live in this new reality and the news stopped reporting it because it's not new. I'm not sure what you're getting at with the hollowpoint comment. In theory it could be applied for evil (murder) or could be applied for good (self defense, public defense, food, etc depending on one's beliefs about what is good vs evil).
  - isk517 3y ago
    > But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. 20 years ago long before iPads when I was in high school there were definitely kids that would have done this exact same thing if the technology was available. Lets not pretend as though lose morals are a recent invention.
- sim7c00 3y ago
  I think technology isnt neutral, and people creating tech that can cause harm and subsequently making it available to the general public and easy to use are as immoral as the people using it. i love writing malware, but once it works, i delete it. If you want to see if you can create an AI that can make nudes of people thats fine, you can use it for yourself. or when it works decide atleast not to publish it. theres arguments for publishing it, and people ofcourse in favor of those arguments. thats ok, i am not though. i think its bad. you can write a paper about it and show some benign examples in it if you want the world to know these things are possible. there is no need imho to hand everyone a gun simply to prove guns are dangerous, and that is akin to what these tech creators are doing..
  - palata 3y ago
    I tend to agree. I feel like in today's world (where society may well collapse very soon because of what we did with technology - I'm thinking about climate and biodiversity), we engineers should think much more about what we do and the consequences it may have on the world.
- Miner49er 3y ago
  I agree, but I think deep fake type technology is neutral. Some technology isn't. Deep fakes aren't the issue here. This same thing could have been done before AI with Photoshop or even scissors and glue. If any technology is at fault here, it's social media, because that's likely where whoever is making these is finding the pictures of the girls to feed the AI.
  - sim7c00 3y ago
    you can also kill someone with your hands or a knife, that doesnt make guns less deadly. perhaps theres good use cases for deep fake tech, but realising people will use technologies for their own use cases instead the one you envisioned i think is important. That being said, this is a difficult topic and i suppose both sides xn have valid arguments. guns also save lives for instance. (sorry for keepong to use the gun example but i think ite easy relatable maybe a not a perfect example)
  - palata 3y ago
    > I agree, but I think deep fake type technology is neutral. Some technology isn't. That's probably a valid argument, but you should elaborate. What does this particular technology bring to society that balances for deep fakes that can destroy someone's life? Cryptography can protect people, which sounds like a good thing, but it can protect bad actors, which is a bad thing. One could debate it, but overall it feels like cryptography is useful in many cases. On the other hand, I feel like we would be better off without e.g. blockchain or without e.g. SpaceX making space flights affordable for businesses in a world where the climate crisis may soon destroy us. What about deep fakes then? What good does it bring that would balance the bad use-cases?
- pixl97 3y ago
  "that kind of technology" I have a feeling you're going to have a problem with any kind of "general" technology in the future. Or, lets change the initial premise this this Heinlein quote >"Specialization is for insects," the science fiction author Robert A. Heinlein famously wrote. "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly." The thing is with a generalized image creation software is it will do things you won't like. Be that representation of minors, or offending your king/glorious leader, religious sacrilege, or whatever it is you don't like. Generalized software, much like humans, is capable of heresy.
  - palata 3y ago
    > I have a feeling you're going to have a problem with any kind of "general" technology in the future. I can agree to the argument that "everything can be used in a bad way". But that is not my point. My point is: is any technology always worth existing? As a thought experiment, I can easily imagine a technology that would only be useful for torture, and I would consider it fundamentally evil. To the point where I would not want this technology to exist at all. If you agree with that, then it means that it is possible to imagine a technology that should not exist. And consequently, it raises the question for every new technology: should this one exist, or not?
    - pixl97 3y ago
      There is a big difference between having a degree in theoretical physics and a theoretical degree in physics. In real life, the ones that humans actually occupy true black and white situations almost never exist. Your last sentence will always fall afoul of "In my moral framework" and since moral frameworks differ between cultures and individuals you will always find disagreement on the question of worth. The idea of fundamental evil is not a scientific measurable concept. To turn around your concept and reverse the danger. I see any non-Christian* as fundamentally evil and it is my moral and just duty to remove any non-believers from this planet in order to preserve the sanctity of what is good. At best we can make a rule of laws at attempt to fairly balance the needs of others and attempt to deter actions that are so dangerous for society they cannot be allowed. But never believe that your rule of law is universal. * replace with whatever religion/belief system suits your needs.
- giantg2 3y ago
  'Raises the point: "is technology always neutral"?' Is technology ever neutral by those standards? Seems like anything that has legitimate uses could be used for evil too.
  - palata 3y ago
    > Seems like anything that has legitimate uses could be used for evil too. Ok, let's assume that anything that has legitimate uses can be used for evil. But what if the technology is mostly useful for evil? Would you call it neutral?
    - giantg2 3y ago
      Of course the technology is still neutral. If there's lopsided use, then it points to a people problem.
      - palata 3y ago
        Wait I did not say "what if it is mostly used for evil", but "what if it is mostly useful for evil". Say a torture machine. Would you consider that it is perfectly neutral to build a torture machine, and that the problem lies with those who use it?
        giantg2 3y ago
        Your differentiation makes no difference - a machine that is mostly useful for evil will end up mostly used for evil. Your example of a torture machine would depend on if you consider any scenarios where torture would be justified/good/useful. If it does have legitimate uses, then there should be no problem creating it and trying to limit its use. This is where your analogy breaks down though. You can't just cp/ctrl-c a torture machine like you can data/software. Restricting software and its use becomes much more difficult when we talk about stuff like deep fakes while trying to balance liberty.
        palata 3y ago
lrvick 3y ago
It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. There are potentially a couple of positive social outcomes that may emerge from this trend though. 1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online. 2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.
- ToucanLoucan 3y ago
  > It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy. > It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels. Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world. And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.

21 points by agubelu 3y ago

I would not be super proud to have contributed to that kind of technology. Raises the point: "is technology always neutral"? At least those working in technology should regularly think about it.

nik282000 3y ago
Yes technology is neutral (exceptions being things like the hollow point bullet) but not all applications are neutral. Remember when cameras got small enough to be included in pocket sized devices? They were abused and spent some time at the front of the 24hr news cycle as the subject of a moral panic. Eventually laws were adjusted and some technological adjustment's were made and the same will happen with "AI". In this particular case the issue isn't even the technology it's parenting. The students generating these images are in their teens and are more than capable of understanding that using images of their classmates is not only immoral but also likely illegal. But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. Until 18, parents should be held equally responsible for the actions of their kids.
- Miner49er 3y ago
  > Yes technology is neutral (exceptions being things like the hollow point bullet) This is a bad example, hollow points have a legitimate use in the prevention of collateral damage, among other things. Unless you mean it's an exception because it's clearly a good technology, but I'd also say that's not true. It falls under neutral.
  - Nevermark 3y ago
    Good point. Given most offensive weapons have a defensive purpose, both active and deterrent, their is a case for them being neutral. A bio weapon, near shore detonated nuclear device, or other doomsday devices that are designed to indiscriminately kill large numbers of civilians strike me as evil tech.
  - zolbrek 3y ago
    How dare you load bullets meant for killing in your gun! Don't you know you're supposed to shoot the bad guy in the leg?
- palata 3y ago
  > Yes technology is neutral (exceptions being ...) I find it interesting how you contradict yourself in your very first sentence. "Technology is neutral, except when it isn't". To me it means that it is not neutral, doesn't it?
  - diffeomorphism 3y ago
    No, it does not mean that at all. Deserts are dry places except for rare occasions, when they can even be flooded. No contradiction just how language works.
    - palata 3y ago
      But then you missed my point. If I ask "Are deserts really always dry?" (in a context where you can reasonably assume that I understand the concept of a desert) and you answer "Yes they are, except on occasions where they aren't", then what you actually meant was "No, they aren't. Sometimes they can be flooded, but most of the time they are dry". I did not mean to ask "Is technology sometimes neutral?", but rather "Is technology always neutral, and in this case, is this particular technology neutral?". I did not write all that, but I assumed that it would be inferred from the context. And that, too, is how language works. Also note that my above reaction was not aggressive. I genuinely find it interesting that they think "technology is neutral" but start their explanation with a sentence that clearly shows that they think technology is not neutral.
      - diffeomorphism 3y ago
        No, I did not miss it but told you that people and language don't work that way. In common language there is no difference between "Yes, almost always, except.." and "No, not 100% always, there are some very rare exceptions". That sounds contradictory at first because one starts with yes and one with no, but both say the same thing and people are perfectly fine with shortening "90% true" to true. The reason for that is that your question is kinda trivial to answer and kills any discussion for no gain of insight. "Is A literally, always B with absolutely no exceptions?". No, unless you talk about mathematics you don't get absolute statements.
- giantg2 3y ago
  I fail to see the points you are trying to make. Ok, so some laws were adjusted around the use of small cameras. Yet there are still issues today. I mean, you have to actually catch someone using it. How many people are redlight scanning their airbnbs? Many people breaking the law just aren't getting caught. I fail to see how the laws really fixed anything. All that happened is people learned to live in this new reality and the news stopped reporting it because it's not new. I'm not sure what you're getting at with the hollowpoint comment. In theory it could be applied for evil (murder) or could be applied for good (self defense, public defense, food, etc depending on one's beliefs about what is good vs evil).
- isk517 3y ago
  > But when left to raise themselves using the iPad-Babysitter you end up with the moral free for all that we are seeing today. 20 years ago long before iPads when I was in high school there were definitely kids that would have done this exact same thing if the technology was available. Lets not pretend as though lose morals are a recent invention.
sim7c00 3y ago
I think technology isnt neutral, and people creating tech that can cause harm and subsequently making it available to the general public and easy to use are as immoral as the people using it. i love writing malware, but once it works, i delete it. If you want to see if you can create an AI that can make nudes of people thats fine, you can use it for yourself. or when it works decide atleast not to publish it. theres arguments for publishing it, and people ofcourse in favor of those arguments. thats ok, i am not though. i think its bad. you can write a paper about it and show some benign examples in it if you want the world to know these things are possible. there is no need imho to hand everyone a gun simply to prove guns are dangerous, and that is akin to what these tech creators are doing..
- palata 3y ago
  I tend to agree. I feel like in today's world (where society may well collapse very soon because of what we did with technology - I'm thinking about climate and biodiversity), we engineers should think much more about what we do and the consequences it may have on the world.
Miner49er 3y ago
I agree, but I think deep fake type technology is neutral. Some technology isn't. Deep fakes aren't the issue here. This same thing could have been done before AI with Photoshop or even scissors and glue. If any technology is at fault here, it's social media, because that's likely where whoever is making these is finding the pictures of the girls to feed the AI.
- sim7c00 3y ago
  you can also kill someone with your hands or a knife, that doesnt make guns less deadly. perhaps theres good use cases for deep fake tech, but realising people will use technologies for their own use cases instead the one you envisioned i think is important. That being said, this is a difficult topic and i suppose both sides xn have valid arguments. guns also save lives for instance. (sorry for keepong to use the gun example but i think ite easy relatable maybe a not a perfect example)
- palata 3y ago
  > I agree, but I think deep fake type technology is neutral. Some technology isn't. That's probably a valid argument, but you should elaborate. What does this particular technology bring to society that balances for deep fakes that can destroy someone's life? Cryptography can protect people, which sounds like a good thing, but it can protect bad actors, which is a bad thing. One could debate it, but overall it feels like cryptography is useful in many cases. On the other hand, I feel like we would be better off without e.g. blockchain or without e.g. SpaceX making space flights affordable for businesses in a world where the climate crisis may soon destroy us. What about deep fakes then? What good does it bring that would balance the bad use-cases?
pixl97 3y ago
"that kind of technology" I have a feeling you're going to have a problem with any kind of "general" technology in the future. Or, lets change the initial premise this this Heinlein quote >"Specialization is for insects," the science fiction author Robert A. Heinlein famously wrote. "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly." The thing is with a generalized image creation software is it will do things you won't like. Be that representation of minors, or offending your king/glorious leader, religious sacrilege, or whatever it is you don't like. Generalized software, much like humans, is capable of heresy.
- palata 3y ago
  > I have a feeling you're going to have a problem with any kind of "general" technology in the future. I can agree to the argument that "everything can be used in a bad way". But that is not my point. My point is: is any technology always worth existing? As a thought experiment, I can easily imagine a technology that would only be useful for torture, and I would consider it fundamentally evil. To the point where I would not want this technology to exist at all. If you agree with that, then it means that it is possible to imagine a technology that should not exist. And consequently, it raises the question for every new technology: should this one exist, or not?
  - pixl97 3y ago
    There is a big difference between having a degree in theoretical physics and a theoretical degree in physics. In real life, the ones that humans actually occupy true black and white situations almost never exist. Your last sentence will always fall afoul of "In my moral framework" and since moral frameworks differ between cultures and individuals you will always find disagreement on the question of worth. The idea of fundamental evil is not a scientific measurable concept. To turn around your concept and reverse the danger. I see any non-Christian* as fundamentally evil and it is my moral and just duty to remove any non-believers from this planet in order to preserve the sanctity of what is good. At best we can make a rule of laws at attempt to fairly balance the needs of others and attempt to deter actions that are so dangerous for society they cannot be allowed. But never believe that your rule of law is universal. * replace with whatever religion/belief system suits your needs.
giantg2 3y ago
'Raises the point: "is technology always neutral"?' Is technology ever neutral by those standards? Seems like anything that has legitimate uses could be used for evil too.
- palata 3y ago
  > Seems like anything that has legitimate uses could be used for evil too. Ok, let's assume that anything that has legitimate uses can be used for evil. But what if the technology is mostly useful for evil? Would you call it neutral?
  - giantg2 3y ago
    Of course the technology is still neutral. If there's lopsided use, then it points to a people problem.
    - palata 3y ago
      Wait I did not say "what if it is mostly used for evil", but "what if it is mostly useful for evil". Say a torture machine. Would you consider that it is perfectly neutral to build a torture machine, and that the problem lies with those who use it?
      - giantg2 3y ago
        Your differentiation makes no difference - a machine that is mostly useful for evil will end up mostly used for evil. Your example of a torture machine would depend on if you consider any scenarios where torture would be justified/good/useful. If it does have legitimate uses, then there should be no problem creating it and trying to limit its use. This is where your analogy breaks down though. You can't just cp/ctrl-c a torture machine like you can data/software. Restricting software and its use becomes much more difficult when we talk about stuff like deep fakes while trying to balance liberty.
        palata 3y ago

lrvick 3y ago

It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. There are potentially a couple of positive social outcomes that may emerge from this trend though. 1. everyone who sees a deepfake of themselves will be forever viscerally skeptical of any claim relying only on photos and videos socially shared online. 2. everyone who sees actual nude photos or videos of people they know will be skeptical they are real, thus taking away the power of revenge porn or ransom porn.

ToucanLoucan 3y ago
> It is inevitable that basically everyone in the next generation will have deepfake porn made of them by some asshat classmate. It is about as hard to stop as someone creating an nude painting of someone from imagination. This is just casually dropped as though a photo-realistic AI image == an amateur painting, in production quality, in required effort and time, as though they're even remotely comparable. Which by the by, I also think the amateur painting should be punished as making nude pictures of people you don't get permission from is creepy. > It is not a nice thing to do, and really mean if shared in public, but the types of measures it would take to stop it are incompatible with a free society. I do not wish to live in a free society then. This is patently absurd. We also cannot stop people murdering each other, stealing from each other, or otherwise causing harm, but we damn well punish it when it happens (when our useless judiciary system can be bothered anyway but that's a different conversation) and that people are just like "well I guess kids now have to worry about classmates making photo realistic depictions of them performing sex acts" is so horrifying. So horrifying, on so many levels. Is this what we're building? Is this the society we all envisioned as kiddos getting into the Internet and our computers? I think we can do so very, very much better than this and it makes me incredibly depressed seeing what we're permitting to be done with the tools we make. Everyone involved in this should be ashamed. I don't know how I'd live with myself knowing I played even a small role in ushering in entire new vectors of sexual harassment directed at people who already struggle to get a modicum of justice in this world. And for what? "Freedom?" My right to swing my fist ends when it contacts your face, but up until then, you can't do a thing? You just get to have dozens if not hundreds of people swinging at you faking you out every minute of your life and you just have to sit there and take it, because they technically haven't hit you yet? Bollocks. Absolute nonsense. This society might be "free" but it cannot function and the longer we delay in gaining this understanding the more people will be driven out of their minds waiting for us to do so.