17 ms·

Regardless of what I think about this technology in particular, I want to respond to this line from the second comment: [1] > on the contrary, it must be devel

• jolmg 4y ago
  "must" is probably the wrong word, but if they don't develop it, someone else will. At most, it would be delayed, and I'm not sure there's much difference in that.
  • jakelazaroff 4y ago
    Can we agree that “someone would have done this anyway” is a poor justification for one’s actions? That’s what you say when you 1) recognize that you did a bad thing, but 2) don’t want to be blamed for it.
    • jolmg 4y ago
      Can we agree that it's fundamentally not a bad thing to develop a technology? If you were a legislator, how would you like to criminalize the development of technology? You know, the most difficult part of this tech is probably the facial tracking, which is probably also used for the animation of 3d avatars. The jump is probably not huge.
      • MattRix 4y ago
        Many things can be unethical/immoral without being illegal, and of course there are some technologies that are fundamentally bad.
        • jolmg 4y ago
          Tech is such a malleable thing. Let's say that this repo doesn't exist, but repos exist for doing facial tracking on webcam feeds and pictures (for facial recognition), and others for modifying a 3d model according to a facial model output by facial trackers (for 3d avatars). Let's say that the tech around this DeepFaceLive repo develops to the point that the repo can become just a 10-line file of glue that makes a 3d model that's just a grid plane with a picture on it that moves according to a webcam feed. Such a repo is redundant and anyone that wants this tech can just apply the repos mentioned in the previous paragraph to do the equivalent. Are those repos now bad? If not, it seems the ethical question can be resolved by some restructuring of code so bad uses are not made so obvious, while being almost as easy.
        • sneak 4y ago
          What technologies are fundamentally bad? I don't know of any outside of perhaps torture devices.
          • lovemenot 4y ago
            There's surely at least some that you don't know of because their developer made the decision to not release.
          • blooalien 4y ago
            Every single "weapon of mass destruction" is fundamentally bad technology, just for one ultra-easy answer…
            • jolmg 4y ago
              The possibility that nuclear deterrence has helped the world to be more peaceful in comparison to the past means that it's not such an easy answer. Hasn't even been a century since nuclear weapons were invented though, so no clear statistics.
            • deleted 4y ago
              [deleted]
            • 4y ago
      • tgv 4y ago
        No, we can't. Developing "tech" is an action bound by morals like any other action. Cigarette production has tech in it. They don't just grow on trees. It was developed, and keeps being developed, by people well aware of the effects of smoking.
        • jolmg 4y ago
          You're opening another can of worms with that example. An adult, informed smoker that takes care to not cause second-hand smoking and that doesn't have any dependents, causes harm only to themselves. If they think that the reduction of stress or whatever else they gain at the tradeoff of their health is worth it, that's their choice. That's part of their bodily autonomy. As such, the tech is still not inherently bad.
          • tgv 4y ago
            You're conveniently dismissing a few concerns and from that drawing the conclusion it isn't inherently bad?
      • JohnFen 4y ago
        > Can we agree that it's fundamentally not a bad thing to develop a technology? I do not agree with this assertion.
    • prox 4y ago
      It’s simply that it’s a kids/teen level argument. “But they did it too!” Not a lot of thought went into it, is what it usually tells me.
      • jolmg 4y ago
        The argument wasn't "Doing something bad is fine because others are", it's "You're trying to stop a bad thing happening by stopping some people upstream from doing something that's perfectly fine. That's going to be ineffective." But you're right about me not putting a lot of thought. I didn't think this would need more arguing than that. I still believe that tech is fundamentally neutral, and as such so is its development. Even if a developer intends to give it bad use, the use and development are separate questions.
        • lovemenot 4y ago
          There are others - not you - who will justify their own actions thusly: the technology existed, its better that the good guys (me - of course) should use it before the bad guys do. In the space between both equivocations (yours and the other guy's), lies the potential for moral abuse.
          • JohnFen 4y ago
            That's an even more faulty argument, though, because almost everyone considers themselves one of "the good guys".
            • prox 4y ago
              Indeed, I think terming yourself in good vs bad as an we vs them terminology makes several biases go undiscovered that if you truly go for objectivity despite your sense of moral leaning is telling you.
          • jacooper 4y ago
            What even is a good guy use for Deep fakes besides some memes that get old fast?
    • entropi 4y ago
      I actually don't agree with this. Assume that the following premises are true: * action x is legal, but results in a worse world for everyone. * doing action x benefits the first 1000 people who do it. * much more than 1000 are capable of doing x, including me. * if I do it now, I will be one of the first 1000. If these are true, I think one should do action x. I believe one would be completely justified in doing it.
      • trashtester 4y ago
        Basically prisoner's dilemma. I think the answer is inherent to the question of whether it's "justified", and the fact that you're being downvoted. If Prisoner's Dilemma is played once, the optimal strategy is to defect. If it's repeated an unknown number of times, the optimal strategy is tit-for-tat. Now, if you're an identifiable actor in a pool of tit-for-tat-players, and you have a history of defecting first, you're going to face opponents who normally play tit-for-tat start to defect against you first, too. In the end, you will end up as one of the worst peformers in the player pool. So when you're asking if it is "justifiable" to defect in a game like this, it is the same as asking "will I be treated worse if I defect in this game". The downvote confirms this, I think. It means that in this population pool, bad behavior is punished (or people pretend it is) Now, if defecting can either be done secretly or if you're in a popluation pool where everyone defects anyway, then always-defect is probably the nash equillibrium. In that case, defecting will be seen as justifiable.
        • entropi 4y ago
          > Basically prisoner's dilemma > (...)In that case, defecting will be seen as justifiable. Exactly! My argument basically boiled down to this. In certain situations, defecting may be the way to go. I would not blame the defector for doing what is in their best interest. And “someone would have done this anyway” could very well be what's turning a situation into a prisoner's dilemma, so it could actually be a valid justification in some cases. But yeah, it could depend on what is meant by "justification" exactly.
          • harimau777 4y ago
    • courseofaction 4y ago
      It's a self-contradictory and bad-faith argument - it doesn't give another person the opportunity to say no, let alone to not do it.
    • KyeRussell 4y ago
      You’re speaking to the wrong audience. “Someone else would do it anyway” is how everyone here justifies doing unethical shit at FAANG and refusing to earn less than $300k/yr base because “that’s just the industry”. I have no respect for anyone that says this shit with a straight face.
      • KyeRussell 4y ago
        It should also be said that anyone that makes this argument immediately loses their right to complain when governments need to step in and regulate.
        • deleted 4y ago
          [deleted]
        • jolmg 4y ago
          You're commenting even after someone has already provided a legitimate good use of this tech, and are the third person to write as if it were established that it's unethical to develop this tech. I never said that it's fine to do unethical things because others will do it anyway, and I wish you wouldn't imply that I did. From the get-go, it was an open discussion whether the development was good/bad/neutral, and my original comment was to clarify that jakelazaroff's comment probably misinterpreted what they quoted from the use of "must". My position is that development is not unethical. I'm not trying to justify this position because I don't think it needs to be. When I said "others will do it anyway", it's not a justification and my original comment wasn't even about my position on this. My comment was referring to the usefulness of stopping the implicit always-ethically-neutral development of a tech in order to stop the potential misuse of it. I'm saying that even if they stopped the development of this one repo, or a few, or all others implementing this tech, it'd be ineffective. This repo is just one bit of the tech. Most of the tech is in its dependencies. If someone wants it and has nefarious uses in mind, they don't really really need this repo for that.
          • JohnFen 4y ago
            > My position is that development is not unethical. I think this is where we disagree at heart. I don't think that development is an ethically neutral act, any more than any other human activity is ethically neutral.
    • wincy 4y ago
      If I can become rich instead of someone else, and the damage will be done either way, it seems perfectly rational if I can avoid any actual legal punishment (I don’t care about people on Twitter being angry with me, Twitter isn’t real life) to push the big red button and reap the rewards. Once I’m rich and powerful I can afford to be ethical, isn’t that the way of billionaires anyway?
      • JohnFen 4y ago
        If a person isn't ethical in the first place (as proven by becoming wealthy through unethical means), they certainly aren't going to become ethical after they're rich.
  • pfortuny 4y ago
    Right. Tell that to Einstein, who just before Hiroshima realized it was not such a good argument. That is a nice piece of history. Ethics is important. And one consequence is: I may die but at least it was not because I made it possible.
    • jevgeni 4y ago
      Yeah, theory of relativity and tech with use cases mostly in porn and fraud aren't really the same thing.
      • pfortuny 4y ago
        It’s not relativity. Einstein pushed first for the american atom bomb because “the germans will make it anyway”. Then he realised that it was way too harmful and recanted. Nothing to do with relativity, just with “if I don’t do it others will”.
      • dmingod666 4y ago
        Your response doesn't make any sense. Care to elaborate?
        • GeorgeMD 4y ago
          Just search "Atrioc" in google and read any article. So far, this tech's main use is porn with people who do not want to do porn, who do not post adult content, who did not ever consent to their faces being used for adult content. The second use of this technology is fraud. Go read the issue and the comments linked in the post. And you should not even need this much info. The simple fact that they mention "mrdeepfake" as one of the 4 communication groups, the second after their Discord server tells you how much good this tech does to the world.
        • jevgeni 4y ago
          As GeorgeMD pointed out, there's some required reading.
          • 4y ago
• cwillu 4y ago
  From the main page: “Communication groups [...] • mrdeepfakes - the biggest NSFW English deepfake community”, which strains my assumption of good faith debate regarding their ethics.
  • sneak 4y ago
    Nothing about producing or consuming deepfakes suggests bad faith. You might not like the art, but deepfakes are not harmful.
    • actionfromafar 4y ago
      This is a shallow take, no matter which way you look at it. "Guns don't kill people."
    • pavlov 4y ago
      “NSFW deepfakes” literally means “porn with real people’s faces inserted.” Maybe that’s an art form, but it’s also clearly potentially harmful in the same way as revenge porn, teenage nude selfies, and other cases where regular people and porn intersect.
      • jl6 4y ago
        99% agree. The 1% is for the potential for deepfakes to solve blackmail. If producing deepfakes becomes trivial, any blackmail threat can be dismissed with “Whatever, I’ll just say it’s a deepfake”.
        • pjc50 4y ago
          This is not how humans work. False statements about someone damage that person, even once everybody knows they're false.
          • mrparrot 4y ago
            But with this argument you could blackmail someone whether or not you have some evidence.
• Kiro 4y ago
  We do in fact have a gun to our heads. I don't want to ruin your life but those who know know and it starts with R and B. Look it up at your own risk.
  • tremon 4y ago
    Come on, everyone knows that ruining your life starts with metal. R&B, while annoying, is quite safe.
• nix23 4y ago
  >Technology isn’t inevitable. It’s a choice we make. See and that's the problem, you can just talk about "you" not "we", because someone else will do it. Technology, and with it information cannot be stopped.
  • zzbzq 4y ago
    "I didn't murder that man, your honor, he would have died eventually anyway."
    • nix23 4y ago
      Incredible stupid comment...just to extend your logic the thing you type right now is "enabling" that "murdering"...so does the invention of the wheel enabling fast wars...or bring you fast into a hospital per ambulance.
• Felkomo 4y ago
  As you can't enforce stopping development of it, it is better to also develop it to understand it better and create countermeasures like ml which detects deepfakes. This also leads potentially to more or better image/video signing Technologie. And yes just because someone develops this in the open doesn't mean that someone else is not developing it in parallel but hidden.
  • jevgeni 4y ago
    There is ZERO credibility that this development will be used to "create a countermeasure", especially mrdeepfakes is a key communication channel there.
    • Felkomo 4y ago
      I'm not talking special to mrdeepfake. My argument still makes a point that you can't just forbid it. Someone else will do it anyway
      • jevgeni 4y ago
        Of course you can, even if somebody else does it. Case in point: any penal code.
        • Felkomo 4y ago
          How can you force stopping development of software?
          • jevgeni 4y ago
            One of the more egregious examples: https://www.wired.com/2011/03/aleynikov-sentencing/ https://www.wired.com/2011/03/aleynikov-sentencing/
• rbanffy 4y ago
  > No, it mustn’t. Let's imagine you have a horrific face malformation, or went through a terrible accident that left you disfigured. Why not allow you to pass for an ordinary person, at least in video calls, so you can have a somewhat normal life? It took me two minutes to come up with a legitimate use for this technology. I imagine there are many more only a couple minutes away.
  • blowski 4y ago
    I took the OP's comment as meaning "it's not necessary for this to happen" rather than "it's necessary for this not to happen", a nuance which can be confusing, especially for non-native English speakers. The comment in the article suggests there is no choice but for this technology to be developed, and I think the OP is disagreeing with that assessment and saying "there is a choice". If I've understood that right, both your comments can agree. There are strong arguments to say we _should_ develop this technology, but there also many good counter-arguments.
    • rbanffy 4y ago
      > there also many good counter-arguments. Someone will develop it, and someone will use it for nefarious purposes - as one terrorist organization spokesperson once said, "We only need to get lucky once. You need to get lucky every time". If it is developed like this, in the open, at least we have a better chance to understand its capabilities and limitations, and to be better equipped to build countermeasures.
      • blowski 4y ago
        This is exactly the Prisoner's Dilemma, and also shows up in arguments why we need to build insane numbers of nuclear weapons. The logic rests on a faulty assumption that there is a binary choice - either you build it this way, or somebody else builds it this way. It doesn't consider any other possible paths that could lead to better outcomes.
        • rbanffy 4y ago
• ilyt 4y ago
  For technology not to be developed, everyone with means to develop would need to choose not to. It's not too far from "inevitable"
  • jakelazaroff 4y ago
    Potential technology is an infinite set. A gigabyte contains 2^800,000,000 possibilities; a number billions of digits long. Many are junk or functionally duplicates — but many are not, and even out of that we choose a small subset that we think is worth pursuing. Saying “everyone with means to develop would need to choose not to” is like saying “every writer would need to choose not to write this book”. It’s not inevitable.
• low_tech_love 4y ago
  The problem is that there is a big confusion about building the technology and using it in some specific, wrong way. Surely anyone here will agree that building the technology itself should not be forbidden. You can do it for some valid reason, or simply for fun. It’s not for anyone to judge or stop you from doing it. Those who use it for some nefarious reason must then answer for that. Guns are the obvious comparison (although maybe not the best) as in they are needed for protection also. But another example is how explosive devices and technologies can be used in a variety of useful ways, such as mining, terraforming, etc. Using someone’s image without their consent is already a crime and applicable in many other non-DF scenarios.
  • vault_ 4y ago
    What's the confusion? The allegation is that choosing to build this represents, at best, incredibly poor judgment. The GitHub issue and GP post are good faith appeals to kindest possible interpretations of this work. To use your own example, explosives can be useful, but if you discovered that your neighbor was building pipe bombs in his garage, you'd probably want them to stop. "I'm not gonna use them. I just think they're neat." is not a convincing argument for them to continue. To extend the analogy a bit further, this seems comparable to discovering a box full of pipe bombs with a big "free" sign in front of their house.
    • low_tech_love 4y ago
      My analogies are horrible…
    • ngcazz 4y ago
      Plus... they're going to use it.
• suilied 4y ago
  I understand the sentiment but I disagree. It's just another disruptive piece of technology that people will have to adjust to. If you can't trust a digital face, then nobody will and using this tech for scamming purposes will simply fizzle out after an initial "adjustment period". I know that sounds rough but society might react differently than you think, i.e.: with digital faces being useless for identifying people, and maybe even becoming creepy or unsettling because of the implicated fakeness, meeting people in real life (opening bank accounts, transactions in general, anything where trust is valuable) will have more value. Don't fight progress in the hopes you can one day become complacent.
  • noAnswer 4y ago
    It will fizzle out right after fraud involving the telephone or email died out.
  • bayindirh 4y ago
    I don't see a difference between any technology. So if we can frown upon some nation for developing nukes, we can frown upon a group people for developing this. On the other hand somebody can say "if we don't, then others will". So regulate and ban the technology, then.
    • GoblinSlayer 4y ago
      > So regulate and ban the technology, then. Except for criminals and CIA.
    • grendelt 4y ago
      > So regulate and ban the technology, then. When has that ever worked? What banned technology is conceived of but not developed because some government entity said not to? I think the "if we don't, then others will" is part of the natural progression of technology. Whatever the next logical step of development is, that's where development efforts will flow. Some might not want to go there, but some will. Someone banning it will likely only fan the flames and drive more interest into the space - ala the "Streisand Effect". When the US government (et al) labeled certain numbers "illegal" because they could be used to break DRM or certain encryption types, academia and hackers alike openly mocked the notion. T-shirts, stickers, and websites sprung up further spreading this "illegal" knowledge. People who had no idea about how a number could be so "dangerous" suddenly wanted to know. By telling people they can't know or do something will absolutely drive people toward that knowledge. The hacker mentality often answers the question "why?" with "because I can". Saying you cannot only encourages more to jump in.
• Jemm 4y ago
  I disagree. We should develop this technology and make people aware of the implications and teach them to be vigilant.
  • kayodelycaon 4y ago
    Because being vigilant works so well now... Knowledge isn't useful if you can't apply it in practice. My work's IT has a habit of sending out phishing emails that closely match emails legitimate emails our employees and vendors send. We're supposed to be vigilant for looking at the Reply-To field, which is hidden by default on mobile devices. (As is the url things are going to.) I will never not trigger those. Why? My brain can't tap the header field for every single email I read. And I can't ignore legitimate emails. Every time on my phone, I'll click the link, then check the url in the browser. I can't get my brain to check before clicking the link. (Trust me, I'm neuro-divergent and I'd tried for years. It won't reprogram.)
• tenebrisalietum 4y ago
  I interpreted this as basically equivalent to "Someone will develop it somewhere."
• Zababa 4y ago
  > Technology isn’t inevitable. It’s a choice we make. I think it is inevitable actually. The history of humanity seems to suggest so at least.
• lumb63 4y ago
  Agreed. Convincing folks who think that of anything else is quite difficult. I’ll promote here one of my favorite reads, The Technological Society by Jacques Ellul. It feels very apropos here as we embark on an new smorgasbord of technologies that can do us harm and good. There will always be people who want to develop technologies even if they are harmful. We need not only ask ourselves “what is the benefit”, but “at what cost?”
• penjelly 4y ago
  > technology isnt inevitable i disagree, society could stop inventing and humanity would likely continue to exist. However, that would would require the population to stop being interested in the development of technology which i think wont happen. The point is, if this engineer doesnt write this code, someone else will. If a person can imagine a tool, someone will eventually want to make it a reality. i think deepfakes and ai generation are going to change the world in a way that im a bit leery of, it feels safer to just stop here, but that isnt a possibility, even if deepfakes are made illegal, people will still create their own tools and have their own harddrives for storage. maybe these things should be illegal or something, hard to say. But the tech will move forward either way imo.
  • rsr 4y ago
    We all have responsibilities as technologists to be ethical (to our own standards, at least), and to keep in mind the impact of the things we build. That applies regardless of whether other tech with a negative social impact exists, or for that matter whether it will exist. You can still be a good person, even if no one else is doing so, and you're under no pressure to be a bad person (in this context, anyway).