13 ms·

> Authors and contributing developers assume no liability and are not responsible for any misuse or damage caused by the use of this program. Anything that can

• davidguetta 4y ago
  Having morals when the other side doesn't is a weakness. Make the tools available to everyone. As Elon musk says, sun is the best disinfectant.
  • Planktonne 4y ago
    Having morals when the other side doesn't is the best possible reason to be on different sides.
    • davidguetta 4y ago
      Morals are cultural.. in some countries it means burning gay people. Power however, is amoral and neutral
• ls15 4y ago
  Unless we cut off the chain of responsibility somewhere, the creators of the programming language, the computer and its components, as well as the people who have designed the computer and those who mined the required minerals are responsible as well.
  • alexjurkiewicz 4y ago
    Good thing nobody is suggesting we *checks notes* jail silica miners for building the Deepfake Offensive Toolkit.
  • Planktonne 4y ago
    > Unless we cut off the chain of responsibility somewhere We can do that. We do it all the time, assigning varying levels of blame to different parties for different things. Determining between proximal and ultimate causes, or assigning more weight to one cause or another, is not some impossible burden.
    • deleted 4y ago
      [deleted]
• carnitine 4y ago
  That’s a legal disclaimer, not a moral one.
  • jacquesm 4y ago
    It is an attempt at a legal disclaimer. It may not hold up in court.
    • EMIRELADERO 4y ago
      Under what charges? If we're talking about the US, they have the first ammendment and all that. As for defamation laws and similar, the person who used the tool for those purposes would be on the hook, not the creators.
      • jacquesm 4y ago
        That entirely depends on the circumstances. But given you are mixing criminal, constitutional, and civil law (libel, sometimes also criminal) in one comment (oh, and it's 'amendment') I don't think that this will lead to a fruitful discussion. My only point was: you can disclaim anything you want, but that won't stop people from suing you if they feel like it (this can happen anyway, even if you disclaim any and all responsibility) and creating something that can very easily be abused and then thrown out into the public even though it is standard practice in the security community may well end up being a boomerang. There is some precedent for the creators of such software being sued: https://knowtechie.com/a-company-that-makes-iphone-hacking-software-is-being-sued-by-apple/ https://knowtechie.com/a-company-that-makes-iphone-hacking-s... So I would not be too quick to dismiss that this could happen to smaller entities or even open source creators of such tools.
    • t0mas88 4y ago
      It's the same as the age old "guns don't kill people" that seems to work fine for the weapons industry in the US.
      • jacquesm 4y ago
• wdroz 4y ago
  > If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. I don't agree, but people will never reach consensus on this moral topic. So please don't call the other side "at best naive".
  • Planktonne 4y ago
    If there's no duty to consider potential harm when releasing tools, then why would there be a duty to avoid criticising people who release them?
    • dividedbyzero 4y ago
      But if such things exist in secret and there's neither awareness nur a reference implementation to develop countermeasures against, wouldn't that also cause harm?
      • Planktonne 4y ago
        Absolutely, and there's definitely a strong argument to be made for increasing awareness of existing tools and attack vectors. Importantly though, that aim is served by proofs-of-concept and information sharing more than it is by releasing point-and-shoot attacks. This comment puts it rather well [1]. As above, I'm not saying this shouldn't have been created, or even that it shouldn't have been released. I am saying that there are important ethical considerations and questions raised that should be addressed directly by the creators, rather than being handwaved away as in the quoted disclaimer. [1] https://news.ycombinator.com/item?id=31652008 https://news.ycombinator.com/item?id=31652008
        • jcims 4y ago
          OpenAI attempted this with GPT-2’s incremental release strategy and was lambasted on moral and ethical grounds as well. https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ So individuals working on a technology like this have no clear societal consensus to use as a guide when making the decision. Strong arguments can be made in both directions.
• yieldcrv 4y ago
  All I learned from this response was to just release tools like this on Darknet Marketplaces and maybe Telegram and just forget the disclaimer
• krageon 4y ago
  This has a clear benign use. Of course it sucks that you can also use it in a hostile manner - but the fact that this tool is publicly available rather than hidden in the pocket of some unscrupulous blackhat means that every space that uses verification with these methods can now incorporate this type of testing. That's a net benefit for society. I do think disclaimers like this are a little juvenile (it reeks of a US-ian litigation mentality), but you can easily imagine why they put it there. Perhaps instead of the author being less naive, you need to be more empathetic.
  • tgsovlerkhgsel 4y ago
    More generically, I think there's a big difference between releasing proof of concepts, and fully weaponized tools. While the latter is also usable by red teams, it also gives attackers (who often wouldn't have the resources to build the same) the weapons they need. Personally, I like when people release proof of concepts, and hate it when they release weaponized tools. Especially when I inevitably end up reading reports where APT groups are using those tools.
    • ls15 4y ago
      Is a pen that contains ink a weaponized tool?
  • Planktonne 4y ago
    This is a tool with a clear benign use, along with a bunch of malign ones. If you create something with significant potential for harm, then you should at least think about that, the potential consequences, and possible mitigations. I wouldn't term it a failure of empathy to ask that people consider the impact of their actions on others. I totally get why they have the disclaimer, and I wouldn't ask them to remove it, but I don't think it's enough. Given the clear potential for harm here, the potential uses and misuses of this tool should have been addressed directly.
• RektBoy 4y ago
  What do you do? IMHO, only person naive here is you. People should be happy, that there are still white-hats who're reporting exploits, even for no profit. I personally switched to gray-market, can't take the shit we get from companies, anymore. This tool released publicly is only to bring an awareness to the topic. Everybody else who needed to exploit this, have these tools developed in private.
• tehbeard 4y ago
  So, if we take your naive take to its logical conclusion; The Apache Foundation should be considered partly responsible for all the malware distributed via their http software?
  • Planktonne 4y ago
    There is a conceptual difference between releasing a weapon (even if for research/defensive purposes) and releasing tools which could later be adapted negatively. That's not to say that weapons should never be created/released, but that there is an extra onus on the creators to at least consider the harm they are enabling and possible mitigations. The Apache Foundation - great example, thank you - despite not creating weapons, has clearly put thought into how their work interacts with wider society and how to ensure positive outcomes as far as possible [1]. People absolutely don't have to agree on moral issues, but it is irresponsible not to have considered them. [1] https://apache.org/theapacheway/index.html https://apache.org/theapacheway/index.html
  • IfOnlyYouKnew 4y ago
    That’s not a naive take but a stupid one, and it wasn’t made by OP. Slippery slope is a fallacy, not an argument.
• bongoman37 4y ago
  By this argument Linus Torvalds would be responsible for everything from child porn to drug marketplaces to nuclear weapons.
• barnabee 4y ago
  I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.
  • thrwawy283 4y ago
    I'm not saying it's a good idea to give everyone a gun, but I do like the argument that the disadvantaged have the same opportunity to pull a trigger. I hope humanity learns wisdom as quickly as we innovate. We've made it this far..
    • mavhc 4y ago
      And as well as not trusting that video of someone is really that person, I choose to live far away from places where anyone can get a gun
      • Vecr 4y ago
        Where anyone can get a gun? If you live in the middle of nowhere, lots of people around you will have guns (though they could be dozens of miles away in the best case), and obviously where there are more people the police can get guns, or call up the military if they can't for some reason (what country is that though?)
        • ciceryadam 4y ago
          Most UK police officers don't have a gun: An authorised firearms officer (AFO) is a British police officer who is authorised, and has been trained, to carry and use firearms. The designation is significant because in the United Kingdom most police officers do not routinely carry firearms, although they can be equipped with tasers. In 2019/20 fiscal year, there were 19,372 police operations throughout England and Wales in which the deployment of firearms was authorised and 6,518 firearms officers, 4.9% of the 132,467 active FTE officers. Following the November 2015 Paris attacks it was decided to significantly increase the numbers of armed officers, particularly in London. https://en.wikipedia.org/wiki/Authorised_firearms_officer https://en.wikipedia.org/wiki/Authorised_firearms_officer
• wiz21c 4y ago
  Reminds me of Oppenheimer: "I am become Death, the shatterer of worlds."
  • throwaway71271 4y ago
    you can hear Oppenheimer himself: https://www.youtube.com/watch?v=lb13ynu3Iac https://www.youtube.com/watch?v=lb13ynu3Iac
• stef25 4y ago
  What's the difference between this and Metasploit, sqlmap, ... ? Not saying you're wrong, it's just that while these tools have valid and legal uses (pentesting) they're also used in black hat scenarios and one could say the same about DOT.
  • jalk 4y ago
    Which is why Metasploit triggered the same discussion.
• ramblerman 4y ago
  > If you create something, make it freely accessible and easy to use, then I think you are partly responsible for its misuse. That's a dangerous precedent. Would you apply the same logic to a kitchen knife? Or if for some reason only freemium products count (not sure why), then a pentesting tool? I understand the underlying point you are trying to make but what are you proposing as an alternative exactly? Who gets to decide which products fall into a gray zone, whilst others are only for bad use. We already see this kind of shoddy thinking leading to keeping DALE-2 out of the public's hands (or at least that is their claim).
  • Francesco_deep 4y ago
    I sell airplanes do I am to feel responsible for the 9/11?
    • FollowingTheDao 4y ago
      Do you sell meth?
      • Francesco_deep 4y ago
        Yes, do you need it?
  • Planktonne 4y ago
    A knife is a multi-purpose tool that can be used as a weapon, and one that has so many important non-weapon uses that not having it would cause far more harm than having it would. This is intended primarily as a weapon, even when used defensively. There's an important qualitative difference there, though it's tempting to gallop down the slippery slope. Regardless, I'm not proposing to ban either knives or this tool - I've been very clear about that. I do think that - with anything that has potential for harm - it's important for people to consider the possible consequences and to actually engage in the discussion about usage, rather than either washing their hands of the issue or declaring that any consideration will soon lead to arresting everyone for everything. This is a path we've trodden countless times. With some things, we've collectively decided that no controls are necessary. With other things - poisons, nuclear weapons, a host of things depending on location - society has enacted controls of various efficacy and validity. The responsible choice, regardless of whether you end up being for or any any given restriction for [thing], is to spend time thinking about it, discussing it, and - particularly when you've chosen to release something - acknowledging the potential issues.
• bjt2n3904 4y ago
• A4ET8a8uTh0 4y ago
  Hmm. It is a tough one. I am closer to your line of thinking than not, but, at the same time, from where I sit, it seems that data monitoring went too far already and regular user has to have a way to circumvent it. For that reason alone, this tool, even if it will result in some abuse, it just evens out the playing field.
• forgingahead 4y ago
  We're actually super fortunate that the development of deepfake technologies have been done relatively out in the open, with source codes, concepts, and pre-trained models often being readily shared. This allows for a broader based understanding of what is possible, and then hopefully develop ways for folks to inoculate themselves, or at least have some societal-level resistance to being hoodwinked by them. If this tech was only developed in secret, and was being used in a targeted manner, who knows what kind of large-scale manipulations would be being undertaken.
• belter 4y ago
  Should Ron Rivest, Adi Shamir, and Leonard Adleman have responsibility if criminals use their discovery?
• systemvoltage 4y ago
  This is morally corrupt, dangerous and would lead to an oppressive, violent society. A knife maker killed for murders, a watch maker killed for the tyranny of time. We should do the exact opposite. Science and reason would cease to exist otherwise. Individuals wouldn't need to be held accountable, innovators/engineers/scientists/entrepreneurs would be. End of a free society. Have you thought of the chilling effect this might bring?
  • Planktonne 4y ago
    This is not a good faith argument. I've been extremely clear throughout this comment section - I don't want censorship, I don't think this should be banned, and nowhere have I called for legal consequences for releasing this. I want people to think about their actions, and to discuss the ethical issues arising from them. And yet you've jumped to calling me morally corrupt because I want to murder craftsmen. That's not a reasonable reading of my comments, or in any way a proportionate response. If you want to talk about chilling effects and the importance of science and reason, how would you describe your comments? You're shouting down discussion with wild accusations.
    • systemvoltage 4y ago
      I am discussing larger societal implications of what happens when this type of morality expands to the extreme. Absolutely zero to do with you personally. You're building a strawman. To you personally: If you're going around touting "other side's morality", you should be open to seeing what happens when this kind of thinking is adopted by the society. Don't play the victim game.
      • Planktonne 4y ago
        > A watch maker killed for the tyranny of time. This isn't so much a strawman as a slippery slope made from them. It is not reasonable to leap from 'people should acknowledge and discuss the ethics of their actions' to 'blood-drenched tyranny'.
        • 4y ago
• Hendrikto 4y ago
  If I stab someone with a knife, who is responsible? The inventor of knives? The knife‘s manufacturer? The store who sold me the knife? I would say the responsibility lays 100% with myself. I do not think it makes sense to pursue inventors for what happens to their creations, unless they actively encourage misuse.
  • FollowingTheDao 4y ago
    They arrest people for making and selling meth everyday. You have a bias because making deep fakes is not illegal yet. Knives are used for cooking, what alternative good do deep fakes provide?
    • DoItToMe81 4y ago
      Thousands of cheap special effects that would not otherwise be possible without a studio. Comparing this to meth, which has extreme detrimental effects on the human body, brain and the society that has to deal with the dangerous junkies it creates is completely absurd.
    • Hendrikto 4y ago
      Cardiovascular diseases are the number of cause of death worldwide BY FAR. Millions of people die every year from consuming fast food. Fast food does not really serve a purpose apart form being tasty/quick/cheap. You could fulfill your dietary needs much better with different food. Should McDonald's be banned? I do not think so. Anything can be misused. To answer your question about what "good" uses there are for deep fakes: Cheaper and better special effects, available to a MUCH broader audience than before. I cannot even imagine all of the great use cases creative people will find for this.
      • FollowingTheDao 4y ago
        > Should McDonald's be banned? Yes. I believe it should. Regarding the deep fakes being good, I meant societal good. Not just fun tacky things that some people have a good time with. What social value does it bring to the world? We can do everything we need to without deep fakes. It has no value to the world.
• golergka 4y ago
  You are a partly reason for misuse, but you are not morally responsible for it. Causal relationship doesn't necessarily imply a moral one.
• deleted 4y ago
  [deleted]
• solsane 4y ago
  I disagree. The moral responsibility really rests on the person who uses the tool. For instance, I once cheated by using gcc/godbolt to generate assembly output for a class from C code. By this logic, Richard Stallman should be blamed for my misconduct. There are any number of reasons for supplanting another face onto your own, many of which are simply good fun. If you choose to use this for scamming or perverted reasons, so be it. Moral posturing aside, perhaps there could be an invisible watermark or something included by default to easily identify less technically inclined actors as users of this tool.
• bufferoverflow 4y ago
  Do you think knife manufacturers are responsible for stabbings and knife attacks?
• ospray 4y ago
  The main value in releasing tools like this is to demonstrate weakness in our current security controls. A key weakness of biometrics is that there is no secret data. Open source tooling like this help people understand that.
  • polartx 4y ago
    I wish I didn’t have to scroll this far to find your rational perspective. Lets take the famous LockPickingLawyer of YouTube, is he responsible for every crime where the thief defeats a lock that he has demonstrated the weaknesses of? I would say “no!”. Exposing a weakness puts the onus of securing said weaknesses on those that sell technology/devices/services that market themselves as “secure”.
    • burrows 4y ago
      This is just some rationalization that security nerds like to regurgitate on each other.
      • polartx 4y ago
        Exposing a vulnerability in a public forum is the fastest way to patch said vulnerability. You don’t have to be a security nerd to connect the dots
        • burrows 4y ago
          > Exposing a vulnerability in a public forum is the fastest way to patch said vulnerability. Unjustified empirical claim. And even if it were true, it would still be insufficient for justifying public disclosure, classic is-ought. Also, only nerds stand up for security nerds.
• rockemsockem 4y ago
  Do you ask for the same level of culpability from a hammer manufacturer? Should hammer makers have trouble sleeping at night because someone bludgeoned another person to death with a hammer made by them? No. You don't.
• jlg23 4y ago
  > pretend that any misuse was entirely separate to you is at best naive. No, it is naive to pretend facial recognition is worth anything when creating tools to defeat it is a mere academic exercise in reading papers and implementing algorithms described in those. Don't shoot the messenger.
• deleted 4y ago
  [deleted]
  • deleted 4y ago
    [deleted]
• hegzploit 4y ago
  Same applies to a lot of security oriented tools, most notably metasploit which had these same accusations when it came out long ago, nowadays it's no big deal as exploitation frameworks are now more accessible. same applies to a lot of niche technologies that can be misused.
• disintegore 4y ago
  Is this the first pen test tool you've seen? No, they aren't responsible, morally or legally or however you want to slice it. At all. Not one bit.
• sedatk 4y ago
  Exposing these tools to broader use will accelerate the development of mitigations, which is a net win for regular users IMHO.
• _andrei_ 4y ago
  > then I think you are partly responsible for its misuse irrelevant
• trashtester 4y ago
  What authors _could_ do is to add some kind of secret watermark that would only be shared with select government agencies and perhaps software companies that could be trusted to keep it secret. That way, the software could be used for pen testing, but it could cause a silent trigger to go off. That could even be a way to monetize the software....
• kobalsky 4y ago
  this authentication mechanism is flawed and needs to go asap. these guys are doing something positive by speeding up the process.
• nurettin 4y ago
  It is not a moral statement, just lawbabble.