7 ms·

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose wh

I’d argue there is a moral imperative to create and release tools like this as free and open source software so that anyone has access to them and can choose whether to use them, rather than only sophisticated and well resourced adversaries. IMO the creators should feel good about their actions, even if they feel bad or apprehensive about the direction of the world because this technology exists at all.

thrwawy283 4y ago
I'm not saying it's a good idea to give everyone a gun, but I do like the argument that the disadvantaged have the same opportunity to pull a trigger. I hope humanity learns wisdom as quickly as we innovate. We've made it this far..
- mavhc 4y ago
  And as well as not trusting that video of someone is really that person, I choose to live far away from places where anyone can get a gun
  - Vecr 4y ago
    Where anyone can get a gun? If you live in the middle of nowhere, lots of people around you will have guns (though they could be dozens of miles away in the best case), and obviously where there are more people the police can get guns, or call up the military if they can't for some reason (what country is that though?)
    - ciceryadam 4y ago
      Most UK police officers don't have a gun: An authorised firearms officer (AFO) is a British police officer who is authorised, and has been trained, to carry and use firearms. The designation is significant because in the United Kingdom most police officers do not routinely carry firearms, although they can be equipped with tasers. In 2019/20 fiscal year, there were 19,372 police operations throughout England and Wales in which the deployment of firearms was authorised and 6,518 firearms officers, 4.9% of the 132,467 active FTE officers. Following the November 2015 Paris attacks it was decided to significantly increase the numbers of armed officers, particularly in London. https://en.wikipedia.org/wiki/Authorised_firearms_officer https://en.wikipedia.org/wiki/Authorised_firearms_officer
Planktonne 4y ago
> there is a moral imperative to create and release tools like this as free and open source software so that _anyone_ has access to them That's an argument I have a lot of time for, but it needs to be made, rather than - as at present - having the whole issue sidestepped. With any tool like this, I think there's a need for ethical due diligence as much as there is for engineering, and there's no evidence that this aspect has been considered at all.
- Reubensson 4y ago
  To me it sounds like it boils down to either you release the software or you don't. If you decide to release it into public, I don't think there is much you can realistically do to prevent abuse.
  - Planktonne 4y ago
    Release/don't release is definitely the choice with the highest impact, but it's not the only thing that can be considered. Who you release things to, in what form, with what framing - all of these impact how something will be used. You can never eliminate bad actors, but you can encourage responsible use.
    - dr_zoidberg 4y ago
      If you know to use Linux properly you can tamper with digital evidence and make a mess for a digital forensics investigator. The thing is, despite Linux making it easier than Windows, it doesn't come with a "here's how to erase digital evidence!" manual[0]. It's "left as an exercise to the reader". Probably same on macOS, but it's (mostly) not open source, so you can't really argue about it being an open source thing that lets you do evil stuff. You could argue about its availability though. And it boils down to the point you (and others) were making: these are tools, the ethics are in how you use them. [0] I know there are some forensic distros that come with anti-forensic tools. Still, Linux in general is the environment that allows that kind of thing to work in the first place, going back to the original point of whether it should be open source or not.
ethanbond 4y ago
Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.
- rlayton2 4y ago
  No, but it does apply to the distribution of slippery slopes. The OP specifically said tools "like this"
  - ceejayoz 4y ago
    Some slopes are slipperier than others.
  - ethanbond 4y ago
    I guess I don’t understand what “like this” means. While it’s convenient to dismiss any discussion about A and something slightly similar to A as “slippery slope,” an inconvenient fact is that the slope often is slippery. It’s increasingly evident to anyone paying attention that information warfare can yield disastrous effects in a variety of ways: disrupting peaceful transition of power; disrupting critical infrastructure; disrupting command and control of ”traditional” WMD systems, even. So that’s sort of the spirit of my question: how far does this argument go? What is the definition of “like this?”
- rolisz 4y ago
  For nuclear weapons, even if plans were publicly available, getting the right kind of uranium would still be a major hurdle for 99.9% of the people.
  - FollowingTheDao 4y ago
    But what are the consequences of the .1% of people who get over that hurdle?
IfOnlyYouKnew 4y ago
I can’t see a way this technology could be used defensively. Wide access just leads to more abuse. There’s no principle by which it serves some sort of justice to make some crime more accessible. It’s surprising how often this argument is made compared to, say, equal-opportunity access to fancy cars or good housing.
- bjt2n3904 4y ago
  They literally have a demonstration of the tool being used to evade an intrusive "send us a video of your face, turn your head both ways, blink". That isn't a defensive use!?
  - mensetmanusman 4y ago
    Definitely useful for scammers.
    - bjt2n3904 4y ago
      Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going "nooo, unpublish this" doesn't protect people from scammers, who will continue to use it. Society must unlearn that seeing is believing. That is what is morally irresponsible, not this software.
      - Planktonne 4y ago
        I don't think anyone so far has called for this to be unpublished, just for it to be more carefully considered and for the ethical aspect to be addressed when publishing. That's part of 'building a culture of distrust'. Knee-jerk refusal to even consider thinking/talking about the use of technology is far more irresponsible than any other stance.
bko 4y ago
Honest question, do you apply the same reasoning to gun control?
- barnabee 4y ago
  Good question. I would apply that reasoning to the information about how a gun works for sure, but IMO there is some (admittedly hard to define) amount of danger or perhaps destructive power beyond which we might or perhaps should choose as a society to restrict the ability to wield a tool and certainly at least some guns exceed that limit for me. I’m not sure that any software, on its own, does though, at least I can’t think of any immediate examples. I admit that this limit is hard to define. It’s also extremely hard to enforce for pure data, if that turns out to be what we want to do.
  - bko 4y ago
    I agree its a very hard line to draw. But I think certain technology is more contagious and dangerous. For instance, a single person with a gun can inflict damage locally with dire consequences on his or her own personal safety in the process. But if someone has a virus (physical or digital), then it caries a kind of contagion that can disproportionately hurt many more people. That's why we can't equate something like deaths due to car accidents to pandemic deaths. A single car accident doesn't increase the risk of all car accidents. It's localized. But a single infection increases the risk of future infections due to contagion. Whether you think any technology falls into the dangerous contagious category or not is up for debate.
  - Scea91 4y ago
    In this analogy information about how gun works would be an equivalent of Deepfake research paper. This tool is analogous to an actual gun.
hackernewds 4y ago
This sounds like it's making the gun control argument regularly made in America, and I disagree with it vehemently
bergenty 4y ago
formerkrogemp 4y ago
We open sourced 3d printed AK-47s. We disclaim any responsibility for the ensuing mass shootings. We produced the software to run the drones. We didn't personally deploy them to kill those children. We built software to guide the rockets. We didn't personally fire them at those hospitals. We chose to startup our company providing profiles of suspects using scraped, publically available data to any government agency. What they do with the information isn't our problem. We wrote the software, but it's not our fault or problem what happens because of who uses and because of how they use it. Our intent was good. The market and demand was there. What's wrong with providing a supply? How many software creators feel culpable? Vanishingly few. Who cares when comp is high?
- krapp 4y ago
  "Once the rockets are up, who cares where they come down? That's not my department!" says Wernher von Braun

thrwawy283 4y ago

I'm not saying it's a good idea to give everyone a gun, but I do like the argument that the disadvantaged have the same opportunity to pull a trigger. I hope humanity learns wisdom as quickly as we innovate. We've made it this far..

mavhc 4y ago
And as well as not trusting that video of someone is really that person, I choose to live far away from places where anyone can get a gun
- Vecr 4y ago
  Where anyone can get a gun? If you live in the middle of nowhere, lots of people around you will have guns (though they could be dozens of miles away in the best case), and obviously where there are more people the police can get guns, or call up the military if they can't for some reason (what country is that though?)
  - ciceryadam 4y ago
    Most UK police officers don't have a gun: An authorised firearms officer (AFO) is a British police officer who is authorised, and has been trained, to carry and use firearms. The designation is significant because in the United Kingdom most police officers do not routinely carry firearms, although they can be equipped with tasers. In 2019/20 fiscal year, there were 19,372 police operations throughout England and Wales in which the deployment of firearms was authorised and 6,518 firearms officers, 4.9% of the 132,467 active FTE officers. Following the November 2015 Paris attacks it was decided to significantly increase the numbers of armed officers, particularly in London. https://en.wikipedia.org/wiki/Authorised_firearms_officer https://en.wikipedia.org/wiki/Authorised_firearms_officer

Planktonne 4y ago

> there is a moral imperative to create and release tools like this as free and open source software so that _anyone_ has access to them That's an argument I have a lot of time for, but it needs to be made, rather than - as at present - having the whole issue sidestepped. With any tool like this, I think there's a need for ethical due diligence as much as there is for engineering, and there's no evidence that this aspect has been considered at all.

Reubensson 4y ago
To me it sounds like it boils down to either you release the software or you don't. If you decide to release it into public, I don't think there is much you can realistically do to prevent abuse.
- Planktonne 4y ago
  Release/don't release is definitely the choice with the highest impact, but it's not the only thing that can be considered. Who you release things to, in what form, with what framing - all of these impact how something will be used. You can never eliminate bad actors, but you can encourage responsible use.
  - dr_zoidberg 4y ago
    If you know to use Linux properly you can tamper with digital evidence and make a mess for a digital forensics investigator. The thing is, despite Linux making it easier than Windows, it doesn't come with a "here's how to erase digital evidence!" manual[0]. It's "left as an exercise to the reader". Probably same on macOS, but it's (mostly) not open source, so you can't really argue about it being an open source thing that lets you do evil stuff. You could argue about its availability though. And it boils down to the point you (and others) were making: these are tools, the ethics are in how you use them. [0] I know there are some forensic distros that come with anti-forensic tools. Still, Linux in general is the environment that allows that kind of thing to work in the first place, going back to the original point of whether it should be open source or not.

ethanbond 4y ago

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

rlayton2 4y ago
No, but it does apply to the distribution of slippery slopes. The OP specifically said tools "like this"
- ceejayoz 4y ago
  Some slopes are slipperier than others.
- ethanbond 4y ago
  I guess I don’t understand what “like this” means. While it’s convenient to dismiss any discussion about A and something slightly similar to A as “slippery slope,” an inconvenient fact is that the slope often is slippery. It’s increasingly evident to anyone paying attention that information warfare can yield disastrous effects in a variety of ways: disrupting peaceful transition of power; disrupting critical infrastructure; disrupting command and control of ”traditional” WMD systems, even. So that’s sort of the spirit of my question: how far does this argument go? What is the definition of “like this?”
rolisz 4y ago
For nuclear weapons, even if plans were publicly available, getting the right kind of uranium would still be a major hurdle for 99.9% of the people.
- FollowingTheDao 4y ago
  But what are the consequences of the .1% of people who get over that hurdle?

IfOnlyYouKnew 4y ago

I can’t see a way this technology could be used defensively. Wide access just leads to more abuse. There’s no principle by which it serves some sort of justice to make some crime more accessible. It’s surprising how often this argument is made compared to, say, equal-opportunity access to fancy cars or good housing.

bjt2n3904 4y ago
They literally have a demonstration of the tool being used to evade an intrusive "send us a video of your face, turn your head both ways, blink". That isn't a defensive use!?
- mensetmanusman 4y ago
  Definitely useful for scammers.
  - bjt2n3904 4y ago
    Then build a cultural revolution that instills distrust in what you see. We already teach the elderly not to trust a link that says, "you are the millionth visitor, click to win your prize". Teach them also to look past the video call and what looks (and sounds) like their grandson to what is actually being said. He's asking for your bank password, that behavior doesn't match! The cat is already out of the bag. Going "nooo, unpublish this" doesn't protect people from scammers, who will continue to use it. Society must unlearn that seeing is believing. That is what is morally irresponsible, not this software.
    - Planktonne 4y ago
      I don't think anyone so far has called for this to be unpublished, just for it to be more carefully considered and for the ethical aspect to be addressed when publishing. That's part of 'building a culture of distrust'. Knee-jerk refusal to even consider thinking/talking about the use of technology is far more irresponsible than any other stance.

bko 4y ago

Honest question, do you apply the same reasoning to gun control?

barnabee 4y ago
Good question. I would apply that reasoning to the information about how a gun works for sure, but IMO there is some (admittedly hard to define) amount of danger or perhaps destructive power beyond which we might or perhaps should choose as a society to restrict the ability to wield a tool and certainly at least some guns exceed that limit for me. I’m not sure that any software, on its own, does though, at least I can’t think of any immediate examples. I admit that this limit is hard to define. It’s also extremely hard to enforce for pure data, if that turns out to be what we want to do.
- bko 4y ago
  I agree its a very hard line to draw. But I think certain technology is more contagious and dangerous. For instance, a single person with a gun can inflict damage locally with dire consequences on his or her own personal safety in the process. But if someone has a virus (physical or digital), then it caries a kind of contagion that can disproportionately hurt many more people. That's why we can't equate something like deaths due to car accidents to pandemic deaths. A single car accident doesn't increase the risk of all car accidents. It's localized. But a single infection increases the risk of future infections due to contagion. Whether you think any technology falls into the dangerous contagious category or not is up for debate.
- Scea91 4y ago
  In this analogy information about how gun works would be an equivalent of Deepfake research paper. This tool is analogous to an actual gun.

hackernewds 4y ago

This sounds like it's making the gun control argument regularly made in America, and I disagree with it vehemently

bergenty 4y ago

formerkrogemp 4y ago

We open sourced 3d printed AK-47s. We disclaim any responsibility for the ensuing mass shootings. We produced the software to run the drones. We didn't personally deploy them to kill those children. We built software to guide the rockets. We didn't personally fire them at those hospitals. We chose to startup our company providing profiles of suspects using scraped, publically available data to any government agency. What they do with the information isn't our problem. We wrote the software, but it's not our fault or problem what happens because of who uses and because of how they use it. Our intent was good. The market and demand was there. What's wrong with providing a supply? How many software creators feel culpable? Vanishingly few. Who cares when comp is high?

krapp 4y ago
"Once the rockets are up, who cares where they come down? That's not my department!" says Wernher von Braun