5 ms·

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we p

Does this apply to nuclear weapons? To biological weapons? “Tool is expensive” is a real, effective deterrent to frivolous use of a tool. There’s a reason we pool resources into governments, which is explicitly so we (via a government) have capabilities that we (as individuals) don’t have.

rlayton2 4y ago
No, but it does apply to the distribution of slippery slopes. The OP specifically said tools "like this"
- ceejayoz 4y ago
  Some slopes are slipperier than others.
- ethanbond 4y ago
  I guess I don’t understand what “like this” means. While it’s convenient to dismiss any discussion about A and something slightly similar to A as “slippery slope,” an inconvenient fact is that the slope often is slippery. It’s increasingly evident to anyone paying attention that information warfare can yield disastrous effects in a variety of ways: disrupting peaceful transition of power; disrupting critical infrastructure; disrupting command and control of ”traditional” WMD systems, even. So that’s sort of the spirit of my question: how far does this argument go? What is the definition of “like this?”
rolisz 4y ago
For nuclear weapons, even if plans were publicly available, getting the right kind of uranium would still be a major hurdle for 99.9% of the people.
- FollowingTheDao 4y ago
  But what are the consequences of the .1% of people who get over that hurdle?
  - pixl97 4y ago
    Having something that can still be detected by most of the planet and making them very angry.
    - picsao 4y ago
- mastersummoner 4y ago
  But what if they weren't? What if one could create something with similar destructive force with materials easily accessible, and the only hurdle is knowledge of how to do so? It's honestly an interesting moral question.
  - barnabee 4y ago
    Over time the number of single humans that have the power to wipe out a sizeable portion of life on Earth (which is almost certainly non-zero already) will almost certainly increase due to advances in technology and dissemination of information. Do we aim for security through obscurity or work out how to deal with that increasingly terrifying reality? I think at some point we will be forced to do latter.
    - ethanbond 4y ago
      This trend is clearly real, but what exactly do you mean “work out how to deal with that?” One of the obvious ways to deal with it, albeit imperfectly, is in fact to keep these technologies as obscure as possible for as long as possible. Maybe an interesting norm to try to generate in this community would be, “if you plan on publishing attack X, you should publish immediately alongside it your best guess as to the effective counter to X.”
      - leereeves 4y ago
        Keeping these technologies in the hands of a few has its own set of severe negative consequences. In the case of nuclear weapons, one of those consequences is that poor countries are denied the ability to defend themselves. This led to the bullying and invasion of countries without nuclear weapons by countries with nuclear weapons (most recently Russia invading Ukraine, but China and the US are equally guilty). I'm reminded of a line from the TV show The West Wing, something like: "India must and will be a nuclear power. That way, we'll never be dictated to again."
      - ryandrake 4y ago
        Even an extreme example -- The technology to build an automated drone with a camera for targeting and a gun on a gimbal and deploy it to do a mass-shooting -- all exists today. All it would take is for some madman to pick the pieces up and assemble them. In fact, I'd be willing to bet $100 that we'll actually see some kind of tragedy to that effect in the USA within the next 5 years. The technology cats are very quickly getting out of the bags.
- munk-a 4y ago
  So then what about releasing data on how to enrich uranium at home or selling it on <web store>? I know usually there are multi-million dollar facilities required for enriched uranium production but why shouldn't the free market allow people to buy enriched uranium - it isn't our place to morally prejudge what people may use it for and the bad people will have access to it anyways... so why can't I buy a kilo of U-235 on Amazon? I'm sure there are some very worthy technical efforts to improve mankind that are being hampered by the lack of easily accessible U-235 so why are we restricting this material from Chemists that want to Chem and Physicists that want to Fizz?
phkahler 4y ago
>> Does this apply to nuclear weapons? To biological weapons? That may not be a fair comparison. Nuclear weapons are difficult to make, and using them will not go undetected - the damage would be enormous. Deep fakes are a real threat today, and anyone willing to pay for the expertise (which isn't that hard to find) can have them made. By putting this out there, deepfakes are now something almost anyone can make which means the threat is imminent and needs to be addressed. I understand the argument, but still not sure about it. Just wanted to say it's different than nuclear weapons.
- rockemsockem 4y ago
  I'd argue that one of the biggest reasons deepfakes pose a threat is because a large number of people don't know they exist. It's just those e-mail chains from the early 00s with poorly photoshopped images that lots of people thought were real. Now that the public knows broadly that digital manipulation is easy and they have seen it with their own eyes a ton things like that are believed less. Notice I do say less though, we still definitely have problems with misinformation being propagated with shitty-looking video/audio/photos that people should not believe in because they look/sound so inauthentic. All that to say, these tools clearly exist. I think the public is better served by them being pervasive and well-known rather than having them be locked up in Hollywood studios, research labs, and government agencies.
  - ethanbond 4y ago
    Right but this means real content is less believable, too.
    - rockemsockem 4y ago
      Indeed. Honestly that sounds like a bit of a plus too. A bit more skepticism should be applied to videos of talking heads in suits saying things. Especially when there's a huge panel of them.
    - nonbirithm 4y ago
      I think a lot of information could be treated like pollution, in that it is easily created, spreads quickly, and becomes harder to clean up the more people are able to learn of it. I have to wonder if forging on like this will have a net positive effect on society, or if these kinds of developments are just fated by human nature. In a perfect world, what should have happened was a separate team inventing a reasonably accurate deepfake detector to the world before the deepfake producer gained prominence (differential technological development), but interest, productivity and excitement on the researchers' part to release their work first seems to have foiled that.

rlayton2 4y ago

No, but it does apply to the distribution of slippery slopes. The OP specifically said tools "like this"

ceejayoz 4y ago
Some slopes are slipperier than others.
ethanbond 4y ago
I guess I don’t understand what “like this” means. While it’s convenient to dismiss any discussion about A and something slightly similar to A as “slippery slope,” an inconvenient fact is that the slope often is slippery. It’s increasingly evident to anyone paying attention that information warfare can yield disastrous effects in a variety of ways: disrupting peaceful transition of power; disrupting critical infrastructure; disrupting command and control of ”traditional” WMD systems, even. So that’s sort of the spirit of my question: how far does this argument go? What is the definition of “like this?”

rolisz 4y ago

For nuclear weapons, even if plans were publicly available, getting the right kind of uranium would still be a major hurdle for 99.9% of the people.

FollowingTheDao 4y ago
But what are the consequences of the .1% of people who get over that hurdle?
- pixl97 4y ago
  Having something that can still be detected by most of the planet and making them very angry.
  - picsao 4y ago
mastersummoner 4y ago
But what if they weren't? What if one could create something with similar destructive force with materials easily accessible, and the only hurdle is knowledge of how to do so? It's honestly an interesting moral question.
- barnabee 4y ago
  Over time the number of single humans that have the power to wipe out a sizeable portion of life on Earth (which is almost certainly non-zero already) will almost certainly increase due to advances in technology and dissemination of information. Do we aim for security through obscurity or work out how to deal with that increasingly terrifying reality? I think at some point we will be forced to do latter.
  - ethanbond 4y ago
    This trend is clearly real, but what exactly do you mean “work out how to deal with that?” One of the obvious ways to deal with it, albeit imperfectly, is in fact to keep these technologies as obscure as possible for as long as possible. Maybe an interesting norm to try to generate in this community would be, “if you plan on publishing attack X, you should publish immediately alongside it your best guess as to the effective counter to X.”
    - leereeves 4y ago
      Keeping these technologies in the hands of a few has its own set of severe negative consequences. In the case of nuclear weapons, one of those consequences is that poor countries are denied the ability to defend themselves. This led to the bullying and invasion of countries without nuclear weapons by countries with nuclear weapons (most recently Russia invading Ukraine, but China and the US are equally guilty). I'm reminded of a line from the TV show The West Wing, something like: "India must and will be a nuclear power. That way, we'll never be dictated to again."
    - ryandrake 4y ago
      Even an extreme example -- The technology to build an automated drone with a camera for targeting and a gun on a gimbal and deploy it to do a mass-shooting -- all exists today. All it would take is for some madman to pick the pieces up and assemble them. In fact, I'd be willing to bet $100 that we'll actually see some kind of tragedy to that effect in the USA within the next 5 years. The technology cats are very quickly getting out of the bags.
munk-a 4y ago
So then what about releasing data on how to enrich uranium at home or selling it on <web store>? I know usually there are multi-million dollar facilities required for enriched uranium production but why shouldn't the free market allow people to buy enriched uranium - it isn't our place to morally prejudge what people may use it for and the bad people will have access to it anyways... so why can't I buy a kilo of U-235 on Amazon? I'm sure there are some very worthy technical efforts to improve mankind that are being hampered by the lack of easily accessible U-235 so why are we restricting this material from Chemists that want to Chem and Physicists that want to Fizz?

phkahler 4y ago

>> Does this apply to nuclear weapons? To biological weapons? That may not be a fair comparison. Nuclear weapons are difficult to make, and using them will not go undetected - the damage would be enormous. Deep fakes are a real threat today, and anyone willing to pay for the expertise (which isn't that hard to find) can have them made. By putting this out there, deepfakes are now something almost anyone can make which means the threat is imminent and needs to be addressed. I understand the argument, but still not sure about it. Just wanted to say it's different than nuclear weapons.

rockemsockem 4y ago
I'd argue that one of the biggest reasons deepfakes pose a threat is because a large number of people don't know they exist. It's just those e-mail chains from the early 00s with poorly photoshopped images that lots of people thought were real. Now that the public knows broadly that digital manipulation is easy and they have seen it with their own eyes a ton things like that are believed less. Notice I do say less though, we still definitely have problems with misinformation being propagated with shitty-looking video/audio/photos that people should not believe in because they look/sound so inauthentic. All that to say, these tools clearly exist. I think the public is better served by them being pervasive and well-known rather than having them be locked up in Hollywood studios, research labs, and government agencies.
- ethanbond 4y ago
  Right but this means real content is less believable, too.
  - rockemsockem 4y ago
    Indeed. Honestly that sounds like a bit of a plus too. A bit more skepticism should be applied to videos of talking heads in suits saying things. Especially when there's a huge panel of them.
  - nonbirithm 4y ago
    I think a lot of information could be treated like pollution, in that it is easily created, spreads quickly, and becomes harder to clean up the more people are able to learn of it. I have to wonder if forging on like this will have a net positive effect on society, or if these kinds of developments are just fated by human nature. In a perfect world, what should have happened was a separate team inventing a reasonably accurate deepfake detector to the world before the deepfake producer gained prominence (differential technological development), but interest, productivity and excitement on the researchers' part to release their work first seems to have foiled that.