9 ms·

I really, really hope that there aren't any people who think the way you've outlined. Technology has empowered small groups or even single individuals to create

by mckirk 4y ago

I really, really hope that there aren't any people who think the way you've outlined. Technology has empowered small groups or even single individuals to create things that have the potential to change the course of civilization, so I for sure hope those individuals think twice about the potential consequences of their actions. How would you feel about people releasing a $100 'build your own Covid-variant' kit?

brigandish 4y ago
> How would you feel about people releasing a $100 'build your own Covid-variant' kit? Not very good but: a) the people who currently have this tech are not what I'd call trustworthy so why should I leave dangerous tech only in the hands of dangerous people? b) it would probably kickstart a "build your own vaccine kit" industry
- Madmallard 4y ago
  Human life is quite fragile
- afarrell 4y ago
  What questions would you ask to decide if someone is a trustworthy steward of that technology?
  - brigandish 4y ago
    Did you fund gain of research function at the labs in Wuhan? Do you know what they found? Were you aware of the low level of biosecurity protocols on the site? Things like that ;-)
    - alexvoda 4y ago
      These would not answer whether someone behaves responsibly with dangerous research. It answers whether someone had any connection to Wuhan. You can find plenty of people who would answer all "no"-s and would be far more destructive if they had access. Would you also claim that no one who worked at Cernobyl was trustworthy? This is just guilt by association.
      - brigandish 4y ago
        > These would not answer whether someone behaves responsibly with dangerous research. Let's see. “Did you fund gain of research function at the labs in Wuhan?” This speaks to trustworthiness and responsibility/recklessness. “Do you know what they found?” This speaks to, again, trustworthiness and transparency. “Were you aware of the low level of biosecurity protocols on the site?” This speaks to, again, trustworthiness and responsibility/recklessness. > You can find plenty of people who would answer all "no"-s and would be far more destructive if they had access. It shows that those with the technology now are not trustworthy, going forward I might have different questions but we should start where we are, not by providing an apologetic and condoning poor behaviour. > Would you also claim that no one who worked at Cernobyl was trustworthy? Did they fuck up? Were they part of an organisation that fucked up? What did they do to stop the incredible fuck up from happening? > This is just guilt by association. Of whom to what?
        alexvoda 4y ago
        Again, your 3 questions do not provide any information with regards to trustworthiness. You have just decided that the Wuhan Lab is an epicenter of evil and therefore anyone tangential is untrustworthy. There is no logical connection, you filled it in. With regards to Cernobyl, other than Anatoly Dyatlov (who can directly be ascribed blame and served prison time), it is really hard to blame any of the people involved. Yet there were all kinds of known and unknown stresses and defects in the system that resulted in a tragedy. Does that mean we should ban nuclear energy?
        brigandish 4y ago
        > You have just decided that the Wuhan Lab is an epicenter of evil and therefore anyone tangential is untrustworthy. I'm so glad you were able to read my mind, perhaps you could open source your mind reading device, or would that be too dangerous to share to the public? What I have decided is that there is evidence of lies by people involved in the funding and the implementation of the research at Wuhan, there really is no need for your clumsy straw men, especially when you can read my mind. > Does that mean we should ban nuclear energy? I'm the one arguing for the further distribution of technology, are you arguing with yourself now or did your mind reading device go on the blink?
- blululu 4y ago
  This is just beyond obtuse. More people having access will mean more people who are untrustworthy having access, which means more malicious action. (Unless you want to setup some toy scenario where the only bad faith actors people on the planet are biochemistry researchers). As for building your own vaccine. Even large nations were not able to develop effective ones. It’s easier to put a bullet in someone than it is to take it out.
  - brigandish 4y ago
    Pandora's box is open, technology will spread regardless of what you do. You want to limit tech that will become cheaper and easier to build through the efforts of legitimate, well meaning research, and try to stop knowledge spreading as we wish for there to be more people in this line of work. More people having access will mean more people who are untrustworthy having access is entirely right but whether "which means more malicious action" is a corollary is up for debate, as we've seen with violent action around the world via many different types of weaponry. In this talk[1] at DEF CON, John Sotos pleads with hackers to put their time into learning how to combat biological weapons. He points out that the tech will move forward to being able to target more and more specific populations. Like any other weapon, knowing that your enemy has it too and can hurt you or those you care about with it is a highly effective deterrent. > As for building your own vaccine. Even large nations were not able to develop effective ones. It’s easier to put a bullet in someone than it is to take it out. Two things here. Firstly, trauma surgery and techniques from the military and places where bullet wounds are common have benefited the rest of us for when we get into scrapes of even a non-violent kind. Secondly, defence does not always lag behind attack, you assume that the future would look like now but that is to ignore the history of any tech of such nature. [1] https://www.youtube.com/watch?v=HKQDSgBHPfY https://www.youtube.com/watch?v=HKQDSgBHPfY
- alexvoda 4y ago
  That you even express the problem like this shows an impressive ammount of bias. By calling them dangerous people you are actually implying malice. What makes you believe people with access to biomedical tech are inherently more malicious than the populace? What makes you believe there aren't far more malicious people who do not yet have access to such tech? I think this is just fear of the unknown at work. Biomedical knowledge is complicated and requires effort to learn therefore most consider it a known unknown therefore something to be feared. Some people do have such knowledge therefore they are to be feared because who knows what nefarious intentions they have and what conspiracies they are part of. Therefore they are dangerous people using dangerous tech. Were the physicists who discovered how to split the atom also dangerous people?
  - t-3 4y ago
    Danger is not a moral judgement, but a rational one. Iranian nuclear scientists weren't assassinated for no reason, and I highly doubt anyone involved had serious moral accusation to make against them. The U.S. also banned export of cryptography in the past (and still does in some cases), solely based on strategic grounds, not because mathematicians are "bad people". People in positions of or close to power and/or possessing certain knowledge are dangerous, not because they are necessarily morally repugnant, but because of their privilege and ability.
  - pjc50 4y ago
    > Were the physicists who discovered how to split the atom also dangerous people? In the ordinary meaning of the word? Yes. That's why they were sworn to secrecy. Not the same as saying they were immoral or wrong to have worked on the Bomb, that's a different debate, but in terms of sheer effectiveness they were incredibly dangerous. An army is dangerous. That's how it works.
    - alexvoda 4y ago
      To you and sibling comment I would argue the comment I responded to conflated dangerous with immoral. Danger is part of the domain of threat modelling. And when doing threat modelling, the morality of the opponent is a distraction. However in the domain of propaganda, threats, immorality and assigning the latter to the former go hand in hand.
- Barrin92 4y ago
  >so why should I leave dangerous tech only in the hands of dangerous people? because handing it to everyone doesn't make things better? I don't like that Putin has nukes, but it's much better than Putin and every offshoot of Al-Qaeda having nukes. Civilization ending tech in the hands of powerful actors is usually subject to some form of rational calculus. Having it in the hands of everyone means basically it's game over. For a lot of dangeorus technologies there is no 'vaccine' (in time).
  - brigandish 4y ago
    > Having it in the hands of everyone means basically it's game over. It might mean it's time to face up to more important questions, like why every offshoot of Al-Qaeda wants to use nukes and then countering that.
    - alexvoda 4y ago
      I do not believe you are arguing in good faith but I will answer. Nukes Are very very effective at creating terror. Terror is somewhat effective at almost any action that requires the participation of others. Ergo, nukes are effective at threatening other to do what you want. Want to counter that, fine. Genetically engineer every person to be invulnerable to radiation and explosions and the desire to use nukes diminishes because they are no longer as effective. Sayonara.
      - brigandish 4y ago
        > I do not believe you are arguing in good faith This isn't Twitter, please keep this juvenile nonsense for there. As to your "answer": > Genetically engineer every person to be invulnerable to radiation and explosions and the desire to use nukes diminishes because they are no longer as effective. aside from it approaching word salad, I was referring to the deeper causes of violence. I would trust (there's that word again, I hope you can cope this time) the Dalai Lama with any weapon known to man and any only dreamt of, because, as he says[1]: > “Real gun control must come from here,” the Dalai Lama said, pointing at his heart. How you failed to deduce that with your enhanced powers of insight into me, I cannot fathom. [1] https://www.sfgate.com/politics/article/Dalai-Lama-says-real-gun-control-comes-from-8313907.php https://www.sfgate.com/politics/article/Dalai-Lama-says-real...
- permo-w 4y ago
  this is the gun debate rephrased
  - bumby 4y ago
    There is a fundamental difference in the US: access to guns is a constitutional right.
    - Spivak 4y ago
      But that fact doesn’t matter in the actual discussion because the debate is the same but the implementation is different. In the US the “get rid of guns” stance is just restricting their use to the maximum extent allowed by the constitution and making them as close to de facto banned as possible.
      - bumby 4y ago
        The fact that there is a constitutional right puts pretty strong limits on where that line is drawn though. I think those guardrails make it a fundamentally different perspective.
    - emptybits 4y ago
      Freedom to speak and publish, even dangerous ideas, is also a right. Beyond the US.
      - bumby 4y ago
        True, but (in the US at least), those rights also stop when they come up against an imminent threat. See the trope about falsely yelling “fire” in a crowded movie theater. The issue here seems to be accurately defining that risk.
        SAI_Peregrinus 4y ago
        The "fire in a crowded theater" case was overturned as unconstitutional prior restraint on speech.
- mannykannot 4y ago
  > it would probably kickstart a "build your own vaccine kit" industry. This is a scenario for a dystopian science fiction novel, as opposed to a a rational plan for our children's future.
  - b112 4y ago
    Tell someone from 1950 about modern cell phones, tracking in them and online, tracking in car ECUs, tracking all purchases, with companies listening in on everything said at home, and that's their dystopian novel!
    - Bakary 4y ago
      It's also a dystopia according to our current sensibilities. Or at least for a plurality of people. Even the people who don't seem to care tend to react adversely once they understand how it can affect them.
    - mannykannot 4y ago
      We have seen your scenario, and we have also had, in the last three years, a taste of brigandish's scenario, so I feel people can make up their own minds about whether they are usefully comparable. And if you do think they are comparable, note that the equivalent of a "build-your-own vaccine kit" in your scenario has not materially improved the situation (right now, "Google Has Most of My Email Because It Has All of Yours" is at position 8 on the HN front page.)
  - brigandish 4y ago
    One of the co-founders of BioNTech had this to say[1]: > “What we have developed over decades for cancer vaccine development has been the tailwind for developing the COVID-19 vaccine, and now the Covid-19 vaccine and our experience in developing it gives back to our cancer work,” Tureci said. [1] https://nypost.com/2022/10/19/covid-vax-makers-say-cancer-vaccine-could-be-available-by-2030/ https://nypost.com/2022/10/19/covid-vax-makers-say-cancer-va...
    - mannykannot 4y ago
w1nst0nsm1th 4y ago
It's already the case. Crispr has changed a lot of things and make possible for an outsider, with 10.000$ and a little dedication, to alter genome of every living form. https://www.ft.com/content/9ac7f1c0-1468-4dc7-88dd-1370ead42371 https://www.ft.com/content/9ac7f1c0-1468-4dc7-88dd-1370ead42...
- ethanbond 4y ago
  Right and every new technology that enters this “high power, high availability” domain, the more civilizational risk we all carry.
  - akimball 4y ago
    Yet the risk of non-civilization continues to exceed the risk of civilization.
  - brigandish 4y ago
    Is it really true that we're more at risk now than our ancestors were? They had smaller numbers, less access to life saving technology, worse conditions to grow up in… I can accept an argument that since the advent of the cold war there may have been more than previously, but even that is conjectural.
    - ethanbond 4y ago
      Fair question. The way I see it there was never a moment before the Cold War that a single person’s decision could even have a chance at destroying human civilization (if not the species itself). Since the Cold War, that has been true every single moment of every day, and there are probably hundreds or thousands of individuals who are capable of making a decision that will trigger nuclear annihilation (check out Daniel Ellsberg’s Doomsday Machine for an alarming inquiry into this topic). Now we have the additional risk of man made biological risk amplified by cheaper and cheaper genetic engineering as well as natural biological risk amplified by a nonstop global travel network. Neither of these risk vectors existed til recently either. IMO the only analogous risks pre-cold war were what, asteroid strike or volcanic event? Those are rare and, more importantly, not modulated upwards by any human action or human system. Nuclear and bio risk probably only climb with more people and more technological advancement.
catiopatio 4y ago
I really, really hope that there aren’t any people who think they way you’ve outlined. AI image generation is not a build-your-own-weaponized-virus kit. It’s a useful tool that can be used to produce creative expression. What people produce is up to them, and the fact that they might misuse their capacity for free speech isn’t an argument for curtailing it.
- ethanbond 4y ago
  OP doesn’t sound like it’s talking exclusively about image generation. Sounds like a general, “I should be able to build, propagate, and use whatever tech however I want no matter the negative externalities.”
  - betwixthewires 4y ago
    I think even the phrase "negative externalities" is overstating it. There's a big difference between "I push this button and now the woods behind my house are destroyed" and "I push this button and I have what looks like a photograph of some important person naked." Photo generating AIs are not a big deal IMO. We might be talking about these things more generally but I doubt we are talking about McNukes here.
    - ethanbond 4y ago
      The problem with extremely powerful forces (like new technologies) is that you can’t always predict what effects they’ll have. This is doubly true with regard to technologies that seem not only powerful, not only adaptable to new domains, but also rapidly improving on both of those dimensions. I don’t know what is the right level or type of limitation, but there is nothing confusing or weird at all about wanting to be careful with such a technology. If technology keeps advancing (it will), new developments will approach “looks kind of alarming” status faster and faster. This is because they will also approach “could destroy everything we know and love” status faster and faster.
      - betwixthewires 4y ago
kspacewalk2 4y ago
Once the cat is out of the bag, the problem exists. Worrying about how long exactly it takes for $irresponsible_person to make it slightly worse by reducing the barrier to access even further is, in my opinion, missing the point. There are many examples of this. - Non-proliferation folks who think they can actually rid the world of nukes. Will not happen. - Does anyone seriously think they can stop human cloning, once it's technically feasible, from happening somewhere on the planet sooner or later? By fiat, by legislation, by moral appeals, etc? Will not happen. If clones can be made, clones will be made. Descriptive, not normative claim. - AI-generated content has reached a certain point where we have to worry about a whole host of issues, most obvious being more sophisticated fakes. "Please think of potential consequences", ad-hoc restrictions, self-imposed or otherwise, are moot in the long run. It's part of our world now.
- chalst 4y ago
  > Non-proliferation folks who think they can actually rid the world of nukes. Will not happen. It looks to me that you're shifting the goalposts here: nonproliferation has effectively reduced the number of countries with access to nukes. Or is worrying about the number of direct military conflicts between nuclear-armed powers an example of what you call 'missing the point'?
  - kspacewalk2 4y ago
    The nuclear non-proliferation treaty has eventual complete disarmament as a stated goal. So the subset of people who believe not just in not increasing the number of countries with nukes, but eventually getting that number of countries to 0, are unrealistic. The fewer the number of nuclear powers, the greater the incentive is to cheat. Maximum incentive is when the number is 0 - get nukes and you rule the world. Until it goes to 2 again, and so on. Larger point being: with some disruptive technology, like nuclear weapons, if it can be done, it will be done.
- derangedHorse 4y ago
  >Worrying about how long exactly it takes for $irresponsible_person to make it slightly worse by reducing the barrier to access even further is, in my opinion, missing the point I disagree with the idea that putting restrictions in place shouldn't be done because 'the problem exists'. The problem exists but that doesn't mean measures can't be taken to keep it manageable. I don't think the majority of people are in your intended demographic of wanting to stop the problem. Most just want to prevent exacerbating the problem.

by mckirk 4y ago

brigandish 4y ago

> How would you feel about people releasing a $100 'build your own Covid-variant' kit? Not very good but: a) the people who currently have this tech are not what I'd call trustworthy so why should I leave dangerous tech only in the hands of dangerous people? b) it would probably kickstart a "build your own vaccine kit" industry

Madmallard 4y ago
Human life is quite fragile
afarrell 4y ago
What questions would you ask to decide if someone is a trustworthy steward of that technology?
- brigandish 4y ago
  Did you fund gain of research function at the labs in Wuhan? Do you know what they found? Were you aware of the low level of biosecurity protocols on the site? Things like that ;-)
  - alexvoda 4y ago
    These would not answer whether someone behaves responsibly with dangerous research. It answers whether someone had any connection to Wuhan. You can find plenty of people who would answer all "no"-s and would be far more destructive if they had access. Would you also claim that no one who worked at Cernobyl was trustworthy? This is just guilt by association.
    - brigandish 4y ago
      > These would not answer whether someone behaves responsibly with dangerous research. Let's see. “Did you fund gain of research function at the labs in Wuhan?” This speaks to trustworthiness and responsibility/recklessness. “Do you know what they found?” This speaks to, again, trustworthiness and transparency. “Were you aware of the low level of biosecurity protocols on the site?” This speaks to, again, trustworthiness and responsibility/recklessness. > You can find plenty of people who would answer all "no"-s and would be far more destructive if they had access. It shows that those with the technology now are not trustworthy, going forward I might have different questions but we should start where we are, not by providing an apologetic and condoning poor behaviour. > Would you also claim that no one who worked at Cernobyl was trustworthy? Did they fuck up? Were they part of an organisation that fucked up? What did they do to stop the incredible fuck up from happening? > This is just guilt by association. Of whom to what?
      - alexvoda 4y ago
        Again, your 3 questions do not provide any information with regards to trustworthiness. You have just decided that the Wuhan Lab is an epicenter of evil and therefore anyone tangential is untrustworthy. There is no logical connection, you filled it in. With regards to Cernobyl, other than Anatoly Dyatlov (who can directly be ascribed blame and served prison time), it is really hard to blame any of the people involved. Yet there were all kinds of known and unknown stresses and defects in the system that resulted in a tragedy. Does that mean we should ban nuclear energy?
        brigandish 4y ago
        > You have just decided that the Wuhan Lab is an epicenter of evil and therefore anyone tangential is untrustworthy. I'm so glad you were able to read my mind, perhaps you could open source your mind reading device, or would that be too dangerous to share to the public? What I have decided is that there is evidence of lies by people involved in the funding and the implementation of the research at Wuhan, there really is no need for your clumsy straw men, especially when you can read my mind. > Does that mean we should ban nuclear energy? I'm the one arguing for the further distribution of technology, are you arguing with yourself now or did your mind reading device go on the blink?
blululu 4y ago
This is just beyond obtuse. More people having access will mean more people who are untrustworthy having access, which means more malicious action. (Unless you want to setup some toy scenario where the only bad faith actors people on the planet are biochemistry researchers). As for building your own vaccine. Even large nations were not able to develop effective ones. It’s easier to put a bullet in someone than it is to take it out.
- brigandish 4y ago
  Pandora's box is open, technology will spread regardless of what you do. You want to limit tech that will become cheaper and easier to build through the efforts of legitimate, well meaning research, and try to stop knowledge spreading as we wish for there to be more people in this line of work. More people having access will mean more people who are untrustworthy having access is entirely right but whether "which means more malicious action" is a corollary is up for debate, as we've seen with violent action around the world via many different types of weaponry. In this talk[1] at DEF CON, John Sotos pleads with hackers to put their time into learning how to combat biological weapons. He points out that the tech will move forward to being able to target more and more specific populations. Like any other weapon, knowing that your enemy has it too and can hurt you or those you care about with it is a highly effective deterrent. > As for building your own vaccine. Even large nations were not able to develop effective ones. It’s easier to put a bullet in someone than it is to take it out. Two things here. Firstly, trauma surgery and techniques from the military and places where bullet wounds are common have benefited the rest of us for when we get into scrapes of even a non-violent kind. Secondly, defence does not always lag behind attack, you assume that the future would look like now but that is to ignore the history of any tech of such nature. [1] https://www.youtube.com/watch?v=HKQDSgBHPfY https://www.youtube.com/watch?v=HKQDSgBHPfY
alexvoda 4y ago
That you even express the problem like this shows an impressive ammount of bias. By calling them dangerous people you are actually implying malice. What makes you believe people with access to biomedical tech are inherently more malicious than the populace? What makes you believe there aren't far more malicious people who do not yet have access to such tech? I think this is just fear of the unknown at work. Biomedical knowledge is complicated and requires effort to learn therefore most consider it a known unknown therefore something to be feared. Some people do have such knowledge therefore they are to be feared because who knows what nefarious intentions they have and what conspiracies they are part of. Therefore they are dangerous people using dangerous tech. Were the physicists who discovered how to split the atom also dangerous people?
- t-3 4y ago
  Danger is not a moral judgement, but a rational one. Iranian nuclear scientists weren't assassinated for no reason, and I highly doubt anyone involved had serious moral accusation to make against them. The U.S. also banned export of cryptography in the past (and still does in some cases), solely based on strategic grounds, not because mathematicians are "bad people". People in positions of or close to power and/or possessing certain knowledge are dangerous, not because they are necessarily morally repugnant, but because of their privilege and ability.
- pjc50 4y ago
  > Were the physicists who discovered how to split the atom also dangerous people? In the ordinary meaning of the word? Yes. That's why they were sworn to secrecy. Not the same as saying they were immoral or wrong to have worked on the Bomb, that's a different debate, but in terms of sheer effectiveness they were incredibly dangerous. An army is dangerous. That's how it works.
  - alexvoda 4y ago
    To you and sibling comment I would argue the comment I responded to conflated dangerous with immoral. Danger is part of the domain of threat modelling. And when doing threat modelling, the morality of the opponent is a distraction. However in the domain of propaganda, threats, immorality and assigning the latter to the former go hand in hand.
Barrin92 4y ago
>so why should I leave dangerous tech only in the hands of dangerous people? because handing it to everyone doesn't make things better? I don't like that Putin has nukes, but it's much better than Putin and every offshoot of Al-Qaeda having nukes. Civilization ending tech in the hands of powerful actors is usually subject to some form of rational calculus. Having it in the hands of everyone means basically it's game over. For a lot of dangeorus technologies there is no 'vaccine' (in time).
- brigandish 4y ago
  > Having it in the hands of everyone means basically it's game over. It might mean it's time to face up to more important questions, like why every offshoot of Al-Qaeda wants to use nukes and then countering that.
  - alexvoda 4y ago
    I do not believe you are arguing in good faith but I will answer. Nukes Are very very effective at creating terror. Terror is somewhat effective at almost any action that requires the participation of others. Ergo, nukes are effective at threatening other to do what you want. Want to counter that, fine. Genetically engineer every person to be invulnerable to radiation and explosions and the desire to use nukes diminishes because they are no longer as effective. Sayonara.
    - brigandish 4y ago
      > I do not believe you are arguing in good faith This isn't Twitter, please keep this juvenile nonsense for there. As to your "answer": > Genetically engineer every person to be invulnerable to radiation and explosions and the desire to use nukes diminishes because they are no longer as effective. aside from it approaching word salad, I was referring to the deeper causes of violence. I would trust (there's that word again, I hope you can cope this time) the Dalai Lama with any weapon known to man and any only dreamt of, because, as he says[1]: > “Real gun control must come from here,” the Dalai Lama said, pointing at his heart. How you failed to deduce that with your enhanced powers of insight into me, I cannot fathom. [1] https://www.sfgate.com/politics/article/Dalai-Lama-says-real-gun-control-comes-from-8313907.php https://www.sfgate.com/politics/article/Dalai-Lama-says-real...
permo-w 4y ago
this is the gun debate rephrased
- bumby 4y ago
  There is a fundamental difference in the US: access to guns is a constitutional right.
  - Spivak 4y ago
    But that fact doesn’t matter in the actual discussion because the debate is the same but the implementation is different. In the US the “get rid of guns” stance is just restricting their use to the maximum extent allowed by the constitution and making them as close to de facto banned as possible.
    - bumby 4y ago
      The fact that there is a constitutional right puts pretty strong limits on where that line is drawn though. I think those guardrails make it a fundamentally different perspective.
  - emptybits 4y ago
    Freedom to speak and publish, even dangerous ideas, is also a right. Beyond the US.
    - bumby 4y ago
      True, but (in the US at least), those rights also stop when they come up against an imminent threat. See the trope about falsely yelling “fire” in a crowded movie theater. The issue here seems to be accurately defining that risk.
      - SAI_Peregrinus 4y ago
        The "fire in a crowded theater" case was overturned as unconstitutional prior restraint on speech.
mannykannot 4y ago
> it would probably kickstart a "build your own vaccine kit" industry. This is a scenario for a dystopian science fiction novel, as opposed to a a rational plan for our children's future.
- b112 4y ago
  Tell someone from 1950 about modern cell phones, tracking in them and online, tracking in car ECUs, tracking all purchases, with companies listening in on everything said at home, and that's their dystopian novel!
  - Bakary 4y ago
    It's also a dystopia according to our current sensibilities. Or at least for a plurality of people. Even the people who don't seem to care tend to react adversely once they understand how it can affect them.
  - mannykannot 4y ago
    We have seen your scenario, and we have also had, in the last three years, a taste of brigandish's scenario, so I feel people can make up their own minds about whether they are usefully comparable. And if you do think they are comparable, note that the equivalent of a "build-your-own vaccine kit" in your scenario has not materially improved the situation (right now, "Google Has Most of My Email Because It Has All of Yours" is at position 8 on the HN front page.)
- brigandish 4y ago
  One of the co-founders of BioNTech had this to say[1]: > “What we have developed over decades for cancer vaccine development has been the tailwind for developing the COVID-19 vaccine, and now the Covid-19 vaccine and our experience in developing it gives back to our cancer work,” Tureci said. [1] https://nypost.com/2022/10/19/covid-vax-makers-say-cancer-vaccine-could-be-available-by-2030/ https://nypost.com/2022/10/19/covid-vax-makers-say-cancer-va...
  - mannykannot 4y ago

w1nst0nsm1th 4y ago

It's already the case. Crispr has changed a lot of things and make possible for an outsider, with 10.000$ and a little dedication, to alter genome of every living form. https://www.ft.com/content/9ac7f1c0-1468-4dc7-88dd-1370ead42371 https://www.ft.com/content/9ac7f1c0-1468-4dc7-88dd-1370ead42...

ethanbond 4y ago
Right and every new technology that enters this “high power, high availability” domain, the more civilizational risk we all carry.
- akimball 4y ago
  Yet the risk of non-civilization continues to exceed the risk of civilization.
- brigandish 4y ago
  Is it really true that we're more at risk now than our ancestors were? They had smaller numbers, less access to life saving technology, worse conditions to grow up in… I can accept an argument that since the advent of the cold war there may have been more than previously, but even that is conjectural.
  - ethanbond 4y ago
    Fair question. The way I see it there was never a moment before the Cold War that a single person’s decision could even have a chance at destroying human civilization (if not the species itself). Since the Cold War, that has been true every single moment of every day, and there are probably hundreds or thousands of individuals who are capable of making a decision that will trigger nuclear annihilation (check out Daniel Ellsberg’s Doomsday Machine for an alarming inquiry into this topic). Now we have the additional risk of man made biological risk amplified by cheaper and cheaper genetic engineering as well as natural biological risk amplified by a nonstop global travel network. Neither of these risk vectors existed til recently either. IMO the only analogous risks pre-cold war were what, asteroid strike or volcanic event? Those are rare and, more importantly, not modulated upwards by any human action or human system. Nuclear and bio risk probably only climb with more people and more technological advancement.

catiopatio 4y ago

I really, really hope that there aren’t any people who think they way you’ve outlined. AI image generation is not a build-your-own-weaponized-virus kit. It’s a useful tool that can be used to produce creative expression. What people produce is up to them, and the fact that they might misuse their capacity for free speech isn’t an argument for curtailing it.

ethanbond 4y ago
OP doesn’t sound like it’s talking exclusively about image generation. Sounds like a general, “I should be able to build, propagate, and use whatever tech however I want no matter the negative externalities.”
- betwixthewires 4y ago
  I think even the phrase "negative externalities" is overstating it. There's a big difference between "I push this button and now the woods behind my house are destroyed" and "I push this button and I have what looks like a photograph of some important person naked." Photo generating AIs are not a big deal IMO. We might be talking about these things more generally but I doubt we are talking about McNukes here.
  - ethanbond 4y ago
    The problem with extremely powerful forces (like new technologies) is that you can’t always predict what effects they’ll have. This is doubly true with regard to technologies that seem not only powerful, not only adaptable to new domains, but also rapidly improving on both of those dimensions. I don’t know what is the right level or type of limitation, but there is nothing confusing or weird at all about wanting to be careful with such a technology. If technology keeps advancing (it will), new developments will approach “looks kind of alarming” status faster and faster. This is because they will also approach “could destroy everything we know and love” status faster and faster.
    - betwixthewires 4y ago

kspacewalk2 4y ago

Once the cat is out of the bag, the problem exists. Worrying about how long exactly it takes for $irresponsible_person to make it slightly worse by reducing the barrier to access even further is, in my opinion, missing the point. There are many examples of this. - Non-proliferation folks who think they can actually rid the world of nukes. Will not happen. - Does anyone seriously think they can stop human cloning, once it's technically feasible, from happening somewhere on the planet sooner or later? By fiat, by legislation, by moral appeals, etc? Will not happen. If clones can be made, clones will be made. Descriptive, not normative claim. - AI-generated content has reached a certain point where we have to worry about a whole host of issues, most obvious being more sophisticated fakes. "Please think of potential consequences", ad-hoc restrictions, self-imposed or otherwise, are moot in the long run. It's part of our world now.

chalst 4y ago
> Non-proliferation folks who think they can actually rid the world of nukes. Will not happen. It looks to me that you're shifting the goalposts here: nonproliferation has effectively reduced the number of countries with access to nukes. Or is worrying about the number of direct military conflicts between nuclear-armed powers an example of what you call 'missing the point'?
- kspacewalk2 4y ago
  The nuclear non-proliferation treaty has eventual complete disarmament as a stated goal. So the subset of people who believe not just in not increasing the number of countries with nukes, but eventually getting that number of countries to 0, are unrealistic. The fewer the number of nuclear powers, the greater the incentive is to cheat. Maximum incentive is when the number is 0 - get nukes and you rule the world. Until it goes to 2 again, and so on. Larger point being: with some disruptive technology, like nuclear weapons, if it can be done, it will be done.
derangedHorse 4y ago
>Worrying about how long exactly it takes for $irresponsible_person to make it slightly worse by reducing the barrier to access even further is, in my opinion, missing the point I disagree with the idea that putting restrictions in place shouldn't be done because 'the problem exists'. The problem exists but that doesn't mean measures can't be taken to keep it manageable. I don't think the majority of people are in your intended demographic of wanting to stop the problem. Most just want to prevent exacerbating the problem.