7 ms·

I think this is a radical simplification of what is a complex problem. Genetic engineering as a whole is known to be an area strewn with moral hazards, and not

I think this is a radical simplification of what is a complex problem. Genetic engineering as a whole is known to be an area strewn with moral hazards, and not least harks back to some of the darkest days of 20th century science. Having an ethical outlook isn't a form of heresy, it's a form of societal safety. We need to accept that some of these ideas (probably not all - it's always difficult to get on the right side of the line) are inherently dangerous. Particularly when we start involving healthcare, a "move fast/break things" type approach can be extremely detrimental, for example.

mkagenius 9y ago
> We need to accept that some of these ideas are inherently dangerous. We created atom bombs. Some say AI can destroy humanity (perhaps Sam himself) but here we are, I don’t think you can stop science.. it’s what makes us human.
- ak_yo 9y ago
  You can definitely stop science. There's a reason we don't secretly infect populations with various diseases just to see what happens anymore.
  - LoSboccacc 9y ago
    > anymore and I'd also add "to the best of our armchair knowledge"
    - yesenadam 9y ago
      I thought yours a very interesting, worthwhile and thought-provoking comment. I'm puzzled why it's voted down. Maybe too succinct. 3^4 words good, 3^2 words baad?
      - LoSboccacc 9y ago
        it's because rule#324 subsection 20 of this site "usa did nothing wrong"
    - TeMPOraL 9y ago
      And to yours, I'd add "for values of 'anymore' smaller than last 30 years", and kindly remind everyone that US military used to spray pathogens on its own, unsuspecting populace, in order to experimentally explore the areas of bioweapons defense and offense.
      - LoSboccacc 9y ago
        yeah honestly puzzled why that's so downed since there are still living person whom did that shit, it is not even one generation past and this place is like "we're the supreme moral force of the world! hurr durr"!
- na85 9y ago
  Certain religions have effectively stopped science at the local/regional scale. Religion is the anathema to science.
stestagg 9y ago
I think the difference is if you consider the /idea/ inherently dangerous, or the potential /outcome/ dangerous. Yes, it's subtle, but the current shift in the US is towards ideas being dangrous
jstanley 9y ago
Having an ethical outlook is a great idea. And it's good to criticise ideas you think are unethical. The point of the post is that just because a person presents an unethical idea doesn't make that person universally unethical. The point is to separate the ideas from the person presenting them, so that people can feel free to present novel ideas without fearing damage to their reputation if the ideas are unpopular.
- SamBam 9y ago
  > The entire point of the post is to separate the ideas from the person presenting them. But that wasn't the entire point of the post. The post also didn't like people criticizing life-extending research, when the examples were people criticizing the direction of the research itself. The post was hazy about what, exactly, it wanted people to stop doing, but it seemed to dislike any criticism. Whether the "life extension is bad for the environment" argument was a good argument or not, criticism of the ethics of work must be part of free discussion.
  - sskates 9y ago
    The article made the distinction between criticism of an idea and criticism of the person behind the idea. "Of course we can and should say that ideas are mistaken, but we can’t just call the person a heretic. We need to debate the actual idea." It says right there that criticism of an idea is welcome.
    - TeMPOraL 9y ago
      Moreover, "heresy" implies that a person becomes heretic - socially despicable - by embracing an idea. This is what the author opposes. And for good reason - embracing the concept of heresy means you're only allowed to think things sanctioned by the current sociopolitical philosophy. And since every society has some amount of batshit insane ideas it believes, allowing for heresy means cutting out the safest / most peaceful way of correcting that insanity.
  - jstanley 9y ago
interurban 9y ago
Complex indeed, it's a multidimensional balancing act. Both technology and ethics can progress and regress across a huge range of concepts. Too much unexamined progress (of either tech or ethics) in one area can lead to terrible outcomes. Being able to debate and have these conversations is essential to finding the balance. I'm reading this as complaint about opinions condemning discussion of certain concepts, which is fair. But to ignore that other concepts face the same type of condemnation in China is... amazing.
LoSboccacc 9y ago
> We need to accept that some of these ideas (probably not all - it's always difficult to get on the right side of the line) are inherently dangerous. ostracizing hypothetical talks is the problem here, not pursuing the unethical goal; just talking about what part of the goal makes it unethical is being shut down (according to the article)
interfixus 9y ago
Are you saying ideas you consider inherently dangerous are good candidates for censoring?
hellcow 9y ago
> We need to accept that some of these ideas ... are inherently dangerous. This line of reasoning leads to censorship. It's the same line of reasoning used by every dictator and demagogue as well as China today. Why? The idea of democracy seems dangerous to a king. Ideas are not dangerous. Ideas are never dangerous. Ideas acted upon might cause danger or harm, but an idea itself never does, never has, and never will cause harm directly. Open debate of ideas, or the testing of opposing ideas to find the best among them, is necessary for progress... not only to find and establish new ideas, but to question widely held ideas today in the hopes of finding something better. To have open debate--to find the best ideas--you need to listen to ideas with which you might not agree, or which you might find--if acted upon--dangerous.
- nine_k 9y ago
  Yes, some ideas are inherently dangerous. An idea of atoms splitting leads to an atomic bomb, for instance. Some substances, tools, machines, etc are also inherently dangerous, and can't be made not to be. Does this mean that we should never use a saw, light a fire, or discuss [insert a controversial topic here]? Somehow people came up with protocols to work with inherently dangerous material things that lower the risk of a serious injury to acceptable bounds. I don't see why this would not apply to inherently dangerous ideas. I'd hazard to say that for the last 2500 years quite a few devices were invented in this space, allowing to think much of the previously unthinkable without turning into ravaging monsters or becoming insane. I suspect that we could just continue using these devices (e.g. "freedom of speech", "critical thinking", separation of author's personality form their ideas, etc), instead of running in terror from "dangerous ideas" while swinging a banhammer.
  - scrollaway 9y ago
    > Yes, some ideas are inherently dangerous. An idea of atoms splitting leads to an atomic bomb, for instance. You're missing the parent's point. An idea not being acted upon is not dangerous. The idea of splitting atoms is not dangerous and the atomic bomb is not dangerous: An atomic bomb getting built (the idea being acted upon) is dangerous. Why are full disclosure policies recommended by security researchers? Why is open source healthy for security in general? Why is "security through obscurity" frowned upon? Because it is important to be able to discuss threats in order to better protect yourself from them. Burying your head in the sand is just a shitty tactic for dealing with danger. Being able to discuss controversial ideas is extremely important, whether it's to better protect yourself from them (eg. extreme-right ideology) or to independently research facts hidden from you by people who don't have your best interests in mind (eg. north korean censorship). And let's be honest here, from what I've seen of my (left leaning) twitter feed, Sam is being attacked extremely harshly for his "controversial" post. If people here think that the article is wrong, this is a terrible way to prove it so.
staunch 9y ago
The idea that "ideas can be dangerous" is a dangerous idea.
jedharris 9y ago
The criticism quoted wasn't about ethical problems in genetic engineering research. It was about life extension as a goal. I can imagine a debate about that, but I can't see how it is appropriate to create a toxic environment for life extension advocates. Apparently these researchers / advocates found the environment so toxic they were motivated to move. Perhaps they were over-reacting but it seems likely something is wrong. And "move fast / break things" applied to human biology research is entirely an insertion of the commenter, not remotely an implication of Altman's piece.
ixtli 9y ago
It's critical to understand that it's not an intentional simplification. Altman and the rest of the rarified VC-connected people on both coasts are very, very disconnected from what its like to live in the lower or middle classes and especially what it's like to be a part of a systemically marginalized community.
rhizome 9y ago
What do you mean by "ethical outlook?" Who doesn't have one? You may not agree with their ethics, but they're there. If you're talking about shared ethics, that's known as morality.
Hattyflanagan 9y ago
That's exactly what the blog post wasn't realizing. Throwing out ethics is like throwing out law and order because it inhibits you from doing whatever you'd like to do. Does anyone really think they can stand upright against the winds when the weather turns nasty? Just like laws, ethics are there for a reason, and the social changes going on are happening for a reason.

mkagenius 9y ago

> We need to accept that some of these ideas are inherently dangerous. We created atom bombs. Some say AI can destroy humanity (perhaps Sam himself) but here we are, I don’t think you can stop science.. it’s what makes us human.

ak_yo 9y ago
You can definitely stop science. There's a reason we don't secretly infect populations with various diseases just to see what happens anymore.
- LoSboccacc 9y ago
  > anymore and I'd also add "to the best of our armchair knowledge"
  - yesenadam 9y ago
    I thought yours a very interesting, worthwhile and thought-provoking comment. I'm puzzled why it's voted down. Maybe too succinct. 3^4 words good, 3^2 words baad?
    - LoSboccacc 9y ago
      it's because rule#324 subsection 20 of this site "usa did nothing wrong"
  - TeMPOraL 9y ago
    And to yours, I'd add "for values of 'anymore' smaller than last 30 years", and kindly remind everyone that US military used to spray pathogens on its own, unsuspecting populace, in order to experimentally explore the areas of bioweapons defense and offense.
    - LoSboccacc 9y ago
      yeah honestly puzzled why that's so downed since there are still living person whom did that shit, it is not even one generation past and this place is like "we're the supreme moral force of the world! hurr durr"!
na85 9y ago
Certain religions have effectively stopped science at the local/regional scale. Religion is the anathema to science.

stestagg 9y ago

I think the difference is if you consider the /idea/ inherently dangerous, or the potential /outcome/ dangerous. Yes, it's subtle, but the current shift in the US is towards ideas being dangrous

jstanley 9y ago

Having an ethical outlook is a great idea. And it's good to criticise ideas you think are unethical. The point of the post is that just because a person presents an unethical idea doesn't make that person universally unethical. The point is to separate the ideas from the person presenting them, so that people can feel free to present novel ideas without fearing damage to their reputation if the ideas are unpopular.

SamBam 9y ago
> The entire point of the post is to separate the ideas from the person presenting them. But that wasn't the entire point of the post. The post also didn't like people criticizing life-extending research, when the examples were people criticizing the direction of the research itself. The post was hazy about what, exactly, it wanted people to stop doing, but it seemed to dislike any criticism. Whether the "life extension is bad for the environment" argument was a good argument or not, criticism of the ethics of work must be part of free discussion.
- sskates 9y ago
  The article made the distinction between criticism of an idea and criticism of the person behind the idea. "Of course we can and should say that ideas are mistaken, but we can’t just call the person a heretic. We need to debate the actual idea." It says right there that criticism of an idea is welcome.
  - TeMPOraL 9y ago
    Moreover, "heresy" implies that a person becomes heretic - socially despicable - by embracing an idea. This is what the author opposes. And for good reason - embracing the concept of heresy means you're only allowed to think things sanctioned by the current sociopolitical philosophy. And since every society has some amount of batshit insane ideas it believes, allowing for heresy means cutting out the safest / most peaceful way of correcting that insanity.
- jstanley 9y ago

interurban 9y ago

Complex indeed, it's a multidimensional balancing act. Both technology and ethics can progress and regress across a huge range of concepts. Too much unexamined progress (of either tech or ethics) in one area can lead to terrible outcomes. Being able to debate and have these conversations is essential to finding the balance. I'm reading this as complaint about opinions condemning discussion of certain concepts, which is fair. But to ignore that other concepts face the same type of condemnation in China is... amazing.

LoSboccacc 9y ago

> We need to accept that some of these ideas (probably not all - it's always difficult to get on the right side of the line) are inherently dangerous. ostracizing hypothetical talks is the problem here, not pursuing the unethical goal; just talking about what part of the goal makes it unethical is being shut down (according to the article)

interfixus 9y ago

Are you saying ideas you consider inherently dangerous are good candidates for censoring?

hellcow 9y ago

> We need to accept that some of these ideas ... are inherently dangerous. This line of reasoning leads to censorship. It's the same line of reasoning used by every dictator and demagogue as well as China today. Why? The idea of democracy seems dangerous to a king. Ideas are not dangerous. Ideas are never dangerous. Ideas acted upon might cause danger or harm, but an idea itself never does, never has, and never will cause harm directly. Open debate of ideas, or the testing of opposing ideas to find the best among them, is necessary for progress... not only to find and establish new ideas, but to question widely held ideas today in the hopes of finding something better. To have open debate--to find the best ideas--you need to listen to ideas with which you might not agree, or which you might find--if acted upon--dangerous.

nine_k 9y ago
Yes, some ideas are inherently dangerous. An idea of atoms splitting leads to an atomic bomb, for instance. Some substances, tools, machines, etc are also inherently dangerous, and can't be made not to be. Does this mean that we should never use a saw, light a fire, or discuss [insert a controversial topic here]? Somehow people came up with protocols to work with inherently dangerous material things that lower the risk of a serious injury to acceptable bounds. I don't see why this would not apply to inherently dangerous ideas. I'd hazard to say that for the last 2500 years quite a few devices were invented in this space, allowing to think much of the previously unthinkable without turning into ravaging monsters or becoming insane. I suspect that we could just continue using these devices (e.g. "freedom of speech", "critical thinking", separation of author's personality form their ideas, etc), instead of running in terror from "dangerous ideas" while swinging a banhammer.
- scrollaway 9y ago
  > Yes, some ideas are inherently dangerous. An idea of atoms splitting leads to an atomic bomb, for instance. You're missing the parent's point. An idea not being acted upon is not dangerous. The idea of splitting atoms is not dangerous and the atomic bomb is not dangerous: An atomic bomb getting built (the idea being acted upon) is dangerous. Why are full disclosure policies recommended by security researchers? Why is open source healthy for security in general? Why is "security through obscurity" frowned upon? Because it is important to be able to discuss threats in order to better protect yourself from them. Burying your head in the sand is just a shitty tactic for dealing with danger. Being able to discuss controversial ideas is extremely important, whether it's to better protect yourself from them (eg. extreme-right ideology) or to independently research facts hidden from you by people who don't have your best interests in mind (eg. north korean censorship). And let's be honest here, from what I've seen of my (left leaning) twitter feed, Sam is being attacked extremely harshly for his "controversial" post. If people here think that the article is wrong, this is a terrible way to prove it so.

staunch 9y ago

The idea that "ideas can be dangerous" is a dangerous idea.

jedharris 9y ago

The criticism quoted wasn't about ethical problems in genetic engineering research. It was about life extension as a goal. I can imagine a debate about that, but I can't see how it is appropriate to create a toxic environment for life extension advocates. Apparently these researchers / advocates found the environment so toxic they were motivated to move. Perhaps they were over-reacting but it seems likely something is wrong. And "move fast / break things" applied to human biology research is entirely an insertion of the commenter, not remotely an implication of Altman's piece.

ixtli 9y ago

It's critical to understand that it's not an intentional simplification. Altman and the rest of the rarified VC-connected people on both coasts are very, very disconnected from what its like to live in the lower or middle classes and especially what it's like to be a part of a systemically marginalized community.

rhizome 9y ago

What do you mean by "ethical outlook?" Who doesn't have one? You may not agree with their ethics, but they're there. If you're talking about shared ethics, that's known as morality.

Hattyflanagan 9y ago

That's exactly what the blog post wasn't realizing. Throwing out ethics is like throwing out law and order because it inhibits you from doing whatever you'd like to do. Does anyone really think they can stand upright against the winds when the weather turns nasty? Just like laws, ethics are there for a reason, and the social changes going on are happening for a reason.