4 ms·

Good questions. * One thing that can stop them is the law; it could be made illegal, which does stop a lot of behavior. International treaties often work too.

Good questions. * One thing that can stop them is the law; it could be made illegal, which does stop a lot of behavior. International treaties often work too. * Also, if it's considered out-of-bounds by the scientific community then it could ruin the scientists' career; that's probably not a risk that a talented scientist with a promising career would take. They couldn't even publish a paper, publish in arXiv, get feedback on their work, or participate in any online discussion. They couldn't go to a conference (there wouldn't be one in their field) or even tell anyone what they are working on. What a life to choose. * Significant progress likely requires more resources than could be mustered covertly, including financial, technological (where do you buy equipment and supplies?), and especially labor: Not only do they need talented PhD students and postdocs, but progress in most fields requires a whole community of scientists who build on each other's work, not one who discovers it all themself.

visarga 10y ago
The technology advances when it can advance, no matter what. The raise of surveillance would have happened even without NSA. The Deep Learning advancements happened because of faster computers, even though humans have discovered some of the best ideas 20+ years ago. <rant> There is potential for achieving superhuman intelligence by genetic selection. By scanning millions of people's DNA and their personal traits (predicting phenotype characteristics from genotype), they want to build a classifier that would rate a DNA before it got developed. Then they would be able to select embryos before IVF. This is done today too, but only to filter out some simple genetic defects. A selection of 1:2 gives a 5 IQ point boost. A selection of 1:1000 embryos gives a boost of 40 IQ points. If we repeat this over a few generations, humanity will evolve much faster. This iterative improvement by AI rating of genomes could be sped up by creating sperm and ova directly from embryo stem cells. This way we don't need to wait for a generation to pass to have the kids become parents. In a few years many DNA selection -> embryo iterations could be tested and selected and evolution would be sped up even more. In the short run, 40 to 80 years, we face the appearance of AGI and human-computer interfaces. We will evolve together with it, and this kind of DNA synthesis would be an essential tool for the future. Even if a few people have been enhanced, when they have kids with non-enhanced people the new genetic advancements could be passed to the whole population in a few generations. The final aim is to extend life, intelligence and perhaps gain immortality and become much more than we were before. Even if we don't do AI based DNA enhancement, we still need to prepare for the possible catastrophe at the point of singularity. We need to create databases of human embryos so we could create humanity back in case it is destroyed. I'd also put in these databases a copy of all the human produced data and culture, just in case they need to start from scratch. These databases would have to be hidden and remain incommunicado for long enough to make sure we didn't get wiped out. They could even be cared for by robots for millions of years before opening up, as an insurance ticket that we survive. </rant> My point is that we don't really control the evolution of these technologies. They will be developed because they can be developed. In the past they weren't developed because it was impossible. We, humans, created even the a-bomb immediately after we understood nuclear physics. We spied on people as soon as the internet (and even telephone) was available. Why would we be able to stop from making synthetic DNA? The temptation to prolong life or to enhance future children exists, the means also exist.
- hackuser 10y ago
  > The technology advances when it can advance, no matter what. I know it's a popular theory but I don't know that it stands up to scrutiny. Nuclear weapons technology hasn't advanced in many years, for example; the few who could do the R&D don't seem interested. Technologies involving cannibis and LSD have not advanced in large part, AFAICT, because they are illegal. How do you overcome those and other obstacles I raised above? Another: Powerful interests freeze development in many industries. Here's another way of thinking of it, more theoretically: There are an effectively infinite number of technologies that could be developed; only a few are chosen. Almost all tech is ignored. I think that calling it inevitable is a way to duck difficult questions and obligations, similar to the way that some blame the free market and self-interest for their choices (as if those are gods who forced the mortal's hands).
  - Zigurd 10y ago
    That's a good point, but there are many reasons to think it will be much harder to contain genomics than atomics: 1. You can sense atomic weapons materials at a distance. 2. It takes significant capital equipment to make atomic weapons. 3. It takes precision equipment to make atomic weapons. 4. While theoretical knowledge of atomic weapons is readily disseminated, it is easy to find and target experienced experts in the making of atomic weapons. I can't find corresponding "choke points" that could control genomics, even on the scale of a synthetic human genome.
- mathheaven 10y ago
  Genes don't encode all the necessary information for growing up. There are time windows for activation of high level functions. (Cats get blind if their time window for visual learning is not used). So you need to take into account the time window and the other interactions before trying to speed up intelligence.
- kanzure 10y ago
  > This iterative improvement by AI rating of genomes could be sped up by creating sperm and ova directly from embryo stem cells. This way we don't need to wait for a generation to pass to have the kids become parents. You don't need to create sperm or ova to create a genome. That's adding layers of indirection. Bostrom/Sandberg should study biology instead of handwaving like that. It's much faster to forego some of those steps completely. They are so busy being excited about "you don't need to wait an entire generation" that they don't realize "you don't need a sperm nor ova to rate a genome".

visarga 10y ago

The technology advances when it can advance, no matter what. The raise of surveillance would have happened even without NSA. The Deep Learning advancements happened because of faster computers, even though humans have discovered some of the best ideas 20+ years ago. <rant> There is potential for achieving superhuman intelligence by genetic selection. By scanning millions of people's DNA and their personal traits (predicting phenotype characteristics from genotype), they want to build a classifier that would rate a DNA before it got developed. Then they would be able to select embryos before IVF. This is done today too, but only to filter out some simple genetic defects. A selection of 1:2 gives a 5 IQ point boost. A selection of 1:1000 embryos gives a boost of 40 IQ points. If we repeat this over a few generations, humanity will evolve much faster. This iterative improvement by AI rating of genomes could be sped up by creating sperm and ova directly from embryo stem cells. This way we don't need to wait for a generation to pass to have the kids become parents. In a few years many DNA selection -> embryo iterations could be tested and selected and evolution would be sped up even more. In the short run, 40 to 80 years, we face the appearance of AGI and human-computer interfaces. We will evolve together with it, and this kind of DNA synthesis would be an essential tool for the future. Even if a few people have been enhanced, when they have kids with non-enhanced people the new genetic advancements could be passed to the whole population in a few generations. The final aim is to extend life, intelligence and perhaps gain immortality and become much more than we were before. Even if we don't do AI based DNA enhancement, we still need to prepare for the possible catastrophe at the point of singularity. We need to create databases of human embryos so we could create humanity back in case it is destroyed. I'd also put in these databases a copy of all the human produced data and culture, just in case they need to start from scratch. These databases would have to be hidden and remain incommunicado for long enough to make sure we didn't get wiped out. They could even be cared for by robots for millions of years before opening up, as an insurance ticket that we survive. </rant> My point is that we don't really control the evolution of these technologies. They will be developed because they can be developed. In the past they weren't developed because it was impossible. We, humans, created even the a-bomb immediately after we understood nuclear physics. We spied on people as soon as the internet (and even telephone) was available. Why would we be able to stop from making synthetic DNA? The temptation to prolong life or to enhance future children exists, the means also exist.

hackuser 10y ago
> The technology advances when it can advance, no matter what. I know it's a popular theory but I don't know that it stands up to scrutiny. Nuclear weapons technology hasn't advanced in many years, for example; the few who could do the R&D don't seem interested. Technologies involving cannibis and LSD have not advanced in large part, AFAICT, because they are illegal. How do you overcome those and other obstacles I raised above? Another: Powerful interests freeze development in many industries. Here's another way of thinking of it, more theoretically: There are an effectively infinite number of technologies that could be developed; only a few are chosen. Almost all tech is ignored. I think that calling it inevitable is a way to duck difficult questions and obligations, similar to the way that some blame the free market and self-interest for their choices (as if those are gods who forced the mortal's hands).
- Zigurd 10y ago
  That's a good point, but there are many reasons to think it will be much harder to contain genomics than atomics: 1. You can sense atomic weapons materials at a distance. 2. It takes significant capital equipment to make atomic weapons. 3. It takes precision equipment to make atomic weapons. 4. While theoretical knowledge of atomic weapons is readily disseminated, it is easy to find and target experienced experts in the making of atomic weapons. I can't find corresponding "choke points" that could control genomics, even on the scale of a synthetic human genome.
mathheaven 10y ago
Genes don't encode all the necessary information for growing up. There are time windows for activation of high level functions. (Cats get blind if their time window for visual learning is not used). So you need to take into account the time window and the other interactions before trying to speed up intelligence.
kanzure 10y ago
> This iterative improvement by AI rating of genomes could be sped up by creating sperm and ova directly from embryo stem cells. This way we don't need to wait for a generation to pass to have the kids become parents. You don't need to create sperm or ova to create a genome. That's adding layers of indirection. Bostrom/Sandberg should study biology instead of handwaving like that. It's much faster to forego some of those steps completely. They are so busy being excited about "you don't need to wait an entire generation" that they don't realize "you don't need a sperm nor ova to rate a genome".