3 ms·

If I could attach a software program to every replicating chunk of DNA, I'm pretty sure I could write a checksum and diagnostic and instructions to shut down if

by ericb 2y ago

If I could attach a software program to every replicating chunk of DNA, I'm pretty sure I could write a checksum and diagnostic and instructions to shut down if they failed. That's not a variable that does anything to feasibility.

krapp 2y ago
What would this "software program" be made of? Millions of years of evolution have already granted DNA the capacity for diagnostics, error correction and self-repair. Despite that, genetic replication is still error prone, and everything still fails, sometimes catastrophically. DNA is made of molecular bonds, and that "software" is necessarily also made of molecular bonds, there is no distinction between "software" and "hardware" at that level of physical granularity. You're not going to get perfect replication of anything over millions of generations. That violates the laws of thermodynamics. The system has to change, randomly and unexpectedly, and it has to fail over time. Entropy must accumulate. And that's not even taking into account the radioactive hellscape of interstellar space. This is the difference between mathematics and physics. Mathematicians can handwave away or ignore inconvenient or uninteresting complexities, and thus something like the infinite exponential progress of self-replicating probes across the galaxy seems obvious because the math is obvious, but reality doesn't allow that.
- ericb 2y ago
  > The system has to change, randomly and unexpectedly. Entropy must accumulate. Yes to the first two, but "has to fail over time" is something you are making up--not a law of physics. Local decreases of entropy happen continually. Resilient, error-checking, self-healing systems are possible.
  - krapp 2y ago
    Self-replicating probes aren't closed systems. They exist within the universe, consume matter (which is how they replicate) and emit waste heat (as all physical systems must,) and are bound to the laws of physics which, yes, include the second law of thermodynamics. There may be short term local reductions in entropy, but eventually, inevitably, entropy must increase. >Resilient, error-checking, self-healing systems are possible. You cannot have such systems be perfectly efficient. That isn't physics, it's magic.
    - ericb 2y ago
      No one suggested perfect efficiency. I'm debating with you, a product of a similar reproductive process (evolution) where a system of throwing away the bad offspring has worked just fine. Magical indeed!
      - ben_w 2y ago
        In fairness, that's a bad argument because this same process also gives us cancer. I'd instead point out that while "perfect" isn't possible, we can relatively simply design the system to have an error rate such that there's less than a 1e-12 chance of an error occurring anywhere in the universe even if you did turn the entire mass of the universe into probes. I'd counter that with the point that people are very bad at accounting for all the possible ways that systems can fail, and that while it's easy to create an error correction code that good, the actual failure rate of the system as a whole is likely to be much, much worse.
        nadermx 2y ago
        But the reason DNA is an apt analogy is because as we see in reality cancer does exist. So a self replicating probe would in theory be following same principles of DNA, it would develop a cancerous probe, that in theory could defeat the original design plan and maybe kill all previous probes. Or say by a bacteria on a planet laying latent that causes unexpected issues, etc.
        ben_w 2y ago
        DNA/cancer tells you mutation is possible, it does not say it is (in practice) mandatory. Organisms' mutation rate is not constant: there's a need to mutate more due to the need to be able to adapt over generations to novel threats, and a need to mutate less due to the risk of premature (before reproduction) death. This adaptability also explains why whales, which are much bigger than us, don't all get cancer almost immediately and die young; and also why dogs, which are much smaller, still often manage to get it despite only living to 12 or so. In computers… well, there's multiple layers of error correction. There's error correction in the link layer, in the transport layer with TCP, at the OSI application layer implicitly in TLS because errors would break the signing, and important files also get separate checksums (and these days security signatures) to be tested when the transfer is complete. These were all designed with arbitrary standards of "acceptable" error rates, there's nothing to prevent a new design with a different idea of what's "acceptable" and setting that threshold as low as I suggested, or even lower. But! Every time I see someone make a pronouncement that they're more than 99% confident of something they've never actually tested, I think they've likely not thought of all the ways their thing can go wrong. That means that while I can be confident that we can design a system that appears, according to every scenario we can imagine, to have less than a one-in-a-trillion chance of a mutation surviving even if every atom of the universe is converted into more von Neumann probes implementing that system, even then I still expect that if it were built it would rapidly encounter an outside context problem.

by ericb 2y ago

krapp 2y ago

What would this "software program" be made of? Millions of years of evolution have already granted DNA the capacity for diagnostics, error correction and self-repair. Despite that, genetic replication is still error prone, and everything still fails, sometimes catastrophically. DNA is made of molecular bonds, and that "software" is necessarily also made of molecular bonds, there is no distinction between "software" and "hardware" at that level of physical granularity. You're not going to get perfect replication of anything over millions of generations. That violates the laws of thermodynamics. The system has to change, randomly and unexpectedly, and it has to fail over time. Entropy must accumulate. And that's not even taking into account the radioactive hellscape of interstellar space. This is the difference between mathematics and physics. Mathematicians can handwave away or ignore inconvenient or uninteresting complexities, and thus something like the infinite exponential progress of self-replicating probes across the galaxy seems obvious because the math is obvious, but reality doesn't allow that.

ericb 2y ago
> The system has to change, randomly and unexpectedly. Entropy must accumulate. Yes to the first two, but "has to fail over time" is something you are making up--not a law of physics. Local decreases of entropy happen continually. Resilient, error-checking, self-healing systems are possible.
- krapp 2y ago
  Self-replicating probes aren't closed systems. They exist within the universe, consume matter (which is how they replicate) and emit waste heat (as all physical systems must,) and are bound to the laws of physics which, yes, include the second law of thermodynamics. There may be short term local reductions in entropy, but eventually, inevitably, entropy must increase. >Resilient, error-checking, self-healing systems are possible. You cannot have such systems be perfectly efficient. That isn't physics, it's magic.
  - ericb 2y ago
    No one suggested perfect efficiency. I'm debating with you, a product of a similar reproductive process (evolution) where a system of throwing away the bad offspring has worked just fine. Magical indeed!
    - ben_w 2y ago
      In fairness, that's a bad argument because this same process also gives us cancer. I'd instead point out that while "perfect" isn't possible, we can relatively simply design the system to have an error rate such that there's less than a 1e-12 chance of an error occurring anywhere in the universe even if you did turn the entire mass of the universe into probes. I'd counter that with the point that people are very bad at accounting for all the possible ways that systems can fail, and that while it's easy to create an error correction code that good, the actual failure rate of the system as a whole is likely to be much, much worse.
      - nadermx 2y ago
        But the reason DNA is an apt analogy is because as we see in reality cancer does exist. So a self replicating probe would in theory be following same principles of DNA, it would develop a cancerous probe, that in theory could defeat the original design plan and maybe kill all previous probes. Or say by a bacteria on a planet laying latent that causes unexpected issues, etc.
        ben_w 2y ago
        DNA/cancer tells you mutation is possible, it does not say it is (in practice) mandatory. Organisms' mutation rate is not constant: there's a need to mutate more due to the need to be able to adapt over generations to novel threats, and a need to mutate less due to the risk of premature (before reproduction) death. This adaptability also explains why whales, which are much bigger than us, don't all get cancer almost immediately and die young; and also why dogs, which are much smaller, still often manage to get it despite only living to 12 or so. In computers… well, there's multiple layers of error correction. There's error correction in the link layer, in the transport layer with TCP, at the OSI application layer implicitly in TLS because errors would break the signing, and important files also get separate checksums (and these days security signatures) to be tested when the transfer is complete. These were all designed with arbitrary standards of "acceptable" error rates, there's nothing to prevent a new design with a different idea of what's "acceptable" and setting that threshold as low as I suggested, or even lower. But! Every time I see someone make a pronouncement that they're more than 99% confident of something they've never actually tested, I think they've likely not thought of all the ways their thing can go wrong. That means that while I can be confident that we can design a system that appears, according to every scenario we can imagine, to have less than a one-in-a-trillion chance of a mutation surviving even if every atom of the universe is converted into more von Neumann probes implementing that system, even then I still expect that if it were built it would rapidly encounter an outside context problem.