9 ms·

The Road to Self-Reproducing Machines

44 points by jkuria 5y ago

haspoken 5y ago
https://archive.is/lvjvT https://archive.is/lvjvT
infogulch 5y ago
Maybe we should not.
ericbarrett 5y ago
Where is modern-day materials science with regard to self-replicating nanomachines? I know we aren’t there yet, but every year is closer than the one before. I have no insight into the state of the art, however. Downstream of that question, has there been any serious study into the likelihood of a “gray goo” scenario? Or is it too far over the horizon?
- throwawaysea 5y ago
  I think this is still science fiction. About ten years ago a lot of the focus was on DNA-centric self-replicating structures but it didn’t go anywhere interesting. More recent attempts (example: https://news.wsu.edu/2018/03/08/self-replicating-materials-2/ https://news.wsu.edu/2018/03/08/self-replicating-materials-2...) are still very much early stage, focusing on exploratory fundamental research.
sandworm101 5y ago
>> Self-reproducing machines could unleash the power of exponential growth, thus enabling audacious engineering projects. Grey goo. The Borg. Skynet. I think some audacious engineering might have to be curtailed if we flesh-and-bloods want to maintain some sort of relevancy. The key may be evolution. We may need to banish unexpected variation from all self-replicating machines. Variation would have to be limited to software. No automaton would be allowed to evolve away from having an off switch.
- noduerme 5y ago
  Not just an off switch, a kill switch. Anything that can reproduce only needs a software modification to change the structure of its hardware on the next generation. If a self reproducing machine's fitness is measured only by real world conditions, and if its software is free to mutate, it will become the dominant microscopic life form very quickly. At that point maybe only an EMP would wipe it out, if it were possible to wipe it out at all. If not, so long carbon based life.
  - deleted 5y ago
    [deleted]
  - mhb 5y ago
    Why not just keep it in sealed hardware that can’t affect the world? https://www.yudkowsky.net/singularity/aibox https://www.yudkowsky.net/singularity/aibox
    - someguyorother 5y ago
      There are ways to keep an AI in sealed hardware and making sure it can't affect the world, for instance by using an objective function that only deals with mathematics, and doesn't deal with the real world at all. E.g. the AI is given a fixed amount of hardware and told to produce an algorithm that solves some NP-complete problem (say integer programming) in expected time as close to polytime as possible, as well as a mathematical proof that the algorithm satisfies the claimed close-to-polytime complexity bound. Then humanity can just solve the NP-complete problems separately once they have the algorithm. This objective function doesn't care about the physical world -- it doesn't even know that a physical world exist -- and so it's about as likely to directly affect the physical world as MCTS or AlphaGo. The "AI is going to run out of control" is a very compelling narrative (as everybody who has read the Sorcerer's Apprentice understands). But that doesn't make it true. Beware the availability heuristic. (Incidentally, I think AI destroying mankind because it's too smart is an unlikely outcome. It's much easier for the AI to subvert the human-designed sensors linked to its objective function; and if the AI is sufficiently smart and the sensors aren't perfect, then it can always do so.)
smackeyacky 5y ago
If you don't understand the reference to grey goo, this Wired article from Bill Joy (ex Sun microsystems) is the first place I heard it. https://www.wired.com/2000/04/joy-2/ https://www.wired.com/2000/04/joy-2/
- geomark 5y ago
  Good article and references some excellent books.
reilly3000 5y ago
Where are these self-reproducing machines getting their materials? What substrate is so available that they couldn’t be contained? If they could thrive on a simple, commonplace chemical structure, while hasn’t life already done it? What energy will sustain them?
- system2 5y ago
  Solar can take care of the energy easily. For the materials, I would expect a prebuilt facility to autonomously mine things to provide raw materials. Easier said than done of course. Without true AI, it wouldn't be possible to make them 100% autonomous because eventually they would run out of materials.
- rogual 5y ago
  I would think it's probably possible for there to be effective organism designs that nature has never come up with, just because there's no continuous path of gradual improvements that leads there from single-celled life.
  - imtringued 5y ago
    I disagree. We are dependent on plants for food and oxygen. Specialization is key. If we build self reproducing machines they will consist of countless of sub species specializing in one task. Most of them will specialize in extraction of different resources.
- deleted 5y ago
  [deleted]
deleted 5y ago
[deleted]
rishikeshs 5y ago
I wish this was not behind a paywall!
playing_colours 5y ago
The title brought memories in me about two quite uncomfortable dystopian short stories by Philip Dick: Second Variety and Autofac.
thinkabout2 5y ago
This is a much bigger risk than an AI. Evolution will be in place there, and evolution will beat any kind of measure we want to set in place to control the output
chubot 5y ago
Hm this article is very light on details. I remember hearing about the RepRap ~15 years ago: https://reprap.org/wiki/RepRap https://reprap.org/wiki/RepRap It's an open source project to make a 3D printer that prints itself. I thought it was a very cool idea, but I haven't heard anything about it in 10 years. The home page has a video from 2010, which makes me think the project is dormant. I'd be interested in updates from people who have followed more closely. It looks like the article didn't mention any recent progress on ANY self-replicating machine project? It does feel like people get overexcited about downsides before anything actually works.
- tbrownaw 5y ago
  That's from when 3d printing had the same sort of, er, enthusiasm that Blockchain (and isn't it interesting that my phone decided that needed to be capitalized) has more recently. Proponents insisting that or was going to fundamentally change the world and all that.
  - fragmede 5y ago
    More recently? Bitcoin's initial release was 12 years ago, which isn't too far off from when 3D printing first started hitting the scene. 3D printing has changed the world, but because there's an expensive printer to get access to, vs Bitcoin's accessibility as pure software (you don't need to buy mining hardware up get into Bitcoin) means its reach has been limited to the corners of the world that has them.
- marcosdumay 5y ago
  Hum... You linked to the page of the original RepRap. Yeah, it was never too popular and everybody stopped using it long ago. People were replacing it since it was published. Last time I looked, the most popular RepRaps were variants of the Prusa i3 design (that one being more than 10 years old too). But there has been such a large explosion of designs some 5 or 6 years ago that it may be on the minority, even if it's the most popular one. Anyway, that wiki is full of great information that is perfectly applicable today. Just because the one design that gets the name of the project was improved into an unrecognizable format, it doesn't mean the idea died.

44 points by jkuria 5y ago

haspoken 5y ago

https://archive.is/lvjvT https://archive.is/lvjvT

infogulch 5y ago

Maybe we should not.

ericbarrett 5y ago

Where is modern-day materials science with regard to self-replicating nanomachines? I know we aren’t there yet, but every year is closer than the one before. I have no insight into the state of the art, however. Downstream of that question, has there been any serious study into the likelihood of a “gray goo” scenario? Or is it too far over the horizon?

throwawaysea 5y ago
I think this is still science fiction. About ten years ago a lot of the focus was on DNA-centric self-replicating structures but it didn’t go anywhere interesting. More recent attempts (example: https://news.wsu.edu/2018/03/08/self-replicating-materials-2/ https://news.wsu.edu/2018/03/08/self-replicating-materials-2...) are still very much early stage, focusing on exploratory fundamental research.

sandworm101 5y ago

>> Self-reproducing machines could unleash the power of exponential growth, thus enabling audacious engineering projects. Grey goo. The Borg. Skynet. I think some audacious engineering might have to be curtailed if we flesh-and-bloods want to maintain some sort of relevancy. The key may be evolution. We may need to banish unexpected variation from all self-replicating machines. Variation would have to be limited to software. No automaton would be allowed to evolve away from having an off switch.

noduerme 5y ago
Not just an off switch, a kill switch. Anything that can reproduce only needs a software modification to change the structure of its hardware on the next generation. If a self reproducing machine's fitness is measured only by real world conditions, and if its software is free to mutate, it will become the dominant microscopic life form very quickly. At that point maybe only an EMP would wipe it out, if it were possible to wipe it out at all. If not, so long carbon based life.
- deleted 5y ago
  [deleted]
- mhb 5y ago
  Why not just keep it in sealed hardware that can’t affect the world? https://www.yudkowsky.net/singularity/aibox https://www.yudkowsky.net/singularity/aibox
  - someguyorother 5y ago
    There are ways to keep an AI in sealed hardware and making sure it can't affect the world, for instance by using an objective function that only deals with mathematics, and doesn't deal with the real world at all. E.g. the AI is given a fixed amount of hardware and told to produce an algorithm that solves some NP-complete problem (say integer programming) in expected time as close to polytime as possible, as well as a mathematical proof that the algorithm satisfies the claimed close-to-polytime complexity bound. Then humanity can just solve the NP-complete problems separately once they have the algorithm. This objective function doesn't care about the physical world -- it doesn't even know that a physical world exist -- and so it's about as likely to directly affect the physical world as MCTS or AlphaGo. The "AI is going to run out of control" is a very compelling narrative (as everybody who has read the Sorcerer's Apprentice understands). But that doesn't make it true. Beware the availability heuristic. (Incidentally, I think AI destroying mankind because it's too smart is an unlikely outcome. It's much easier for the AI to subvert the human-designed sensors linked to its objective function; and if the AI is sufficiently smart and the sensors aren't perfect, then it can always do so.)

smackeyacky 5y ago

If you don't understand the reference to grey goo, this Wired article from Bill Joy (ex Sun microsystems) is the first place I heard it. https://www.wired.com/2000/04/joy-2/ https://www.wired.com/2000/04/joy-2/

geomark 5y ago
Good article and references some excellent books.

reilly3000 5y ago

Where are these self-reproducing machines getting their materials? What substrate is so available that they couldn’t be contained? If they could thrive on a simple, commonplace chemical structure, while hasn’t life already done it? What energy will sustain them?

system2 5y ago
Solar can take care of the energy easily. For the materials, I would expect a prebuilt facility to autonomously mine things to provide raw materials. Easier said than done of course. Without true AI, it wouldn't be possible to make them 100% autonomous because eventually they would run out of materials.
rogual 5y ago
I would think it's probably possible for there to be effective organism designs that nature has never come up with, just because there's no continuous path of gradual improvements that leads there from single-celled life.
- imtringued 5y ago
  I disagree. We are dependent on plants for food and oxygen. Specialization is key. If we build self reproducing machines they will consist of countless of sub species specializing in one task. Most of them will specialize in extraction of different resources.
deleted 5y ago
[deleted]

deleted 5y ago

[deleted]

rishikeshs 5y ago

I wish this was not behind a paywall!

playing_colours 5y ago

The title brought memories in me about two quite uncomfortable dystopian short stories by Philip Dick: Second Variety and Autofac.

thinkabout2 5y ago

This is a much bigger risk than an AI. Evolution will be in place there, and evolution will beat any kind of measure we want to set in place to control the output

chubot 5y ago

Hm this article is very light on details. I remember hearing about the RepRap ~15 years ago: https://reprap.org/wiki/RepRap https://reprap.org/wiki/RepRap It's an open source project to make a 3D printer that prints itself. I thought it was a very cool idea, but I haven't heard anything about it in 10 years. The home page has a video from 2010, which makes me think the project is dormant. I'd be interested in updates from people who have followed more closely. It looks like the article didn't mention any recent progress on ANY self-replicating machine project? It does feel like people get overexcited about downsides before anything actually works.

tbrownaw 5y ago
That's from when 3d printing had the same sort of, er, enthusiasm that Blockchain (and isn't it interesting that my phone decided that needed to be capitalized) has more recently. Proponents insisting that or was going to fundamentally change the world and all that.
- fragmede 5y ago
  More recently? Bitcoin's initial release was 12 years ago, which isn't too far off from when 3D printing first started hitting the scene. 3D printing has changed the world, but because there's an expensive printer to get access to, vs Bitcoin's accessibility as pure software (you don't need to buy mining hardware up get into Bitcoin) means its reach has been limited to the corners of the world that has them.
marcosdumay 5y ago
Hum... You linked to the page of the original RepRap. Yeah, it was never too popular and everybody stopped using it long ago. People were replacing it since it was published. Last time I looked, the most popular RepRaps were variants of the Prusa i3 design (that one being more than 10 years old too). But there has been such a large explosion of designs some 5 or 6 years ago that it may be on the minority, even if it's the most popular one. Anyway, that wiki is full of great information that is perfectly applicable today. Just because the one design that gets the name of the project was improved into an unrecognizable format, it doesn't mean the idea died.