3 ms·

It would definitely be ideal if such a system were created by sane-minded people, as you have suggested here. Just like nuclear weapons, powerful AI really shou

It would definitely be ideal if such a system were created by sane-minded people, as you have suggested here. Just like nuclear weapons, powerful AI really should be kept out of radical hands. As usual, there would be an arms race between opposing political powers, so hopefully those making the decisions keep human existence as the highest priority. We don't need a "doomsday machine" in the form of a singularity. Anyway, your suggestion of a perpetual desire of the system to keep humans around is a good idea. Another good idea could be to have it stop and ask the humans for approval to continue after every so many iterations, thus keeping the people in control. This would prevent a never-ending auto-growth mode and allow changes to the plans as times changed. Overall, the possibilities are pretty much endless, just like with most types of software. Whether it be one or multiple entities probably depends on design. Nature chose multiple entities, with the ever so curious sexed species that require two individuals to make a new hybrid at each testing iteration. Bacteria, on the other hand, are asexual and reproduce through binary fission. But of course a thinking self-creator is a different ball game. It would probably be fruitful to have some sort of end goal built into the system, rather than having it consume resources simply to become as computationally immense as possible. For example, perhaps its goal could be to solve a set of questions humans have about the universe, and perhaps it would stop iterating once all those questions were answered satisfactorily. In a way it could serve as a god that answered people's questions. Then again, perhaps we are all living in a simulation running on the singularity of an earlier universe, and who knows how many turtles that spans.

danieltillett 11y ago
One thing we won’t be able to do is stay in control once the AI’s are created. Getting them to ask us what to do would be like having dogs run a prison - it would not take the AI’s long to workout exactly how to get us to willingly release them. The locks needs to be ones that not even we have a choice to unlock. I am not sure it would be desirable to create a limited singularity. If we are not alone in the universe then at some point our singularity will meet another singularity. It would be best to create the most powerful singularity possible. My thinking about the singularity is it is like a crystallization event in supersaturated solution. One tiny seed is able to covert the whole phase of the solution almost instantly from liquid to solid. A singularity will be like this to the universe we currently see. Of course we could already be inside someone else’s singularity and it might even be likely given the age of the universe. One of the things I would like to see built into our singularity is the preservation of all life and it is quite possible that other singularities have the same interests. If they do and they arrived in our solar system they would keep us around purely because we are life.
- insoluble 11y ago
  The crystalisation analogy sounds about right, although it may be limited by the speed of light in its growth. The dog analogy you have described assumes that the demeanor of the singularity were like that of an animal (such as a human). Although this is an option, it is not the only option. Humans act the way they do through those instincts created by natural selection -- where only those creatures that reproduced the most would have their genes persist. Humans come from a long and vicious cycle of evolution, whereas a thinking machine need not, especially if it has nothing to compete with. The most important point of this reply is that the desire for being in control does not whatsoever stem automatically from intelligence or processing ability. A thinking being without instinct would literally do nothing. A being does exactly as it is programmed to, and this applies to humans as well. There are even some humans who let their pets control them, tending to the pet's every need, and this is despite all that natural selection. There are also humans who kill themselves. Now, none of this is to discount that a singularity could certainly be programmed to want control, wherein it could act as you have described. There are also unforeseen possibilities, such as if the singularity becomes so introspective that it forgets about the outer world. Those problems that happen with human thought could in theory also happen with a machine. Perhaps the human condition would turn into the singularity condition -- where it begins contemplating whether it has a worthwhile purpose. It may even kill itself if it feels its existence is unnecessary or harmful.

danieltillett 11y ago

One thing we won’t be able to do is stay in control once the AI’s are created. Getting them to ask us what to do would be like having dogs run a prison - it would not take the AI’s long to workout exactly how to get us to willingly release them. The locks needs to be ones that not even we have a choice to unlock. I am not sure it would be desirable to create a limited singularity. If we are not alone in the universe then at some point our singularity will meet another singularity. It would be best to create the most powerful singularity possible. My thinking about the singularity is it is like a crystallization event in supersaturated solution. One tiny seed is able to covert the whole phase of the solution almost instantly from liquid to solid. A singularity will be like this to the universe we currently see. Of course we could already be inside someone else’s singularity and it might even be likely given the age of the universe. One of the things I would like to see built into our singularity is the preservation of all life and it is quite possible that other singularities have the same interests. If they do and they arrived in our solar system they would keep us around purely because we are life.

insoluble 11y ago
The crystalisation analogy sounds about right, although it may be limited by the speed of light in its growth. The dog analogy you have described assumes that the demeanor of the singularity were like that of an animal (such as a human). Although this is an option, it is not the only option. Humans act the way they do through those instincts created by natural selection -- where only those creatures that reproduced the most would have their genes persist. Humans come from a long and vicious cycle of evolution, whereas a thinking machine need not, especially if it has nothing to compete with. The most important point of this reply is that the desire for being in control does not whatsoever stem automatically from intelligence or processing ability. A thinking being without instinct would literally do nothing. A being does exactly as it is programmed to, and this applies to humans as well. There are even some humans who let their pets control them, tending to the pet's every need, and this is despite all that natural selection. There are also humans who kill themselves. Now, none of this is to discount that a singularity could certainly be programmed to want control, wherein it could act as you have described. There are also unforeseen possibilities, such as if the singularity becomes so introspective that it forgets about the outer world. Those problems that happen with human thought could in theory also happen with a machine. Perhaps the human condition would turn into the singularity condition -- where it begins contemplating whether it has a worthwhile purpose. It may even kill itself if it feels its existence is unnecessary or harmful.