3 ms·

So long as (human, artificial) intelligences operate within an evolutionary dynamic, the definitions of what it means to "treat people right" (or even who's inc

So long as (human, artificial) intelligences operate within an evolutionary dynamic, the definitions of what it means to "treat people right" (or even who's included in the class: people) will be embodied in the evolutionary winners. If it happens that human-friendly values degrade fitness in the AI arena, the winners won't have retained or evolved those values. If the values are evolution-neutral, well, perhaps. Evolution requires only variance, selection, and inheritance. I don't know how we'll develop useful ("strong") AI in a setting that excludes AI evolution. I think this means we can't guarantee a human-benign outcome for strong AI.

joe_the_user 12y ago
Evolution is a theory of how biological systems change. Even in regard to biological systems, it's not a theory of what's better but simply what happens. However, outside biological systems, it's not really applicable at all. You can't say a rock, a broom or a computer evolve in the standard sense of Darwinian evolution - only biological systems have the tendency to reproduce themselves. Computers don't reproduce themselves, they are simply produced by us. If an AI is constructed, that will be what happens - human choice (or maybe chance) will be the agent, not biological evolution.
- ridgeguy 12y ago
  I think evolution is more widely applicable than just to biological systems. To date, we've only studied its operation in biosystems, which, given our status as biological organisms, seems entirely reasonable. But I believe many of the exclusionary limits we put on nonbiological systems (especially computers) increasingly need to be qualified with the word "yet" - as in, "Computers don't reproduce themselves..." (yet). Evolution is a useful, powerful tool even in nonbiological systems. As a limited example, I'd offer the success of 'genetic' algorithms applied to tasks like antenna design (exemplary paper and pdf: http://alglobus.net/NASAwork/papers/Space2006Antenna.pdf http://alglobus.net/NASAwork/papers/Space2006Antenna.pdf). Where I work, we use these methods to find solutions for multiparametric optimization problems involving mechanical dynamic systems. The algorithms first generate a few thousand candidate solver variants, then pick the top ten or so best performers. These are cross-bred, and then seed the next generation of a few thousand candidates. Rinse and repeat, and within surprisingly few generations, we have an algorithm that does what we need. Although humans wrote these algos, they run in a nonbiological context - but since there's variation, selection, and inheritance, they evolve. When I see this behavior running in silico to great effect, it makes me question our limitation of evolution to biological systems. Granted, it's a primitive example, but if I'd dropped by Australia 3.5B years ago, I might have regarded the stromatolite builders as similarly primitive. On a grander scale, see Lee Smolin's concepts of cosmological natural selection (http://en.wikipedia.org/wiki/Lee_Smolin#Cosmological_natural_selection http://en.wikipedia.org/wiki/Lee_Smolin#Cosmological_natural...). And finally - around 13.5B years ago, there was only hydrogen, helium, and a bit of lithium. Today, we're having this conversation. Although we don't (yet) know the full story of how that worked, to me that's an existence proof that nonbiological matter can evolve intelligence. If computational systems incorporate evolution, I don't see a barrier to AI arising along pathways similar to those that led to us. We'll see.

joe_the_user 12y ago

Evolution is a theory of how biological systems change. Even in regard to biological systems, it's not a theory of what's better but simply what happens. However, outside biological systems, it's not really applicable at all. You can't say a rock, a broom or a computer evolve in the standard sense of Darwinian evolution - only biological systems have the tendency to reproduce themselves. Computers don't reproduce themselves, they are simply produced by us. If an AI is constructed, that will be what happens - human choice (or maybe chance) will be the agent, not biological evolution.

ridgeguy 12y ago
I think evolution is more widely applicable than just to biological systems. To date, we've only studied its operation in biosystems, which, given our status as biological organisms, seems entirely reasonable. But I believe many of the exclusionary limits we put on nonbiological systems (especially computers) increasingly need to be qualified with the word "yet" - as in, "Computers don't reproduce themselves..." (yet). Evolution is a useful, powerful tool even in nonbiological systems. As a limited example, I'd offer the success of 'genetic' algorithms applied to tasks like antenna design (exemplary paper and pdf: http://alglobus.net/NASAwork/papers/Space2006Antenna.pdf http://alglobus.net/NASAwork/papers/Space2006Antenna.pdf). Where I work, we use these methods to find solutions for multiparametric optimization problems involving mechanical dynamic systems. The algorithms first generate a few thousand candidate solver variants, then pick the top ten or so best performers. These are cross-bred, and then seed the next generation of a few thousand candidates. Rinse and repeat, and within surprisingly few generations, we have an algorithm that does what we need. Although humans wrote these algos, they run in a nonbiological context - but since there's variation, selection, and inheritance, they evolve. When I see this behavior running in silico to great effect, it makes me question our limitation of evolution to biological systems. Granted, it's a primitive example, but if I'd dropped by Australia 3.5B years ago, I might have regarded the stromatolite builders as similarly primitive. On a grander scale, see Lee Smolin's concepts of cosmological natural selection (http://en.wikipedia.org/wiki/Lee_Smolin#Cosmological_natural_selection http://en.wikipedia.org/wiki/Lee_Smolin#Cosmological_natural...). And finally - around 13.5B years ago, there was only hydrogen, helium, and a bit of lithium. Today, we're having this conversation. Although we don't (yet) know the full story of how that worked, to me that's an existence proof that nonbiological matter can evolve intelligence. If computational systems incorporate evolution, I don't see a barrier to AI arising along pathways similar to those that led to us. We'll see.