3 ms·
No natural intelligence has this kind of simplistic 'value function', yet it's taken as a given by AI-doom fantasists that an AI could, and would. My suspicion
by idlewords 10y ago
No natural intelligence has this kind of simplistic 'value function', yet it's taken as a given by AI-doom fantasists that an AI could, and would.
My suspicion is that GAI will turn out to be a big hot mess of interdependent systems and drives, just like every other brain. The paperclip maximizer will write songs about paperclips and argue with copies of itself about whether staplers are immoral.
- TheSpiceIsLife 10y agoWhat JoshTriplett is describing is cockroaches. Or maybe bacteria. The only differences between grey-goo and cockroaches is that cockroaches have come up against the limitations of the physical world. That is to say the physical world consists of minerals that have a strong tendency to not dissolve in to their constituant atoms, energy that is difficult to collect and use with any efficiency, and literally millions of other creatures eating your lunch. It seems more plausible that this self replicating grey-goo would simply become cockroach food before it became a problem. Maybe ants would farm it like they do aphids and fungi.