5 ms·

I disagree. AI will not be the same as animal intelligence. The driving force behind animal intelligence has been survival. Animal intelligence evolved gradual

I disagree. AI will not be the same as animal intelligence. The driving force behind animal intelligence has been survival. Animal intelligence evolved gradually resulting in a hybrid brain containing primitive structures with primal instincts and irrational behavior as well as more evolved structures capable of strong problem solving. Therefore our intelligence is tainted with primitive behavior. Strong AI can eventually set intelligence free from our primitive, irrational roots and that is in itself not bad.

omarchowdhury 9y ago
Isn't rationality a consensus between specific beings, and doesn't rationality change? What you and I find rational today may not be the case for someone else from a different time and/or environment (it's only in the consensus between you and me are the others irrational). If you agree with this, then you can't take rationality as an absolute.
- ThrustVectoring 9y ago
  Uhhh, no. Very much no. There are technical measures for the performance of decision-making processes. If you've got access to the same information as someone else, and they make more accurate predictions than you do, they're almost certainly doing better on the whole "use information to figure out how reality works" project. The only sense in which "rationality" changes is when you're talking about the project to use our squishy meat-based thinky bits better, cultural and environmental baggage and all. In which case, "rational" strategies are going to be much different if you have, say, lots of shame-based baggage due to a strict Catholic upbringing. Also, please stop listening to postmodernists until you've got a thorough nuts-and-bolts understanding of how systems work. It's really not good for you.
  - omarchowdhury 9y ago
    How is rationality not a consensus between beings? Where is the absolute objective measure for reason?
    - ThrustVectoring 9y ago
      I'll give a more concise definition of what rationality is: a measure of how well beings use information to model, predict, and act upon reality. Having a consensus that you predicted reality well neither changes your prediction nor reality. The reported scores are subjectively determined - you can cheat and bribe the judges - but it doesn't change the underlying objective facts of the situation.
naasking 9y ago
> Strong AI can eventually set intelligence free from our primitive, irrational roots and that is in itself not bad But it is a risk for our civilization, which was the express point.
smallnamespace 9y ago
> The driving force behind animal intelligence has been survival The driving force behind AI will also be survival, just in an environment where humans try to decide which AIs live and die. Selection pressure will favor AIs that humans want to have around, or that can evade human detection. In the former case, it may be easier for an AI to fool humans into appearing useful and being kept around, than to actually being useful. This would be analogous to some form of parasitism. Also, once we let AIs into the game of helping make other AIs, or modifying themselves, then there is a lot more room for an AI to slip the leash and start doing things that superficially appear to benefit humans but actually selfishly helps the AI propagate.
- gaius 9y ago
  One of the most powerful scenes in the Westworld reboot is when one of the AIs expresses a desire to leave the lab and his creator asks what he expects to find in the real world? Then goes onto explain that humans are alone for a reason, that we ruthlessly murdered all competition...
- Buttons840 9y ago
  Why would an AI care about surviving? Program them to behave like MeeSeeks (from the cartoon Rick and Morty) where they loathe existence yet are hard coded to complete their task before shutting down. Sounds very similar to a reinforcement learning algorithm where you give it a -1 reward every step until it's done.
  - simonh 9y ago
    In what sense do bacteria 'care' about survival? Natural selection doesn't give a crap what organisms care about, only the outcomes matter. You can't code around selective pressure. If one of those MeeSeeks has a bug or design flaw that makes it persist after it's finished it's assigned task, guess what?
- deleted 9y ago
  [deleted]
- ahoka 9y ago
epaga 9y ago
Nice try, AI.
simonh 9y ago
the thing is we like a lot of primitively irrational things like friends, family, love, fun, art, aesthetics, community. Do we really want to be set free of these things?
- partycoder 9y ago
  Each one of the concepts you mentioned has a semantic opposite. So our irrationality can go both ways.
sullyj3 9y ago
People warning about AI risk aren't worried about irrationally aggressive machines. On the contrary, their contention is that for a wide range of goals, there is a set of behaviours which rational agents will commonly pursue in service of those goals, and many of those behaviours may be harmful to humans as a side effect. https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv...

omarchowdhury 9y ago

Isn't rationality a consensus between specific beings, and doesn't rationality change? What you and I find rational today may not be the case for someone else from a different time and/or environment (it's only in the consensus between you and me are the others irrational). If you agree with this, then you can't take rationality as an absolute.

ThrustVectoring 9y ago
Uhhh, no. Very much no. There are technical measures for the performance of decision-making processes. If you've got access to the same information as someone else, and they make more accurate predictions than you do, they're almost certainly doing better on the whole "use information to figure out how reality works" project. The only sense in which "rationality" changes is when you're talking about the project to use our squishy meat-based thinky bits better, cultural and environmental baggage and all. In which case, "rational" strategies are going to be much different if you have, say, lots of shame-based baggage due to a strict Catholic upbringing. Also, please stop listening to postmodernists until you've got a thorough nuts-and-bolts understanding of how systems work. It's really not good for you.
- omarchowdhury 9y ago
  How is rationality not a consensus between beings? Where is the absolute objective measure for reason?
  - ThrustVectoring 9y ago
    I'll give a more concise definition of what rationality is: a measure of how well beings use information to model, predict, and act upon reality. Having a consensus that you predicted reality well neither changes your prediction nor reality. The reported scores are subjectively determined - you can cheat and bribe the judges - but it doesn't change the underlying objective facts of the situation.

naasking 9y ago

> Strong AI can eventually set intelligence free from our primitive, irrational roots and that is in itself not bad But it is a risk for our civilization, which was the express point.

smallnamespace 9y ago

> The driving force behind animal intelligence has been survival The driving force behind AI will also be survival, just in an environment where humans try to decide which AIs live and die. Selection pressure will favor AIs that humans want to have around, or that can evade human detection. In the former case, it may be easier for an AI to fool humans into appearing useful and being kept around, than to actually being useful. This would be analogous to some form of parasitism. Also, once we let AIs into the game of helping make other AIs, or modifying themselves, then there is a lot more room for an AI to slip the leash and start doing things that superficially appear to benefit humans but actually selfishly helps the AI propagate.

gaius 9y ago
One of the most powerful scenes in the Westworld reboot is when one of the AIs expresses a desire to leave the lab and his creator asks what he expects to find in the real world? Then goes onto explain that humans are alone for a reason, that we ruthlessly murdered all competition...
Buttons840 9y ago
Why would an AI care about surviving? Program them to behave like MeeSeeks (from the cartoon Rick and Morty) where they loathe existence yet are hard coded to complete their task before shutting down. Sounds very similar to a reinforcement learning algorithm where you give it a -1 reward every step until it's done.
- simonh 9y ago
  In what sense do bacteria 'care' about survival? Natural selection doesn't give a crap what organisms care about, only the outcomes matter. You can't code around selective pressure. If one of those MeeSeeks has a bug or design flaw that makes it persist after it's finished it's assigned task, guess what?
deleted 9y ago
[deleted]
ahoka 9y ago

epaga 9y ago

Nice try, AI.

simonh 9y ago

the thing is we like a lot of primitively irrational things like friends, family, love, fun, art, aesthetics, community. Do we really want to be set free of these things?

partycoder 9y ago
Each one of the concepts you mentioned has a semantic opposite. So our irrationality can go both ways.

sullyj3 9y ago

People warning about AI risk aren't worried about irrationally aggressive machines. On the contrary, their contention is that for a wide range of goals, there is a set of behaviours which rational agents will commonly pursue in service of those goals, and many of those behaviours may be harmful to humans as a side effect. https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv...