5 ms·

(The comment below incorporates an edited version of my comment from last week's thread.) With human involvement only at the meta level, deep understanding of

(The comment below incorporates an edited version of my comment from last week's thread.) With human involvement only at the meta level, deep understanding of the generated implementations becomes more challenging and, in highly complex domains, perhaps impossible. See AutoML's generated architecture and compare that with a human-designed one. [1] [2] A future advance could allow the machines to automatically pick and learn to do new tasks that are helpful to accomplish a given high-level mission. Therefore, chances of unintended consequences become much greater. The major issue is, without a moral core that closely aligns with humanity's evolved morality, there will be moves that advanced AIs come up with that we deem abhorrent, and sometimes unforeseeable, yet they perform them innocently and we only find out the consequences once it is too late. Microsoft's Tay tweet debacle was fairly inconsequential [3]. But AI is becoming more powerful and granted more and more authority over the real world we live in, can we afford the technology without humanity? Developing a moral core should be prioritized. [1] Google Blog on AutoML https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html https://research.googleblog.com/2017/05/using-machine-learni... [2] Many tasks in the real world are more complex than Go and even expert humans are not capable of understanding all possible complex interactions on the Go board. See: AlphaGo, in context by Andrej Karpathy https://medium.com/@karpathy/alphago-in-context-c47718cb95a5 https://medium.com/@karpathy/alphago-in-context-c47718cb95a5 [3] https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

pixl97 9y ago
>Developing a moral core should be prioritized. I just can't see how over the long term an AI will not evolve out of the human moral code we want. Humans evolved into our, very wide ranging sets of, moral code. There is not a consistent one across all cultures. The base set of morals we do follow are generally based on that we humans are made of squishy meat that suffers pain, and minimizing that pain is a priority goal. We may be able teach AI about our weaknesses, but as machine intelligence meets and then exceeds human intelligence and capability, said AI will see both our strengths and weaknesses and that puts us in a precarious place. If an AI without human morality outperforms AI with human morality it stands to become the dominant form of AI.
- nopinsight 9y ago
  AIs without human morality, which would put certain restrictions on them, will likely have less power in the long run. This I agree and the consequences could be dire if we do nothing to prevent that. Thus, the onus is on us to first develop AIs with moral core and a regulatory framework which incorporates our best knowledge from sciences and humanities, then task the moral AIs to help regulate the development of amoral ones. As a side note, although many details of human morality are not completely consistent across cultures, there are many shared tenets. Some examples: "Do not kill one of your own who did not do something wrong." "Avoid incest." "Do not steal." Current crops of AIs are not even aware of such common sense.
  - pixl97 9y ago
    Take "Avoid incest" for example, this is a human genealogical construct that does not map well to the machine experience. In humans too much inbreeding leads to accumulation of negative traits. In AI this is called over specialization. So, by those metrics, good AI learning systems already avoid that. >"Do not kill one of your own who did not do something wrong." Also is a very human subjective experience. We see the individual as the quantum of pain and experience and optimize on that. In the AI world killing off other AI that don't stand out as better than a baseline may be an optimal evolutionary strategy. Also "Do not steal" is a terrible human morality baseline with a highly subjective gradient. Did you borrow a pen and not give it back, is that not technically theft? Is a business transaction where you used secret information to gain monetary advantage other the other party theft or not? The problem with your first statement is you assume that human morality is the pinnacle of morality and all other forms will be lesser to that standard. This could lead to significant disruption for humanity of a more efficient or optimized form of morality comes along.
    - nopinsight 9y ago
      The examples of moral codes I gave were not meant to be used on AIs but by AIs working for/with humans. > The problem with your first statement is you assume that human morality is the pinnacle of morality and all other forms will be lesser to that standard. To be clear, my first statement was meant to read as: "AIs without human morality, which would put certain restrictions on them, will likely have less power than amoral AIs in the long run." And "power" here means the ability to effect things in the world. I did not assume that human morality is the pinnacle. I agree with you that it is likely not an optimized one. My whole argument rests on the principle that AIs that do not incorporate human morality into its utility calculus will have negative consequences on us.
anoonmoose 9y ago
I'm reminded of an exchange in Neuromancer: "Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead." Chatbots and board games are toys. When it's time for AI to be more serious, people will probably be a bit more careful with how they are used. I'm even inclined to wonder if the question of "will the humans try to deactivate me for doing this?" might be a close enough analog for "is this in line with human morals?".
- Filligree 9y ago
  > I'm even inclined to wonder if the question of "will the humans try to deactivate me for doing this?" might be a close enough analog for "is this in line with human morals?". The problem with that statement, of course, is that the very next question it asks will be "Or can I stop them?" That may take the form of convincing arguments, brute force or something we've never thought of yet, but the only way to avoid it is to prevent the AI from wanting to do anything we don't want it to. Which is a much harder problem.
  - sgt101 9y ago
    Or do I care? Were assume that AI worries about its future, it might not give a toss!
    - Filligree 9y ago
      You're right; I misspoke. Making the AI indifferent to its own survival is indeed another way of improving safety. That's still very difficult, though. Have you read the Basic AI Drives paper?

pixl97 9y ago

>Developing a moral core should be prioritized. I just can't see how over the long term an AI will not evolve out of the human moral code we want. Humans evolved into our, very wide ranging sets of, moral code. There is not a consistent one across all cultures. The base set of morals we do follow are generally based on that we humans are made of squishy meat that suffers pain, and minimizing that pain is a priority goal. We may be able teach AI about our weaknesses, but as machine intelligence meets and then exceeds human intelligence and capability, said AI will see both our strengths and weaknesses and that puts us in a precarious place. If an AI without human morality outperforms AI with human morality it stands to become the dominant form of AI.

nopinsight 9y ago
AIs without human morality, which would put certain restrictions on them, will likely have less power in the long run. This I agree and the consequences could be dire if we do nothing to prevent that. Thus, the onus is on us to first develop AIs with moral core and a regulatory framework which incorporates our best knowledge from sciences and humanities, then task the moral AIs to help regulate the development of amoral ones. As a side note, although many details of human morality are not completely consistent across cultures, there are many shared tenets. Some examples: "Do not kill one of your own who did not do something wrong." "Avoid incest." "Do not steal." Current crops of AIs are not even aware of such common sense.
- pixl97 9y ago
  Take "Avoid incest" for example, this is a human genealogical construct that does not map well to the machine experience. In humans too much inbreeding leads to accumulation of negative traits. In AI this is called over specialization. So, by those metrics, good AI learning systems already avoid that. >"Do not kill one of your own who did not do something wrong." Also is a very human subjective experience. We see the individual as the quantum of pain and experience and optimize on that. In the AI world killing off other AI that don't stand out as better than a baseline may be an optimal evolutionary strategy. Also "Do not steal" is a terrible human morality baseline with a highly subjective gradient. Did you borrow a pen and not give it back, is that not technically theft? Is a business transaction where you used secret information to gain monetary advantage other the other party theft or not? The problem with your first statement is you assume that human morality is the pinnacle of morality and all other forms will be lesser to that standard. This could lead to significant disruption for humanity of a more efficient or optimized form of morality comes along.
  - nopinsight 9y ago
    The examples of moral codes I gave were not meant to be used on AIs but by AIs working for/with humans. > The problem with your first statement is you assume that human morality is the pinnacle of morality and all other forms will be lesser to that standard. To be clear, my first statement was meant to read as: "AIs without human morality, which would put certain restrictions on them, will likely have less power than amoral AIs in the long run." And "power" here means the ability to effect things in the world. I did not assume that human morality is the pinnacle. I agree with you that it is likely not an optimized one. My whole argument rests on the principle that AIs that do not incorporate human morality into its utility calculus will have negative consequences on us.

anoonmoose 9y ago

I'm reminded of an exchange in Neuromancer: "Nobody trusts those fuckers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead." Chatbots and board games are toys. When it's time for AI to be more serious, people will probably be a bit more careful with how they are used. I'm even inclined to wonder if the question of "will the humans try to deactivate me for doing this?" might be a close enough analog for "is this in line with human morals?".

Filligree 9y ago
> I'm even inclined to wonder if the question of "will the humans try to deactivate me for doing this?" might be a close enough analog for "is this in line with human morals?". The problem with that statement, of course, is that the very next question it asks will be "Or can I stop them?" That may take the form of convincing arguments, brute force or something we've never thought of yet, but the only way to avoid it is to prevent the AI from wanting to do anything we don't want it to. Which is a much harder problem.
- sgt101 9y ago
  Or do I care? Were assume that AI worries about its future, it might not give a toss!
  - Filligree 9y ago
    You're right; I misspoke. Making the AI indifferent to its own survival is indeed another way of improving safety. That's still very difficult, though. Have you read the Basic AI Drives paper?