8 ms·

Depending on ones level of idealism, the best defense against malicious AI is actually educating programmers / hackers / engineers on the implications of their

by bem94 9y ago

Depending on ones level of idealism, the best defense against malicious AI is actually educating programmers / hackers / engineers on the implications of their work and to have a sense of decency & foresight. It baffles me that we just accept the kind of malice inflicted on people by programmers because "someone will always do it". As a profession / collection of skilled persons, we should really be better than that. Obviously, one cannot see the future, nor would we want to be paralized by fear of doing anything. But there is a certain minimum requirement for collective responsiblity which I really don't think we are meeting at the moment.

skate22 9y ago
Ethics can get pretty subjective, so I think it would be hard to teach a uniform view of decency. Also I hate to sounds pessimistic, but there are a lot of smart people who do much worse things than create malicious software when money is at stake. I think educating the people is a step in the right direction, but I fear it will fall short of your ideal.
- bem94 9y ago
  >> I think it would be hard to teach a uniform view of decency Ofcourse, but we don't avoid studying moral philosophy because we can't find a uniform model. Nor does not being able to find a uniform model mean that there are not things worth learning from it. >> there are a lot of smart people who do much worse things than create malicious software when money is at stake Agreed. Two wrongs do not make a right, and as engineers in the know, we have a duty to stand up to such people when we have the chance. >> I fear it will fall short of your ideal. Me too, but the alternative is worse in my opinion.
amelius 9y ago
The problem is that if engineers in the US don't take the job then someone somewhere in, say, Asia will. Ethical rules are not the same everywhere. You can already tell this by looking at experimentation with stem-cells, cloning, et cetera, which is happening faster in Asia than in the US because of regulations. Also, refraining from building these systems means that other countries get ahead of us. Besides being an economic disadvantage, this could also threaten security. So I do agree that engineers should adhere to ethical guidelines, but they should be considered in a global context.
- cmahler7 9y ago
  this, we need to wake up to the fact that China is willing to do whatever it takes to dominate. They have done massive research on genetics and intelligence and the top schools in china make parents take IQ tests to determine potential for children when accepting students. They don't care about diversity or equality, only results. The US and Europe need to wake up or we'll be facing a billion hypernationalist, gentically modified super-geniuses while the only thing we have is the moral highground. Based on what I've seen over the last decade we should probably start learning Mandarin.
  - joncampbelldev 9y ago
    I'm learning mandarin and have a chinese girlfriend, fully ready for the takeover when it happens ;)
bad_alloc 9y ago
Decency and foresight is highly subjective. Actually encoding what we understand as "decent human behaviour" into AI is a huge problem: If you create a system that decides about giving somebody a loan in the US based on a financial data set it might come up with any of these hypotheses: * Don't give loans to people living in [poor area]. * Avoid people with names that aren't similar to the most common ones in the database (i.e. foreign ones) * When linking the customer data to their social media and you see the picture is dissimilar to [preferred ethnicity], do not give a loan. Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1]. It knows that poor people pay back loans less frequently and makes the rational decision to not give a loan to that group. This of course reinforces the problem. But how are we to solve this? Introduce an additional column "race" to the data and bias the results with it? Would that not be just as racist? How do we give the system an awareness not to discriminate against ethnic groups, if the data contains implicit clues? This comes down to giving such an AI human intuition about such questions. [1] http://www.ssc.wisc.edu/irpweb/faqs/faq3/Figure1.png http://www.ssc.wisc.edu/irpweb/faqs/faq3/Figure1.png
- bem94 9y ago
  I'm not saying teach people some universal moral code, or suggesting we can accurately encode a set of appropriate morals into an AI. Just that people stop and think if what they are doing is appropriate. It seems silly to take problems which humans cannot solve with all the data, intuition and humanity that we have, and then use that data to train machines to make decisions, we know will be flawed.
- bko 9y ago
  >Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1]. If the data really suggests that these factors (poor areas, foreign names, social media) affect liklihood of default, should they be completely ignored? I'm sure credit scores and salaries are correlated to race in the US too. Should those be ignored? Just give out loans indiscriminately? They're not ignored now by loan approval. Worse, inaccurate stereotypes are used as a proxy for some of these factors that may be relevant to a borrowers ability to pay. We don't live in a perfect world where people ignore certain factors.
andbberger 9y ago
Sorry that's how natural selection works. When there's money at stake, someone will always do it. edit: typo
- bem94 9y ago
  And people like that are why we can't have nice things. I'm not saying that education / provoking thought will stop everyone doing something (else) monstrously stupid with AI. Just that we should be doing that educating / provoking more than we currently are.
- ythn 9y ago
  Even to the detriment of themselves... see "Tragedy of the Commons"
NickM 9y ago
I don't think that's the best defense, because even if 99.999% of programmers are perfect, ethical, decent people, all it takes is one psycho to go ahead and let the genie out of the bottle anyway. Imagine there's a big red "end the world" button that anyone on the planet can push. No matter how idealistic and utopian your views are, do you really trust that there's not a single person out of 7 billion people that might push the button? That is the kind of situation I worry we may face once people figure out how to create strong AI.
- bem94 9y ago
  Turning what I think you are saying on its head: There is a 0.001% chance of something bad happening, so there is no point in the other 99.999% changing their behaviour in order to stop that bad thing happening. I don't think that follows either. Second guessing human nature to the point where you resign yourself to catastrophe is surely less worthwhile than striving to create a sense of collective responsibility before it is too late?
  - NickM 9y ago
    I'm not saying there's an 0.0001% chance of it happening, I'm saying that if you give enough people a chance to create an AI, eventually someone will do it, even if 99.999% of people would choose not to. Right now, nobody knows how to do this, but my point is that eventually people will figure out how: once everyone knows how, then it's just a matter of time before someone does it. I'm not second guessing human nature, just saying people do crazy things all the time. The vast, vast majority of people are not suicide bombers, for example, but it still happens sometimes. It doesn't matter how much collective responsibility we have; eventually you may just end up with a brilliant programmer with schizophrenia or something who writes an AI in his basement and unleashes it on the world. I'm also certainly not resigning myself to catastrophe. There are probably ways to prevent or mitigate the negative impact of AI, like the one being suggested in the original article, for instance. I would hardly consider myself a cynic, but forgive me if I'm skeptical of the idea that every human in the world can somehow agree to never do anything that might create a dangerous AI. If it can happen, it eventually will happen.
MarkMMullin 9y ago
By that logic the gamma knife that has saved many people with cancer would not exist - before practical applications comes laying the basic foundation, and once that foundation is down, the genie is out of the bottle - I'm working on an autonomous drone, I am of course deeply opposed to weaponizing what I know is a ridiculously stupid system, but I know that it will happen - implication is not action, and knowledge is mostly neutral, it's the application of that knowledge that might be problematic - whichever way you slice it, it's a really complex issue, and anything worth having is often capable of both great and terrible things, depending on the application. AI is way too primitive to be malicious on it's own and will remain that way for years - now, as for abhorrent malicious uses of what we have now, here's a great example -- https://goo.gl/HmBY9k https://goo.gl/HmBY9k

by bem94 9y ago

skate22 9y ago

Ethics can get pretty subjective, so I think it would be hard to teach a uniform view of decency. Also I hate to sounds pessimistic, but there are a lot of smart people who do much worse things than create malicious software when money is at stake. I think educating the people is a step in the right direction, but I fear it will fall short of your ideal.

bem94 9y ago
>> I think it would be hard to teach a uniform view of decency Ofcourse, but we don't avoid studying moral philosophy because we can't find a uniform model. Nor does not being able to find a uniform model mean that there are not things worth learning from it. >> there are a lot of smart people who do much worse things than create malicious software when money is at stake Agreed. Two wrongs do not make a right, and as engineers in the know, we have a duty to stand up to such people when we have the chance. >> I fear it will fall short of your ideal. Me too, but the alternative is worse in my opinion.

amelius 9y ago

The problem is that if engineers in the US don't take the job then someone somewhere in, say, Asia will. Ethical rules are not the same everywhere. You can already tell this by looking at experimentation with stem-cells, cloning, et cetera, which is happening faster in Asia than in the US because of regulations. Also, refraining from building these systems means that other countries get ahead of us. Besides being an economic disadvantage, this could also threaten security. So I do agree that engineers should adhere to ethical guidelines, but they should be considered in a global context.

cmahler7 9y ago
this, we need to wake up to the fact that China is willing to do whatever it takes to dominate. They have done massive research on genetics and intelligence and the top schools in china make parents take IQ tests to determine potential for children when accepting students. They don't care about diversity or equality, only results. The US and Europe need to wake up or we'll be facing a billion hypernationalist, gentically modified super-geniuses while the only thing we have is the moral highground. Based on what I've seen over the last decade we should probably start learning Mandarin.
- joncampbelldev 9y ago
  I'm learning mandarin and have a chinese girlfriend, fully ready for the takeover when it happens ;)

bad_alloc 9y ago

Decency and foresight is highly subjective. Actually encoding what we understand as "decent human behaviour" into AI is a huge problem: If you create a system that decides about giving somebody a loan in the US based on a financial data set it might come up with any of these hypotheses: * Don't give loans to people living in [poor area]. * Avoid people with names that aren't similar to the most common ones in the database (i.e. foreign ones) * When linking the customer data to their social media and you see the picture is dissimilar to [preferred ethnicity], do not give a loan. Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1]. It knows that poor people pay back loans less frequently and makes the rational decision to not give a loan to that group. This of course reinforces the problem. But how are we to solve this? Introduce an additional column "race" to the data and bias the results with it? Would that not be just as racist? How do we give the system an awareness not to discriminate against ethnic groups, if the data contains implicit clues? This comes down to giving such an AI human intuition about such questions. [1] http://www.ssc.wisc.edu/irpweb/faqs/faq3/Figure1.png http://www.ssc.wisc.edu/irpweb/faqs/faq3/Figure1.png

bem94 9y ago
I'm not saying teach people some universal moral code, or suggesting we can accurately encode a set of appropriate morals into an AI. Just that people stop and think if what they are doing is appropriate. It seems silly to take problems which humans cannot solve with all the data, intuition and humanity that we have, and then use that data to train machines to make decisions, we know will be flawed.
bko 9y ago
>Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1]. If the data really suggests that these factors (poor areas, foreign names, social media) affect liklihood of default, should they be completely ignored? I'm sure credit scores and salaries are correlated to race in the US too. Should those be ignored? Just give out loans indiscriminately? They're not ignored now by loan approval. Worse, inaccurate stereotypes are used as a proxy for some of these factors that may be relevant to a borrowers ability to pay. We don't live in a perfect world where people ignore certain factors.

andbberger 9y ago

Sorry that's how natural selection works. When there's money at stake, someone will always do it. edit: typo

bem94 9y ago
And people like that are why we can't have nice things. I'm not saying that education / provoking thought will stop everyone doing something (else) monstrously stupid with AI. Just that we should be doing that educating / provoking more than we currently are.
ythn 9y ago
Even to the detriment of themselves... see "Tragedy of the Commons"

NickM 9y ago

I don't think that's the best defense, because even if 99.999% of programmers are perfect, ethical, decent people, all it takes is one psycho to go ahead and let the genie out of the bottle anyway. Imagine there's a big red "end the world" button that anyone on the planet can push. No matter how idealistic and utopian your views are, do you really trust that there's not a single person out of 7 billion people that might push the button? That is the kind of situation I worry we may face once people figure out how to create strong AI.

bem94 9y ago
Turning what I think you are saying on its head: There is a 0.001% chance of something bad happening, so there is no point in the other 99.999% changing their behaviour in order to stop that bad thing happening. I don't think that follows either. Second guessing human nature to the point where you resign yourself to catastrophe is surely less worthwhile than striving to create a sense of collective responsibility before it is too late?
- NickM 9y ago
  I'm not saying there's an 0.0001% chance of it happening, I'm saying that if you give enough people a chance to create an AI, eventually someone will do it, even if 99.999% of people would choose not to. Right now, nobody knows how to do this, but my point is that eventually people will figure out how: once everyone knows how, then it's just a matter of time before someone does it. I'm not second guessing human nature, just saying people do crazy things all the time. The vast, vast majority of people are not suicide bombers, for example, but it still happens sometimes. It doesn't matter how much collective responsibility we have; eventually you may just end up with a brilliant programmer with schizophrenia or something who writes an AI in his basement and unleashes it on the world. I'm also certainly not resigning myself to catastrophe. There are probably ways to prevent or mitigate the negative impact of AI, like the one being suggested in the original article, for instance. I would hardly consider myself a cynic, but forgive me if I'm skeptical of the idea that every human in the world can somehow agree to never do anything that might create a dangerous AI. If it can happen, it eventually will happen.

MarkMMullin 9y ago

By that logic the gamma knife that has saved many people with cancer would not exist - before practical applications comes laying the basic foundation, and once that foundation is down, the genie is out of the bottle - I'm working on an autonomous drone, I am of course deeply opposed to weaponizing what I know is a ridiculously stupid system, but I know that it will happen - implication is not action, and knowledge is mostly neutral, it's the application of that knowledge that might be problematic - whichever way you slice it, it's a really complex issue, and anything worth having is often capable of both great and terrible things, depending on the application. AI is way too primitive to be malicious on it's own and will remain that way for years - now, as for abhorrent malicious uses of what we have now, here's a great example -- https://goo.gl/HmBY9k https://goo.gl/HmBY9k