7 ms·

Whenever I read prominent figures talking about the dangers of AI, I feel like they are missing the mark. I don't think the imminent dangers of AI are self-con

Whenever I read prominent figures talking about the dangers of AI, I feel like they are missing the mark. I don't think the imminent dangers of AI are self-conscious machines rebelling and deciding to kill people, but a much more subtle and nuance problem. Let's say that you go to a bank and ask for a loan. The bank has a procedure to decide if they give you the loan or not. If your request is rejected, they can (hopefully) explain why you didn't qualify. Instead, the whole process gets offloaded to a learning AI that makes that decision for the bank. For the most part, we don't really understand why systems like neural networks make a decision. Now if the banks denies your loan, the best they can say why is 'huh, the computer said so'. What if suspiciously the banks start rejecting applications from an specific minority? Has the AI determined that this particular minority is trustworthy? Does it even matter? What when AI systems are introduced in legal procedures? What when an AI decides if someone will go to jail or not? Maybe I'm out of the loop but I feel these are more pressing issues about the widespread use of AI that are not being talked about enough (definitely much less than the doomsday scenario).

white-flame 9y ago
This is already happening with generating income on YouTube. Videos are eventually evaluated by an AI as to whether or not they're advertiser-friendly, and get demonetized if the AI deems it unsuitable, with no explanation or feedback as to how to bring it into compliance. The AI's training likely changes over time, so there's no guarantee you'll be staying in its good graces. Many folks' business & livelihood is now dependent on the whim of a statistical AI.
- speedplane 9y ago
  That may be new to YouTube, but it's been a part of Google search for years. Whether you make the top 3 hits of a particular keyword can make or break many businesses; also based on a statistical AI. Google is not bad at generally pulling 3 decent results, but at least a few searches a day it returns garbage results.
larksimian 9y ago
It's been looked into quite a bit in relation to parole hearings in the US which use computer generated scores to advise the parole committees. https://www.propublica.org/article/bias-in-criminal-risk-scores-is-mathematically-inevitable-researchers-say https://www.propublica.org/article/bias-in-criminal-risk-sco... https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm https://www.propublica.org/article/how-we-analyzed-the-compa... https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing/ https://www.propublica.org/article/machine-bias-risk-assessm...
haltingthoughts 9y ago
I'm more concerned with the opposite problem of how we will behave in response to AI control. AI has no notion of the Lucas critique. It doesn't consider the perverse incentives it's optimized judgement scheme may create. The costs of looking good to proprietary algorithms are only going to grow.
denzil_correa 9y ago
> For the most part, we don't really understand why systems like neural networks make a decision. ... > Maybe I'm out of the loop but I feel these are more pressing issues about the widespread use of AI that are not being talked about enough (definitely much less than the doomsday scenario). This is essentially about Fairness, Accountability and Transparency with algorithms. There has been a lot of discussion and research in this space. http://www.fatml.org http://www.fatml.org
Tenoke 9y ago
>Whenever I read prominent figures talking about the dangers of AI, I feel like they are missing the mark. I don't think the imminent dangers of AI are self-conscious machines rebelling and deciding to kill people, but a much more subtle and nuance problem. People are plenty aware of this. It is just that they are not always talking about the most imminent dangers. Sort of like how people talk about global warming dangers down the road, even though they are aware of e.g. cancer today.
Simon_says 9y ago
Discrimination is bad, but even the worst discrimination will leave millions of human survivors. People talking about the existential dangers of AI are talking about disasters that leave no human survivors.
yters 9y ago
This is not AI. This is our general statistical regression and overall reductive quantitative approach to decision making, of which AI is a subfield. Computation lets us scale up the stupidity.
austinjp 9y ago
> Let's say that you go to a bank and ask for a loan. The bank has a procedure to decide if they give you the loan or not. If your request is rejected, they can (hopefully) explain why you didn't qualify. Instead, the whole process gets offloaded to a learning AI that makes that decision for the bank. For the most part, we don't really understand why systems like neural networks make a decision. Now if the banks denies your loan, the best they can say why is 'huh, the computer said so'. This is similar to the current situation in the UK for various financial and insurance applications. The application is farmed out to an organisation that ensures compliance with whatever regulations apply, but often simply gives a yes or no answer, without details. Or at least, the details are not passed back to the customer. > What if suspiciously the banks start rejecting applications from an specific minority? Has the AI determined that this particular minority is trustworthy? Does it even matter? This is a very real concern. Specific subsets of society tend to build and train AIs and may consider their outputs accurate or as expected, whereas other subsets of society may disagree but may be underrepresented among AI engineers. There is a danger that an AIs ability to distinguish subsets of society will be paired with value judgements. It doesn't have to be this way, we need to involve all members of society in discussions that are currently somewhat restricted.
- tim333 9y ago
  > What if suspiciously the banks start rejecting applications from an specific minority? These issues are not that specific to AI. You can have a look a what applications are turned down whether humans, algorithms or AIs are running things. And then figure if anything should be done about it.
Houshalter 9y ago
Good! Bring this future! Humans should never ever ever be in charge of such things. On most domains where relevant statistical data is available, like your examples, even crude algorithms like linear regression will almost always beat human experts. Humans are biased as hell. Studies show judges give much harsher sentences just before lunch, when they are hungry. Attractive defendants get half the sentences as unattractive ones. Not to mention all the classic race and gender ones, and random factors we can't even measure. It is inexcusable to have humans in charge of any kind of decision process that a superior unbiased algorithm can do.
- gambiting 9y ago
  I think you completely misunderstood the parents concern, or chose to ignore it. The question is - what happens if an algorithm develops a bias? A bias against a specific race/sex/occupation, whatever - what happens to the principle of free an equal society then? Super simple example - you don't need AI to determine that men claim more frequently and more on car insurance than women. So....over the years, the rates for men have risen, until finally it was completely outlawed to take into account the sex of the applicant for car insurance quotes(at least here in the EU). We need the human touch to iron out problems like this. An all knowing-omnipotent algorithm if going to discriminate based on statistics and that's something I don't think society wants.
  - rwallace 9y ago
    > Super simple example - you don't need AI to determine that men claim more frequently and more on car insurance than women. So....over the years, the rates for men have risen, until finally it was completely outlawed to take into account the sex of the applicant for car insurance quotes(at least here in the EU). I believe that law should be repealed, because it would lead to fewer people dying on the roads. So there is a case where not everyone agrees the human touch is actually a good thing.
    - danielbarla 9y ago
      I think it really depends on what parameter(s) the AI is optimising for, or weighting the most. If it's efficiency or profit, there are tons of things that make sense financially, but would probably be considered unethical. E.g. we could encourage people to smoke, as it tends to lower the cost of healthcare overall (since they die younger / quicker, on average). That's just an example, the point is that I'm sure there are tons of "truths" from some perspective that an AI might employ, but we'd find unethical.
speedplane 9y ago
I think the dangers are even more real present than you allude. The idea that social networks and search engines are deciding what media we consume is already a form of AI, and it's already causing real harm. Case-in-point: just yesterday an innocent man was falsely labeled a mass murderer on 4chan and those threads were surfaced on the top of google search results and on facebook.
akvadrako 9y ago
None of that is really an existential threat, i.e. capable of wiping out the human race. Those prominent figures are focusing on the risk of losing our status as the dominant species. Even if that takes 10000 years, it's significantly more serious than being put in jail or failing to get a bank loan.
pdkl95 9y ago
> we don't really understand why systems like neural networks make a decision Maciej Cegłowski gave a very good talk about that specific problem, which incudes a few recommendations about how we can avoid some of the worst problems as machine learning and capitalist democracy merge. "Build a Better Monster: Morality, Machine Learning and Mass Surveillance" https://www.infoq.com/presentations/morality-ml-mass-surveillance https://www.infoq.com/presentations/morality-ml-mass-surveil...
red75prime 9y ago
> I don't think the imminent dangers of AI are self-conscious machines rebelling and deciding to kill people Those descriptions are too anthropomorphic. Why not "optimization algorithm finds and implements a solution for a problem of growing more tomatoes, which is incompatible with continued existence of humanity".
mbaha 9y ago
What you're describing has been basically begun from the first time a spreadsheet was opened by a decision maker in a bank. From then, the calculations only got more advanced, but computers have always hidden how they worked (especially when they reach a certain scale of customers).
chriswarbo 9y ago
Whilst that's an issue with current popular approaches (e.g. neural networks), I don't think it's too catastrophic. One approach to dealing with this is to use algorithms which are more amenable to explanations, like decision trees or inductive logic programming: that way, we can see exactly what criteria are being used to reach a decision, and we can use existing regulations/audits/etc. to keep them in check. Another approach is to have the AI system produce evidence rather than an opaque response. One area where this already exists is in automated theorem proving: given a (suitably formalised) mathematical statement, programs like E and Vampire can (try to) determine whether it's true or false. In theory, we could ask such programs whether e.g. a loan application satisfies our eligiblity criteria, but the resulting true/false answer would be about as opaque as using deep learning. However, we don't have to be content with a simple boolean: programs like metis can output proofs of a statement (if it's true), whilst programs like MACE can output specific counterexamples (if it's false). It doesn't seem too far fetched to have an AI system generate justifications for and/or against an application.
uoaei 9y ago
Yes people are still anthropomorphizing AI. It's a subtle difference talking about their learned judgments of reality v. some sort of human-like logical decision-making process, and that's where most people get tripped up. The AI learns what you feed it. Garbage in, garbage out. The initialization of its world model is arguably the most important thing, and the hardest to make objective. Every decision following that, assuming it learns in an online manner, will be conditioned on its previous decisions since it can only learn about the things it experiences, i.e., chooses to engage with and how.

white-flame 9y ago

This is already happening with generating income on YouTube. Videos are eventually evaluated by an AI as to whether or not they're advertiser-friendly, and get demonetized if the AI deems it unsuitable, with no explanation or feedback as to how to bring it into compliance. The AI's training likely changes over time, so there's no guarantee you'll be staying in its good graces. Many folks' business & livelihood is now dependent on the whim of a statistical AI.

speedplane 9y ago
That may be new to YouTube, but it's been a part of Google search for years. Whether you make the top 3 hits of a particular keyword can make or break many businesses; also based on a statistical AI. Google is not bad at generally pulling 3 decent results, but at least a few searches a day it returns garbage results.

larksimian 9y ago

It's been looked into quite a bit in relation to parole hearings in the US which use computer generated scores to advise the parole committees. https://www.propublica.org/article/bias-in-criminal-risk-scores-is-mathematically-inevitable-researchers-say https://www.propublica.org/article/bias-in-criminal-risk-sco... https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm https://www.propublica.org/article/how-we-analyzed-the-compa... https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing/ https://www.propublica.org/article/machine-bias-risk-assessm...

haltingthoughts 9y ago

I'm more concerned with the opposite problem of how we will behave in response to AI control. AI has no notion of the Lucas critique. It doesn't consider the perverse incentives it's optimized judgement scheme may create. The costs of looking good to proprietary algorithms are only going to grow.

denzil_correa 9y ago

> For the most part, we don't really understand why systems like neural networks make a decision. ... > Maybe I'm out of the loop but I feel these are more pressing issues about the widespread use of AI that are not being talked about enough (definitely much less than the doomsday scenario). This is essentially about Fairness, Accountability and Transparency with algorithms. There has been a lot of discussion and research in this space. http://www.fatml.org http://www.fatml.org

Tenoke 9y ago

>Whenever I read prominent figures talking about the dangers of AI, I feel like they are missing the mark. I don't think the imminent dangers of AI are self-conscious machines rebelling and deciding to kill people, but a much more subtle and nuance problem. People are plenty aware of this. It is just that they are not always talking about the most imminent dangers. Sort of like how people talk about global warming dangers down the road, even though they are aware of e.g. cancer today.

Simon_says 9y ago

Discrimination is bad, but even the worst discrimination will leave millions of human survivors. People talking about the existential dangers of AI are talking about disasters that leave no human survivors.

yters 9y ago

This is not AI. This is our general statistical regression and overall reductive quantitative approach to decision making, of which AI is a subfield. Computation lets us scale up the stupidity.

austinjp 9y ago

> Let's say that you go to a bank and ask for a loan. The bank has a procedure to decide if they give you the loan or not. If your request is rejected, they can (hopefully) explain why you didn't qualify. Instead, the whole process gets offloaded to a learning AI that makes that decision for the bank. For the most part, we don't really understand why systems like neural networks make a decision. Now if the banks denies your loan, the best they can say why is 'huh, the computer said so'. This is similar to the current situation in the UK for various financial and insurance applications. The application is farmed out to an organisation that ensures compliance with whatever regulations apply, but often simply gives a yes or no answer, without details. Or at least, the details are not passed back to the customer. > What if suspiciously the banks start rejecting applications from an specific minority? Has the AI determined that this particular minority is trustworthy? Does it even matter? This is a very real concern. Specific subsets of society tend to build and train AIs and may consider their outputs accurate or as expected, whereas other subsets of society may disagree but may be underrepresented among AI engineers. There is a danger that an AIs ability to distinguish subsets of society will be paired with value judgements. It doesn't have to be this way, we need to involve all members of society in discussions that are currently somewhat restricted.

tim333 9y ago
> What if suspiciously the banks start rejecting applications from an specific minority? These issues are not that specific to AI. You can have a look a what applications are turned down whether humans, algorithms or AIs are running things. And then figure if anything should be done about it.

Houshalter 9y ago

Good! Bring this future! Humans should never ever ever be in charge of such things. On most domains where relevant statistical data is available, like your examples, even crude algorithms like linear regression will almost always beat human experts. Humans are biased as hell. Studies show judges give much harsher sentences just before lunch, when they are hungry. Attractive defendants get half the sentences as unattractive ones. Not to mention all the classic race and gender ones, and random factors we can't even measure. It is inexcusable to have humans in charge of any kind of decision process that a superior unbiased algorithm can do.

gambiting 9y ago
I think you completely misunderstood the parents concern, or chose to ignore it. The question is - what happens if an algorithm develops a bias? A bias against a specific race/sex/occupation, whatever - what happens to the principle of free an equal society then? Super simple example - you don't need AI to determine that men claim more frequently and more on car insurance than women. So....over the years, the rates for men have risen, until finally it was completely outlawed to take into account the sex of the applicant for car insurance quotes(at least here in the EU). We need the human touch to iron out problems like this. An all knowing-omnipotent algorithm if going to discriminate based on statistics and that's something I don't think society wants.
- rwallace 9y ago
  > Super simple example - you don't need AI to determine that men claim more frequently and more on car insurance than women. So....over the years, the rates for men have risen, until finally it was completely outlawed to take into account the sex of the applicant for car insurance quotes(at least here in the EU). I believe that law should be repealed, because it would lead to fewer people dying on the roads. So there is a case where not everyone agrees the human touch is actually a good thing.
  - danielbarla 9y ago
    I think it really depends on what parameter(s) the AI is optimising for, or weighting the most. If it's efficiency or profit, there are tons of things that make sense financially, but would probably be considered unethical. E.g. we could encourage people to smoke, as it tends to lower the cost of healthcare overall (since they die younger / quicker, on average). That's just an example, the point is that I'm sure there are tons of "truths" from some perspective that an AI might employ, but we'd find unethical.

speedplane 9y ago

I think the dangers are even more real present than you allude. The idea that social networks and search engines are deciding what media we consume is already a form of AI, and it's already causing real harm. Case-in-point: just yesterday an innocent man was falsely labeled a mass murderer on 4chan and those threads were surfaced on the top of google search results and on facebook.

akvadrako 9y ago

None of that is really an existential threat, i.e. capable of wiping out the human race. Those prominent figures are focusing on the risk of losing our status as the dominant species. Even if that takes 10000 years, it's significantly more serious than being put in jail or failing to get a bank loan.

pdkl95 9y ago

> we don't really understand why systems like neural networks make a decision Maciej Cegłowski gave a very good talk about that specific problem, which incudes a few recommendations about how we can avoid some of the worst problems as machine learning and capitalist democracy merge. "Build a Better Monster: Morality, Machine Learning and Mass Surveillance" https://www.infoq.com/presentations/morality-ml-mass-surveillance https://www.infoq.com/presentations/morality-ml-mass-surveil...

red75prime 9y ago

> I don't think the imminent dangers of AI are self-conscious machines rebelling and deciding to kill people Those descriptions are too anthropomorphic. Why not "optimization algorithm finds and implements a solution for a problem of growing more tomatoes, which is incompatible with continued existence of humanity".

mbaha 9y ago

What you're describing has been basically begun from the first time a spreadsheet was opened by a decision maker in a bank. From then, the calculations only got more advanced, but computers have always hidden how they worked (especially when they reach a certain scale of customers).

chriswarbo 9y ago

Whilst that's an issue with current popular approaches (e.g. neural networks), I don't think it's too catastrophic. One approach to dealing with this is to use algorithms which are more amenable to explanations, like decision trees or inductive logic programming: that way, we can see exactly what criteria are being used to reach a decision, and we can use existing regulations/audits/etc. to keep them in check. Another approach is to have the AI system produce evidence rather than an opaque response. One area where this already exists is in automated theorem proving: given a (suitably formalised) mathematical statement, programs like E and Vampire can (try to) determine whether it's true or false. In theory, we could ask such programs whether e.g. a loan application satisfies our eligiblity criteria, but the resulting true/false answer would be about as opaque as using deep learning. However, we don't have to be content with a simple boolean: programs like metis can output proofs of a statement (if it's true), whilst programs like MACE can output specific counterexamples (if it's false). It doesn't seem too far fetched to have an AI system generate justifications for and/or against an application.

uoaei 9y ago

Yes people are still anthropomorphizing AI. It's a subtle difference talking about their learned judgments of reality v. some sort of human-like logical decision-making process, and that's where most people get tripped up. The AI learns what you feed it. Garbage in, garbage out. The initialization of its world model is arguably the most important thing, and the hardest to make objective. Every decision following that, assuming it learns in an online manner, will be conditioned on its previous decisions since it can only learn about the things it experiences, i.e., chooses to engage with and how.