4 ms·

The thing about neural nets is that they are pretty opaque from an analyst point of view. It's hard to figure out why they do what they do, except that they ha

The thing about neural nets is that they are pretty opaque from an analyst point of view. It's hard to figure out why they do what they do, except that they have been trained to optimize a particular cost function. I think Strong AI will never happen because the people in charge will not give control over to a system that makes important decisions without explaining why. They will certainly not give control over the cost function to a strong AI because control of determination of the cost function is the axis upon which all power will rest.

relate 11y ago
This is a common criticism. However, almost all ML methods have some built in heuristic choices, that are the result of finding something that both works and is mathematically nice. Each of these choices restricts us to some family of functions where it's hard to justify why it's really relevant to the problem at hand, e.g. convex loss functions (l1, l2, ..), convex regularizers (l1,l2,..), gaussian priors, linear classifiers, some mathematically nice kernel functions, e.t.c. In the end, people usually statistically estimate the performance of the methods and use what works.
__Joker 11y ago
I kind of drifted into the camp of transhumanism as future where human is enhanced by all the smart sub AI problem solver but generally the humans take the decision at the end of the day. Also I think other problem is for strong AI to exist we are not sure what the "objective function" for the AI to work for.
pjc50 11y ago
the people in charge will not give control over to a system that makes important decisions without explaining why They will if it gives the answers they want to hear. History is full of critical decisions based on ridiculous pretexts or unclear processes.
deleted 11y ago
[deleted]
dimatura 11y ago
Our life is dominated by systems we don't understand. I have some understanding of how my cell phone works at the software level, but when it comes to details at the hardware level I just trust the electrical engineers knew what they're doing. I have virtually no understanding of how the engine in the bus operates beyond what I learned in thermodynamics 101. Sure, you might say - someone understands these things. But for some systems, it's hard to pinpoint these people. And for some other complex systems, like the stock market, nobody really understands them or (completely) controls them. But we still use them every day. I think once AI becomes useful enough, people will gladly hand control over.
- imaginenore 11y ago
  But some engineer out there understands how your phone works. With neural nets NOBODY really understands how they work.
  - Nadya 11y ago
    Maybe my understanding of neural networks is wrong... but I'm under the impression they work from weighted criteria. With enough weight an answer is selected as being the most likely. A well-trained neural network has enough data to weight options and pick with high accuracy. Then again, this is essentially black magic to me: http://hplusmagazine.com/2015/02/26/ai-masters-classic-video-games-without-being-told-the-rules/ http://hplusmagazine.com/2015/02/26/ai-masters-classic-video...
  - stefs 11y ago
    and, well, that's just a faulty assumption. of course we do know how they work.
  - Quiark 11y ago
    A trained neural network is like a horrible huge spaghetti code ball you've inherited after a programmer ran over by a bus that for some miraculous reason happens to be working mostly correctly. However, you won't be able to understand why or how it works. That also means you won't be able to modify/improve/fix it using systematic methods. Only trial and error and it will be 'error' most of the time.
btown 11y ago
> the people in charge will not give control Eliezer Yudkowsky would likely disagree with you: http://www.yudkowsky.net/singularity/aibox http://www.yudkowsky.net/singularity/aibox EDIT: Also - http://www.explainxkcd.com/wiki/index.php/1450:_AI-Box_Experiment http://www.explainxkcd.com/wiki/index.php/1450:_AI-Box_Exper...
scarmig 11y ago
It may be the case, though, that companies that relinquish control to neural nets will have better results than companies that don't. In fact, there's a winner-take-all effect in many markets, so in those even a slight improvement over humans would lead to massive benefits, rapidly pushing human analysts out of the market. That's the (morally neutral) wonder of the market--it'll beat ideological or emotional objections into the ground, for better or for worse. And sooner or later, someone might start a company where all decision making is performed by a neural net...
chronolitus 11y ago
What if an AI saves money, though? ( i.e. is cheaper than hiring a real person for a simple task ) " Never! Companies would never sacrifice principle and safety to save money! " We'll see...
m-i-l 11y ago
I remember wanting to train a neural net for my MSc thesis more than 20 years ago, but my tutor recommended against doing so for precisely this reason, i.e. he said it is very difficult to prove your results. While not being able to prove your results might be a bad idea if you're trying to get your MSc, I don't see it holding back other advances.

relate 11y ago

This is a common criticism. However, almost all ML methods have some built in heuristic choices, that are the result of finding something that both works and is mathematically nice. Each of these choices restricts us to some family of functions where it's hard to justify why it's really relevant to the problem at hand, e.g. convex loss functions (l1, l2, ..), convex regularizers (l1,l2,..), gaussian priors, linear classifiers, some mathematically nice kernel functions, e.t.c. In the end, people usually statistically estimate the performance of the methods and use what works.

__Joker 11y ago

I kind of drifted into the camp of transhumanism as future where human is enhanced by all the smart sub AI problem solver but generally the humans take the decision at the end of the day. Also I think other problem is for strong AI to exist we are not sure what the "objective function" for the AI to work for.

pjc50 11y ago

the people in charge will not give control over to a system that makes important decisions without explaining why They will if it gives the answers they want to hear. History is full of critical decisions based on ridiculous pretexts or unclear processes.

deleted 11y ago

[deleted]

dimatura 11y ago

Our life is dominated by systems we don't understand. I have some understanding of how my cell phone works at the software level, but when it comes to details at the hardware level I just trust the electrical engineers knew what they're doing. I have virtually no understanding of how the engine in the bus operates beyond what I learned in thermodynamics 101. Sure, you might say - someone understands these things. But for some systems, it's hard to pinpoint these people. And for some other complex systems, like the stock market, nobody really understands them or (completely) controls them. But we still use them every day. I think once AI becomes useful enough, people will gladly hand control over.

imaginenore 11y ago
But some engineer out there understands how your phone works. With neural nets NOBODY really understands how they work.
- Nadya 11y ago
  Maybe my understanding of neural networks is wrong... but I'm under the impression they work from weighted criteria. With enough weight an answer is selected as being the most likely. A well-trained neural network has enough data to weight options and pick with high accuracy. Then again, this is essentially black magic to me: http://hplusmagazine.com/2015/02/26/ai-masters-classic-video-games-without-being-told-the-rules/ http://hplusmagazine.com/2015/02/26/ai-masters-classic-video...
- stefs 11y ago
  and, well, that's just a faulty assumption. of course we do know how they work.
- Quiark 11y ago
  A trained neural network is like a horrible huge spaghetti code ball you've inherited after a programmer ran over by a bus that for some miraculous reason happens to be working mostly correctly. However, you won't be able to understand why or how it works. That also means you won't be able to modify/improve/fix it using systematic methods. Only trial and error and it will be 'error' most of the time.

btown 11y ago

> the people in charge will not give control Eliezer Yudkowsky would likely disagree with you: http://www.yudkowsky.net/singularity/aibox http://www.yudkowsky.net/singularity/aibox EDIT: Also - http://www.explainxkcd.com/wiki/index.php/1450:_AI-Box_Experiment http://www.explainxkcd.com/wiki/index.php/1450:_AI-Box_Exper...

scarmig 11y ago

It may be the case, though, that companies that relinquish control to neural nets will have better results than companies that don't. In fact, there's a winner-take-all effect in many markets, so in those even a slight improvement over humans would lead to massive benefits, rapidly pushing human analysts out of the market. That's the (morally neutral) wonder of the market--it'll beat ideological or emotional objections into the ground, for better or for worse. And sooner or later, someone might start a company where all decision making is performed by a neural net...

chronolitus 11y ago

What if an AI saves money, though? ( i.e. is cheaper than hiring a real person for a simple task ) " Never! Companies would never sacrifice principle and safety to save money! " We'll see...

m-i-l 11y ago

I remember wanting to train a neural net for my MSc thesis more than 20 years ago, but my tutor recommended against doing so for precisely this reason, i.e. he said it is very difficult to prove your results. While not being able to prove your results might be a bad idea if you're trying to get your MSc, I don't see it holding back other advances.