23 ms·

Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as

by ktta 10y ago

Also, an interesting read related to this would 'the AI effect'[0]. A lot of the stuff Deep Learning/Machine Learning is able to do today would be looked at as something that only 'true' AI (whatever consensus on what that means is; I think of it as AGI) would be able to do. But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. But just a decade ago, if you saw that a computer was able to recognize pictures better than humans, you would think there was something fishy going on, or we have achieved true AI. My opinion is true AI is which is comparable to human intelligence, in that it is sentient and/or capable of abstract thought, not necessarily being able to hold a conversation or solve concrete mathematics problems, or dump out a story by neural networks. [0]https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect

argonaut 10y ago
I hear this repeated a lot. But I've never actually seen a source from, say, the 90's or the 2000s, actually saying something like "if we're able to classify images better than humans, that's true AI." I suspect it's just hearsay that's been repeated over and over again.
- svantana 10y ago
  There used to be a concept of "AI completeness" which meant that some problems could only be solved by True AI, which would also be able to solve all other human-level problems. Doug Hofstadter writes in Gödel-Escher-Bach that he believes grandmaster-level chess is AI complete. That was obviously false in retrospect, and it's becoming increasingly obvious that there is no AI completeness at all.
  - ragebol 10y ago
    For now, I take this blogpost [1] of Andrej Karpathy as a nice measure for AI hardness. [1] http://karpathy.github.io/2012/10/22/state-of-computer-vision/ http://karpathy.github.io/2012/10/22/state-of-computer-visio...
    - csydas 10y ago
      This is a good blog post and articulates well many of the issues I have with calling current work on AI subsets "AI". From my point of view, the results of current work with AI subsets are impressive, but they're hitting self-proclaimed benchmarks or someone poorly defined benchmarks and declaring AI status. Creating an advanced Go machine was very cool, but as Lee Sedol played more, he said he began to understand how AlphaGo "thought". The appended reddit thread here [1] has some interesting insight on why pulling back the curtain on current AI claims is appropriate. A lot of the times, "it's just calculations" is appropriate since we don't really have a machine considering what it's given, it's driving towards a singular goal in the best way possible. In the case of AlphaGo, it disregarded Lee Sedol's move because it had no care about the unusualness of it, it just wrote the move off as sub-optimal. When you see behavior like this where the systems are algorithmically defeated, it's very difficult to not be dismissive of the claims of AI. Again, there is a lot of coolness happening with subsets of AI research, but I don't feel that there is even a clear definition of what AI would entail - spaghetti code to get a desired result doesn't really help either, since it has to be persistent independent successes - to refer to Karpathy's article, it would have to make repeated assertions and understandings of similar photos with a high success rate to really be something spectacular. [1] https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_insight_into_alpha_gos_from_match_4/ https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_i...
  - eli_gottlieb 10y ago
    >it's becoming increasingly obvious that there is no AI completeness at all. Bingo! The "general" part is the ability to learn new structures and task structures from environmental cues, and then construct informed prior beliefs about those new tasks and structures using causal relations to previously-observed tasks and structures. Nothing more!
- reacweb 10y ago
  In the 90's, John McCarthy was calling Go "the new Drosophila of AI". Now, we have AlphaGo that is "just" a bunch of "generic" deep learning with MC. Maybe John McCarthy was right and the ideas behind AlphaGo are the first steps toward true AI. I am very impressed.
  - visarga 10y ago
    The guys at NVIDIA did something very cool recently. They took a simple CNN (convolutional neural net) and in 2-3 days they taught it to drive a car. Article: How Our Deep Learning Tech Taught a Car to Drive - https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3/ https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3... These neural nets are really smart, and we don't know exactly how they work, we know only in principle. But if we asked the guy who made the self driving car what is the role of the 17th neuron in the 14th layer, he would probably have no idea. Just like human brains evolve through learning, so do neural nets. Yet many people think they are just clever tricks and not truly intelligent.
    - c0g 10y ago
      That's because they're not intelligent. They're layered function approximators with some specialisations for vision - conv layers. Their structure is designed by humans as is the data they are fed with. Maybe the human brain is the same, IMO it isn't. Humans seek out new experiences and can introspect and at their best assess their level of knowledge of a sibject. DNNs cannot.
    - argonaut 10y ago
      Did you read the paper? They did not teach it to drive a car. The car just does lane keeping - it doesn't even do turns or lane changes. This is stuff solved 10+ years ago. And even then it only achieves 98% autonomy on lane keeping - this is a task that needs 100% accuracy. You should not be running into the median every couple miles.
  - eli_gottlieb 10y ago
    Sounds like McCarthy was right: the point of Drosophila is that it's a very small, simple model organism. You're not supposed to "solve biology" by modelling something in Drosophila, you're just supposed to be able to narrow the field to hypotheses that are not so flagrantly stupid they fail in a fruit-fly.
- ktta 10y ago
  hey argonaut, we meet again. I think you'd be interested in the parent comments of this[0] thread. [0]https://news.ycombinator.com/item?id=12322536 https://news.ycombinator.com/item?id=12322536
seanwilson 10y ago
> But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence. For example, while AlphaGo is incredibly impressive, it can only play Go and nothing else. I think once we have a comprehensive AI program that can play many kinds of games, can have conversations, can complete school exam questions, can write stories etc. the question of what intelligence is will become more interesting.
- visarga 10y ago
  > once we have a comprehensive AI program that can play many kinds of games Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm. https://arxiv.org/pdf/1312.5602.pdf https://arxiv.org/pdf/1312.5602.pdf Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpose to maximize a reward. It can be used, and was used, in multiple domains. AlphaGo used RL as well. Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game. In fact, if the software was set to learn more games, it could learn them in addition to Go. Also, regarding other tasks: a neural net that recognizes cats can be easily made to recognize dogs too. A program that translates English to French can be made to translate other languages too. We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited. Recently there has been a paper "Learning without forgetting" (http://arxiv.org/abs/1606.09282v2 http://arxiv.org/abs/1606.09282v2) that underlines this very ability to span multiple domains and adapt easily to unseen tasks and data. > Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence. Saying that people can do many tasks is not exactly right because a particular person can only do a few tasks, those tasks she was trained to do. If I never learned German, I don't know German. That doesn't mean the brain itself is limited. AlphaGo was only trained on Go, and its internal architecture was optimized for this one task in order to make it more efficient, but the method is general and reusable. DeepMind said so themselves, the breakthrough is not that they beat Lee Sedol, but that they used a general method that can be used to do other tasks as well. It is not a limitation of AI that we generally make systems that are good at only one thing. If there is a limitation in AlphaGo, it is that it mastered a game where the whole situation is perfectly known (the Go board), while in reality many tasks are only partially known (such as card games, for example) so there is an extra uncertainty. But DeepMind and other researchers are working on that too.
Aissen 10y ago
If it's magic, it's AI; if it's understood, it's algorithms.
- jules 10y ago
  A chess computer isn't magic because you know that it is basically searching the game tree very fast with some heuristics. Neural networks are not like that. You know that it's a network of activation functions, but how that solves so many AI tasks is still more or less a mystery. So maybe the AI effect will be less strong with deep learning.
  - mjn 10y ago
    I disagree that neural networks are necessarily mysterious. In most cases they can be straightforwardly understood as brute-force function approximation (approximation here meaning mostly interpolation, in some cases capable of a limited degree of extrapolation). You take many samples of a function (either because you have a huge data set, or because you can actually sample with a simulator), and incrementally modify a large network to slowly converge it towards an accurate approximation of the function. The advent of very large data sets and GPU training made this possible, but the tech is pretty well understood, at least at the level of a 5-mile-high view.
    - jules 10y ago
      Of course, but compared to how chess algorithms work it remains far more mysterious how exactly that produces an english description of a grid of pixel values.
      - throwaway729 10y ago
        Probably only because every CS undergrad or self-trained person sees at least a few search algorithms. So although they probably couldn't write a performant chess-playing program, they get the general idea. I expect in 20 years undergraduate curricula evolve to cover NNs, and then those courses will have their salient bits adopted into mass market expository books/blog posts/code examples/etc. At that point someone like you will be saying something like you said about whatever the new hotness is.
sguav 10y ago
What if we trained a CNN to "know [it] knows nothing"? https://en.wikipedia.org/wiki/I_know_that_I_know_nothing https://en.wikipedia.org/wiki/I_know_that_I_know_nothing
- ktta 10y ago
  You might have to see someone who took philosophy and a Machine Learning researcher about that :)
cgearhart 10y ago
I would argue that there is actually an unrecognized inverse of this effect, too. People have a tendency to conflate domain-specific human-level performance with "intelligence" (for some definition of that word). Then suddenly we jump to "DeepMind won at go, therefore humanity is on the brink of creating Skynet." Deep learning has been like this so far, because it works in many new domains that have resisted previous techniques like image recognition, speech recognition, go, etc. The "AI Effect" -- saying, "it's just an algorithm" once we succeed -- is an artifact of developing AI by using it as an algorithm for a domain-specific task. It is "just" an algorithm, because that's all it needs to be to win at go, identify cats, or whatever. Fundamentally, it's not very surprising that when you set out to make a system that's really good at playing go...you end up with a system that's really good at playing go. Of course it's hard -- that's what we should appreciate -- but most of the problems we consider "hard" are based on what we think it is hard for a computer to do in order to solve the problem. "Searching the space of all possible go moves is computationally intractable, therefore this problem is hard." In reality, the problem may not be as hard as we think if the agent doesn't have to search the space of all possible moves. "Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function...
- eli_gottlieb 10y ago
  >"Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function... Minimize the informational free-energy of the product of the multiplicatively inverted brain-to-body reinforcement-learned energy function of reward with the brain-to-body reinforcement-learned energy function of punishment, via active inference? Beh, those words are messy. The point is to talk about a Gibbs distribution with energy functions Reward(X, Y) and Punishment(X, Y), such that the "total" energy function is E(X, Y) \propto Punishment(X, Y) - Reward(X, Y). This then gives us a "goal distribution" for an active inference agent (like a human), defined "up to" the reinforcement-learned energy functions, whose limiting functions (need a lot of functional analysis and probabilistic reasoning over function spaces, there) are the "ground truth" causal relations by which the world causes reward signals through the body and its senses. https://arxiv.org/pdf/1503.04187.pdf https://arxiv.org/pdf/1503.04187.pdf
eli_gottlieb 10y ago
>But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts, ex nihilo, rather than an efficient engine for distilling structure from experiences, and to be ontologically special to prove how special we humans are on a universal scale. There are cognitive science and neuroscience labs hard at work on deciphering how actually-existing human thought works, but as far as I can tell, everyone facinated by "AI" is so enamoured with intelligence being special that they don't pay attention to those labs. A great deal of books and papers are thus published, and as far as "AI" is concerned, it's all in vain somehow.
- ktta 10y ago
  >Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts I think we might have to step into borderline philosophical subjects like epistemology and sentience depending on who you talk to to define intelligence and whether you're the free will group or determinism group and so on. I think you might agree with the statement that we can't yet come to a strict definition of what intelligence is, and once we have it, if we are going for intelligence or something more akin to stuff independent of intelligence like sentience and creative thought. I'm personally in the determinist group and agree that there is nothing inherently magical about intelligence, just us being simpletons who cannot comprehend how a microprocessor works when you just give the latest intel chip. It is so complex that we need a lot of time and energy to comprehend it, but like the intel chip, I believe there is a design aspect to the human brain rather than it being a magical black box. The problem is, the majority of us are the simpletons and have a long way to go before we have the knowledge that the intel chip makers have. I apologize for the bad intel chip analogy, I probably could've come up with something better :)
j1vms 10y ago
And on a related note, "better than humans", "comparable to human intelligence", and phrases like these since the early days of AI have always been a red herring. Not the least because these phrases do not imply what we are really comparing - better than all humans? some humans? How about just a person? What about just better than a 4-year-old? The criteria should have always been "faster / more efficiently", "more accurately", or "both" in comparing AI to human intelligence for a given task or activity. By this measure, AI had enduring production-ready successes since at least the 1980s. Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count. True AI is starting to look philosophically like a mirage, not the least because we may not ever have a strict meaningful definition for intelligence. What we consider intelligence varies, evolves, changes shape, consistency and predictability even if we consider just one of the various people we interact with on a given day. As one of the latest, I think Brian Eno (https://news.ycombinator.com/item?id=12027055 https://news.ycombinator.com/item?id=12027055), like others before him who have formalized a similar approach to the matter of intelligence, is on the right track. This approach to deciding what constitutes True AI would do much to counter and eventually prevent the "AI effect".
- ktta 10y ago
  >And on a related note, "better than humans", "comparable to human intelligence", and phrases like these since the early days of AI have always been a red herring Well, I agree that it is not definitive, but it definitely gives us some insight on where AI techniques are, comparing with the 'average human'. Now I know even the term average human doesn't have a strict definition, but the scale of intelligence[0] is so vast compared to the dumbest and smartest human, all that we need to know from that is if the technology is better than humans or not. If it is not, then I'll not care much, but if it is, then you've got my attention. >Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count. I think the problem lies with how we emulate intelligence. Will you be able to emulate me perfectly if you had not billions, but trillions of transistors? I'm assuming the answer is no, because, although we have all the power that the human brain (abstracting the neurons and other 'mechanical parts') has, we cannot simulate even a child's intelligence (now here I'm referring the ability to learn new things and communication, etc other abstract things etc, not computational ability). We need to know the technique just like we know have deep learning and are able to analyze pictures near-perfectly. >True AI is starting to look philosophically like a mirage Well, of course definitions vary. I think most people's (including mine) definition is that 'AI' (AGI, if you want to remove ambiguity) is surpassing human intelligence in the optimization area, like large scale optimizations like designing a spaceship, or curing cancer using techniques unknown to us right now. I remember reading that and considering his view very interesting. With our (normal) perspective we don't really see AI like people see it centuries ago. [0]: http://kruel.co/scale_of_intelligence.png http://kruel.co/scale_of_intelligence.png - I apologize for the poor quality picture; it was a quick google image search
vonnik 10y ago
This intro is pretty good, but may be a bit high level for some readers of HN. It seems like it's written for non-technical readers. The idea is that AI, ML and DL are nested like Russian dolls, each subsequent one a subset of the other. It might be better to explain why deep learning is so effective, in clear language: * Deep artificial neural networks are old, relatively simple combinations of math and code that are now able to produce accurate models through brute force because we have 1) vastly more computational power thanks to NVIDIA and distributed run-times; 2) much more data, and much larger labeled datasets thanks to people like Fei-Fei Li at Stanford; 3) better algorithms thanks to the work of Hinton, LeCun, Bengio, Ng, Schmidhuber and a raft of others. Deep is a technical term. It refers to the number of layers through which data passes in a neural net; that is, the number of mathematical operations it is subjected to, and the number of times it is recombined with other inputs. This recombination of inputs, moving deeper into the net, is the basis of feature hierarchy, which is another way of saying: we can cluster and classify data using more complex and abstract representations. That clustering and classification is at the heart of what deep learning does. Another way to think about it is as machine perception. So the overarching narrative in AI is that we've moved from the symbolic rules engines of the chess victors to the interpretation of complex sensory information. For a long time, people would say AI could beat a 30-year-old at chess but couldn't beat a 3-year old at basic tasks. That's no longer true. We can go around beating 3-year-olds at name games all day. AI mind, beginner's mind. But it's important to note that deep learning actually refers to other algorithms besides artificial neural networks. Deep reinforcement learning is one example. RL is also an old set of algorithms, which are goal-oriented. RL helps agents choose the right action in a given state to maximize rewards from the environment. Basically, they learn the function that converts actions to rewards given certain conditions, and that function is non-differentiable; that is, you can't learn it simply by backpropagating error, the way neural nets do. Deep RL is important because the most amazing algorithms, like AlphaGo, are combining deep neural nets (recognize the state of the Go board) with RL (pick the move most likely to succeed) and other components like Monte Carlo Decision Trees (limit the state space we explore). So we're moving beyond perception to algorithms that can make strategic decisions in increasingly complex environments. We've written more about this, and implemented many of these algorithms: http://deeplearning4j.org/ai-machinelearning-deeplearning.html http://deeplearning4j.org/ai-machinelearning-deeplearning.ht... http://deeplearning4j.org/reinforcementlearning.html http://deeplearning4j.org/reinforcementlearning.html http://github.com/deeplearning4j/rl4j http://github.com/deeplearning4j/rl4j

by ktta 10y ago

argonaut 10y ago

I hear this repeated a lot. But I've never actually seen a source from, say, the 90's or the 2000s, actually saying something like "if we're able to classify images better than humans, that's true AI." I suspect it's just hearsay that's been repeated over and over again.

svantana 10y ago
There used to be a concept of "AI completeness" which meant that some problems could only be solved by True AI, which would also be able to solve all other human-level problems. Doug Hofstadter writes in Gödel-Escher-Bach that he believes grandmaster-level chess is AI complete. That was obviously false in retrospect, and it's becoming increasingly obvious that there is no AI completeness at all.
- ragebol 10y ago
  For now, I take this blogpost [1] of Andrej Karpathy as a nice measure for AI hardness. [1] http://karpathy.github.io/2012/10/22/state-of-computer-vision/ http://karpathy.github.io/2012/10/22/state-of-computer-visio...
  - csydas 10y ago
    This is a good blog post and articulates well many of the issues I have with calling current work on AI subsets "AI". From my point of view, the results of current work with AI subsets are impressive, but they're hitting self-proclaimed benchmarks or someone poorly defined benchmarks and declaring AI status. Creating an advanced Go machine was very cool, but as Lee Sedol played more, he said he began to understand how AlphaGo "thought". The appended reddit thread here [1] has some interesting insight on why pulling back the curtain on current AI claims is appropriate. A lot of the times, "it's just calculations" is appropriate since we don't really have a machine considering what it's given, it's driving towards a singular goal in the best way possible. In the case of AlphaGo, it disregarded Lee Sedol's move because it had no care about the unusualness of it, it just wrote the move off as sub-optimal. When you see behavior like this where the systems are algorithmically defeated, it's very difficult to not be dismissive of the claims of AI. Again, there is a lot of coolness happening with subsets of AI research, but I don't feel that there is even a clear definition of what AI would entail - spaghetti code to get a desired result doesn't really help either, since it has to be persistent independent successes - to refer to Karpathy's article, it would have to make repeated assertions and understandings of similar photos with a high success rate to really be something spectacular. [1] https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_insight_into_alpha_gos_from_match_4/ https://www.reddit.com/r/baduk/comments/4a7wl2/fascinating_i...
- eli_gottlieb 10y ago
  >it's becoming increasingly obvious that there is no AI completeness at all. Bingo! The "general" part is the ability to learn new structures and task structures from environmental cues, and then construct informed prior beliefs about those new tasks and structures using causal relations to previously-observed tasks and structures. Nothing more!
reacweb 10y ago
In the 90's, John McCarthy was calling Go "the new Drosophila of AI". Now, we have AlphaGo that is "just" a bunch of "generic" deep learning with MC. Maybe John McCarthy was right and the ideas behind AlphaGo are the first steps toward true AI. I am very impressed.
- visarga 10y ago
  The guys at NVIDIA did something very cool recently. They took a simple CNN (convolutional neural net) and in 2-3 days they taught it to drive a car. Article: How Our Deep Learning Tech Taught a Car to Drive - https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3/ https://blogs.nvidia.com/blog/2016/05/06/self-driving-cars-3... These neural nets are really smart, and we don't know exactly how they work, we know only in principle. But if we asked the guy who made the self driving car what is the role of the 17th neuron in the 14th layer, he would probably have no idea. Just like human brains evolve through learning, so do neural nets. Yet many people think they are just clever tricks and not truly intelligent.
  - c0g 10y ago
    That's because they're not intelligent. They're layered function approximators with some specialisations for vision - conv layers. Their structure is designed by humans as is the data they are fed with. Maybe the human brain is the same, IMO it isn't. Humans seek out new experiences and can introspect and at their best assess their level of knowledge of a sibject. DNNs cannot.
  - argonaut 10y ago
    Did you read the paper? They did not teach it to drive a car. The car just does lane keeping - it doesn't even do turns or lane changes. This is stuff solved 10+ years ago. And even then it only achieves 98% autonomy on lane keeping - this is a task that needs 100% accuracy. You should not be running into the median every couple miles.
- eli_gottlieb 10y ago
  Sounds like McCarthy was right: the point of Drosophila is that it's a very small, simple model organism. You're not supposed to "solve biology" by modelling something in Drosophila, you're just supposed to be able to narrow the field to hypotheses that are not so flagrantly stupid they fail in a fruit-fly.
ktta 10y ago
hey argonaut, we meet again. I think you'd be interested in the parent comments of this[0] thread. [0]https://news.ycombinator.com/item?id=12322536 https://news.ycombinator.com/item?id=12322536

seanwilson 10y ago

> But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. I think the issue is that current examples of AI can mostly only solve problems in a single domain instead of in a variety of domains. Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence. For example, while AlphaGo is incredibly impressive, it can only play Go and nothing else. I think once we have a comprehensive AI program that can play many kinds of games, can have conversations, can complete school exam questions, can write stories etc. the question of what intelligence is will become more interesting.

visarga 10y ago
> once we have a comprehensive AI program that can play many kinds of games Before AlphaGo, DeepMind released a reinforcement learning algorithm that could play many Atari games just from the raw pixels on the screen, in many games surpassing humans. The same algorithm. https://arxiv.org/pdf/1312.5602.pdf https://arxiv.org/pdf/1312.5602.pdf Reinforcement learning is a general framework for learning behavior from acting in an environment with the purpose to maximize a reward. It can be used, and was used, in multiple domains. AlphaGo used RL as well. Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game. In fact, if the software was set to learn more games, it could learn them in addition to Go. Also, regarding other tasks: a neural net that recognizes cats can be easily made to recognize dogs too. A program that translates English to French can be made to translate other languages too. We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited. Recently there has been a paper "Learning without forgetting" (http://arxiv.org/abs/1606.09282v2 http://arxiv.org/abs/1606.09282v2) that underlines this very ability to span multiple domains and adapt easily to unseen tasks and data. > Developing a program that can generate solutions for a single specific niche of problems (while impressive) is not a convincing demonstration of intelligence. Saying that people can do many tasks is not exactly right because a particular person can only do a few tasks, those tasks she was trained to do. If I never learned German, I don't know German. That doesn't mean the brain itself is limited. AlphaGo was only trained on Go, and its internal architecture was optimized for this one task in order to make it more efficient, but the method is general and reusable. DeepMind said so themselves, the breakthrough is not that they beat Lee Sedol, but that they used a general method that can be used to do other tasks as well. It is not a limitation of AI that we generally make systems that are good at only one thing. If there is a limitation in AlphaGo, it is that it mastered a game where the whole situation is perfectly known (the Go board), while in reality many tasks are only partially known (such as card games, for example) so there is an extra uncertainty. But DeepMind and other researchers are working on that too.

Aissen 10y ago

If it's magic, it's AI; if it's understood, it's algorithms.

jules 10y ago
A chess computer isn't magic because you know that it is basically searching the game tree very fast with some heuristics. Neural networks are not like that. You know that it's a network of activation functions, but how that solves so many AI tasks is still more or less a mystery. So maybe the AI effect will be less strong with deep learning.
- mjn 10y ago
  I disagree that neural networks are necessarily mysterious. In most cases they can be straightforwardly understood as brute-force function approximation (approximation here meaning mostly interpolation, in some cases capable of a limited degree of extrapolation). You take many samples of a function (either because you have a huge data set, or because you can actually sample with a simulator), and incrementally modify a large network to slowly converge it towards an accurate approximation of the function. The advent of very large data sets and GPU training made this possible, but the tech is pretty well understood, at least at the level of a 5-mile-high view.
  - jules 10y ago
    Of course, but compared to how chess algorithms work it remains far more mysterious how exactly that produces an english description of a grid of pixel values.
    - throwaway729 10y ago
      Probably only because every CS undergrad or self-trained person sees at least a few search algorithms. So although they probably couldn't write a performant chess-playing program, they get the general idea. I expect in 20 years undergraduate curricula evolve to cover NNs, and then those courses will have their salient bits adopted into mass market expository books/blog posts/code examples/etc. At that point someone like you will be saying something like you said about whatever the new hotness is.

sguav 10y ago

What if we trained a CNN to "know [it] knows nothing"? https://en.wikipedia.org/wiki/I_know_that_I_know_nothing https://en.wikipedia.org/wiki/I_know_that_I_know_nothing

ktta 10y ago
You might have to see someone who took philosophy and a Machine Learning researcher about that :)

cgearhart 10y ago

I would argue that there is actually an unrecognized inverse of this effect, too. People have a tendency to conflate domain-specific human-level performance with "intelligence" (for some definition of that word). Then suddenly we jump to "DeepMind won at go, therefore humanity is on the brink of creating Skynet." Deep learning has been like this so far, because it works in many new domains that have resisted previous techniques like image recognition, speech recognition, go, etc. The "AI Effect" -- saying, "it's just an algorithm" once we succeed -- is an artifact of developing AI by using it as an algorithm for a domain-specific task. It is "just" an algorithm, because that's all it needs to be to win at go, identify cats, or whatever. Fundamentally, it's not very surprising that when you set out to make a system that's really good at playing go...you end up with a system that's really good at playing go. Of course it's hard -- that's what we should appreciate -- but most of the problems we consider "hard" are based on what we think it is hard for a computer to do in order to solve the problem. "Searching the space of all possible go moves is computationally intractable, therefore this problem is hard." In reality, the problem may not be as hard as we think if the agent doesn't have to search the space of all possible moves. "Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function...

eli_gottlieb 10y ago
>"Hard" problems are multi-optimization, where even the definitions aren't very clear: "learn as much as possible and live a happy life while making a productive living for yourself". Turn that into an objective function... Minimize the informational free-energy of the product of the multiplicatively inverted brain-to-body reinforcement-learned energy function of reward with the brain-to-body reinforcement-learned energy function of punishment, via active inference? Beh, those words are messy. The point is to talk about a Gibbs distribution with energy functions Reward(X, Y) and Punishment(X, Y), such that the "total" energy function is E(X, Y) \propto Punishment(X, Y) - Reward(X, Y). This then gives us a "goal distribution" for an active inference agent (like a human), defined "up to" the reinforcement-learned energy functions, whose limiting functions (need a lot of functional analysis and probabilistic reasoning over function spaces, there) are the "ground truth" causal relations by which the world causes reward signals through the body and its senses. https://arxiv.org/pdf/1503.04187.pdf https://arxiv.org/pdf/1503.04187.pdf

eli_gottlieb 10y ago

>But as soon as we are able to solve a problem that we think (feel?) only true AI (AGI) would be able to solve, as soon as we know how it was solved, it is no longer a mystery that warrants amazement and we argue that it is not real intelligence, just like the link below states. Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts, ex nihilo, rather than an efficient engine for distilling structure from experiences, and to be ontologically special to prove how special we humans are on a universal scale. There are cognitive science and neuroscience labs hard at work on deciphering how actually-existing human thought works, but as far as I can tell, everyone facinated by "AI" is so enamoured with intelligence being special that they don't pay attention to those labs. A great deal of books and papers are thus published, and as far as "AI" is concerned, it's all in vain somehow.

ktta 10y ago
>Well, the problem here is that we keep expecting "real intelligence" to have magical properties: we expect intelligence to be a way of creating new and interesting thoughts I think we might have to step into borderline philosophical subjects like epistemology and sentience depending on who you talk to to define intelligence and whether you're the free will group or determinism group and so on. I think you might agree with the statement that we can't yet come to a strict definition of what intelligence is, and once we have it, if we are going for intelligence or something more akin to stuff independent of intelligence like sentience and creative thought. I'm personally in the determinist group and agree that there is nothing inherently magical about intelligence, just us being simpletons who cannot comprehend how a microprocessor works when you just give the latest intel chip. It is so complex that we need a lot of time and energy to comprehend it, but like the intel chip, I believe there is a design aspect to the human brain rather than it being a magical black box. The problem is, the majority of us are the simpletons and have a long way to go before we have the knowledge that the intel chip makers have. I apologize for the bad intel chip analogy, I probably could've come up with something better :)

j1vms 10y ago

And on a related note, "better than humans", "comparable to human intelligence", and phrases like these since the early days of AI have always been a red herring. Not the least because these phrases do not imply what we are really comparing - better than all humans? some humans? How about just a person? What about just better than a 4-year-old? The criteria should have always been "faster / more efficiently", "more accurately", or "both" in comparing AI to human intelligence for a given task or activity. By this measure, AI had enduring production-ready successes since at least the 1980s. Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count. True AI is starting to look philosophically like a mirage, not the least because we may not ever have a strict meaningful definition for intelligence. What we consider intelligence varies, evolves, changes shape, consistency and predictability even if we consider just one of the various people we interact with on a given day. As one of the latest, I think Brian Eno (https://news.ycombinator.com/item?id=12027055 https://news.ycombinator.com/item?id=12027055), like others before him who have formalized a similar approach to the matter of intelligence, is on the right track. This approach to deciding what constitutes True AI would do much to counter and eventually prevent the "AI effect".

ktta 10y ago
>And on a related note, "better than humans", "comparable to human intelligence", and phrases like these since the early days of AI have always been a red herring Well, I agree that it is not definitive, but it definitely gives us some insight on where AI techniques are, comparing with the 'average human'. Now I know even the term average human doesn't have a strict definition, but the scale of intelligence[0] is so vast compared to the dumbest and smartest human, all that we need to know from that is if the technology is better than humans or not. If it is not, then I'll not care much, but if it is, then you've got my attention. >Let's say that was software that ran on post-VLSI hardware (1+ million transistors). Now, we have deep learning deployed on multiple GPUs each with a 15+ billion transistor count. I think the problem lies with how we emulate intelligence. Will you be able to emulate me perfectly if you had not billions, but trillions of transistors? I'm assuming the answer is no, because, although we have all the power that the human brain (abstracting the neurons and other 'mechanical parts') has, we cannot simulate even a child's intelligence (now here I'm referring the ability to learn new things and communication, etc other abstract things etc, not computational ability). We need to know the technique just like we know have deep learning and are able to analyze pictures near-perfectly. >True AI is starting to look philosophically like a mirage Well, of course definitions vary. I think most people's (including mine) definition is that 'AI' (AGI, if you want to remove ambiguity) is surpassing human intelligence in the optimization area, like large scale optimizations like designing a spaceship, or curing cancer using techniques unknown to us right now. I remember reading that and considering his view very interesting. With our (normal) perspective we don't really see AI like people see it centuries ago. [0]: http://kruel.co/scale_of_intelligence.png http://kruel.co/scale_of_intelligence.png - I apologize for the poor quality picture; it was a quick google image search

vonnik 10y ago

This intro is pretty good, but may be a bit high level for some readers of HN. It seems like it's written for non-technical readers. The idea is that AI, ML and DL are nested like Russian dolls, each subsequent one a subset of the other. It might be better to explain why deep learning is so effective, in clear language: * Deep artificial neural networks are old, relatively simple combinations of math and code that are now able to produce accurate models through brute force because we have 1) vastly more computational power thanks to NVIDIA and distributed run-times; 2) much more data, and much larger labeled datasets thanks to people like Fei-Fei Li at Stanford; 3) better algorithms thanks to the work of Hinton, LeCun, Bengio, Ng, Schmidhuber and a raft of others. Deep is a technical term. It refers to the number of layers through which data passes in a neural net; that is, the number of mathematical operations it is subjected to, and the number of times it is recombined with other inputs. This recombination of inputs, moving deeper into the net, is the basis of feature hierarchy, which is another way of saying: we can cluster and classify data using more complex and abstract representations. That clustering and classification is at the heart of what deep learning does. Another way to think about it is as machine perception. So the overarching narrative in AI is that we've moved from the symbolic rules engines of the chess victors to the interpretation of complex sensory information. For a long time, people would say AI could beat a 30-year-old at chess but couldn't beat a 3-year old at basic tasks. That's no longer true. We can go around beating 3-year-olds at name games all day. AI mind, beginner's mind. But it's important to note that deep learning actually refers to other algorithms besides artificial neural networks. Deep reinforcement learning is one example. RL is also an old set of algorithms, which are goal-oriented. RL helps agents choose the right action in a given state to maximize rewards from the environment. Basically, they learn the function that converts actions to rewards given certain conditions, and that function is non-differentiable; that is, you can't learn it simply by backpropagating error, the way neural nets do. Deep RL is important because the most amazing algorithms, like AlphaGo, are combining deep neural nets (recognize the state of the Go board) with RL (pick the move most likely to succeed) and other components like Monte Carlo Decision Trees (limit the state space we explore). So we're moving beyond perception to algorithms that can make strategic decisions in increasingly complex environments. We've written more about this, and implemented many of these algorithms: http://deeplearning4j.org/ai-machinelearning-deeplearning.html http://deeplearning4j.org/ai-machinelearning-deeplearning.ht... http://deeplearning4j.org/reinforcementlearning.html http://deeplearning4j.org/reinforcementlearning.html http://github.com/deeplearning4j/rl4j http://github.com/deeplearning4j/rl4j