3 ms·

Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope,

Brief description of the issue for non-experts: Supervised learning You can judge the output of your network against ground truth. You say that's a cat? Nope, it's a dog! And then slightly adjust your network so it's less likely to give that wrong answer in the future. How exactly you adjust the network is what backpropagation describes (in combination with something called a learning rate). Unsupervised learning You need to learn without someone telling you what the answer is. Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. If you don't have someone telling you you've made a mistake, how can you know when to adjust your network? And if you don't know what the truth is, exactly how to adjust the network becomes tricky. Somehow biological brains work without that oracle, and there are lots of ideas about how it does that. But right now for artificial intelligence none of those ideas has been shown to work so amazingly well that it has taken off like backprop has in the supervised learning world. Hinton wants to find that amazing algorithm.

aaronblohowiak 9y ago
I wager it has something to do with prediction and rules
sillysaurus3 9y ago
You need to learn without someone telling you what the answer is. Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. There is: Death, without reproducing. If an organism dies quickly but left children, they were biologically successful. If you didn't, then your evolutionary branch goes away. I'm not sure this can be simulated, but it might be worth trying. Biological brains are the product of a billion years of the above strategy, so hopefully it's feasible to do the same thing. Don't people already do that? Well, no. The strategy implies some sort of physics simulation with general rules, not a specific ruleset tuned to your domain.
- striking 9y ago
  >> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. [emphasis mine] > There is: Death, without reproducing. The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulate many possible configurations of actors until the end of their lifetimes.
  - sillysaurus3 9y ago
    Yet it's the only strategy that we're absolutely certain produces intelligent life.
    - striking 9y ago
      Are you absolutely certain we qualify as intelligent life? edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.
      - sillysaurus3 9y ago
        An interesting question, but if we make a program that makes wry jokes about what it means to be alive or argues persuasively about its own freedoms, the world will notice.
      - TheOtherHobbes 9y ago
        It's a good question, and I think the answer is - up to a point. We're biased towards individualism and the illusion of personal intelligence making clever decisions using its own resources, but in fact our real intelligence is collective. Individually, most humans are a little bit smarter than other animals, but not much. Humans are also excellent mimics. We learn by copying, and a lot of what we consider intelligence is copied behaviour and copied beliefs - often applied unconsciously. So we've created systems - spoken language, writing, science, technology - that can externalise, collect, and preserve the best of our learning, so the species as a whole, including average humans, can get the benefit of it. This works fine as far as it does, but our political and economic systems are still founded on animal logic (i.e. personal advantage no matter the cumulative long-term cost) instead of conscious collective intelligence amplification and species optimisation. Based on this, AI won't get far unless someone invents an analogous way to externalise and abstract learning, so different AI projects can bootstrap each other, and there are common systems for abstraction and generalisation. Unsupervised learning isn't going to be all that interesting unless it can do that. It'll be a fun toy for certain classes of problems, but it won't be anywhere close to an AGI.
GuiA 9y ago
Well, the algorithm is death and reproduction, isn't it? If the output of your networks performs well enough when it confronts the real world, you get to reproduce before you die, and some of the essence of your network gets to live for another generation. If not, you die, and your network doesn't get to contribute to the next generation.
eanzenberg 9y ago
Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.
- 31reasons 9y ago
  I think innate intelligence/ animal intelligence is developed by evolutionary algorithm. You don't need to do back prop for it. If the network is wrong, the animal just dies.
  - fngl51 9y ago
    I would agree with this approach. In evolution, it would be the equivalent to a conditional lethal mutation. In humans and even non-humans, may behaviors are learned through a form of adaptive behavior that oftentimes becomes a form of abductive reasoning. What is learned is "good enough" to serve as ground truth until there is evidence to contradict.
  - eanzenberg 9y ago
    What you call "evolutionary algorithm" is just a pre-trained model your DNA created. That took 3.5 billion years to get to this point.
- iandanforth 9y ago
  Can we go with 'incomplete' rather than 'false'? :) One of the most interesting areas of study for me is early childhood motor learning which is a fascinating combination of unsupervised, reinforced, and imitation learning. There are even significant intrinsic knowledge (instinctual/reflexive/morphological) effects. Also I really like to encompass non-human learning when talking about this stuff to remind myself of all the work nature did before we got to cultural transmission of knowledge.
  - eanzenberg 9y ago
phkahler 9y ago
>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. Is it possible that your conscious self is the oracle that provides feedback to subconscious networks?
- tree_of_item 9y ago
  No, why would "your conscious self" know any of the answers? The problems biological organisms face aren't even well defined enough to have "answers" generally.
  - phkahler 9y ago
    People learn many things by being told a set of rules and/or objectives. Over time they can self-evaluate their performance against those rules and objective. I suppose an animal may have a need - it feel hungry. If it can satisfy that need then it should reinforce whatever it did to get not-hungry. I suppose that doesn't require conscious awareness. But at some level of complexity we have to be able to plan more complex actions (hunting, tracking) and be able to self-evaluate performance on the sub-tasks used to get food.
Tenobrus 9y ago
What you've described for unsupervised learning is really closer to reinforcement learning, where you get rewards without any information. True unsupervised learning is more like "figure out the clusters of these n-dimensional vectors".
mhermher 9y ago
Pain and pleasure are the oracles. Maybe not at every moment, but within a given lag.
sjg007 9y ago
This is codifying "instinct" which presumably has already been done at least once (via DNA in some species).
samfisher83 9y ago
Well your teachers, parents, society tell you what is right and wrong so we really aren't learning unsupervised.
- eanzenberg 9y ago
  Very little of what human's learn is done "unsupervised".
- dtech 9y ago
  Also things like pain, hunger and pleasure also are for the most part generated outside of the "learning" parts of the brain, so similar to a pre-defined utility function/oracle.
  - makapuf 9y ago
    Yes a 1yo child has a good way of getting feedback from the environment when learning to walk. They fall and it hurts !
- axontaxi 9y ago
  But who teaches the previous generations and how? Chicken or egg first? I think the origin is communication and memory. When a being (X) signals to another being (Y) the origins of its pain (signal of increased death probability) and X and Y can remember this cause, they can begin to teach.
skybrian 9y ago
Biological brains try to predict the immediate future: what they will sense next. That's the ground truth.
yters 9y ago
Solomonoff induction captures the algorithm, but it cannot be implemented. Perhaps that means the mind is non algorithmic.
snowwrestler 9y ago
Having just raised a human from newborn to walking, talking kid, I have to say that her life so far has been chock full of oracles guiding her learning. Long before language, babies spend hours every day just interacting with the physical environment, which is "true": the toy is here, not there. The body weighs a certain amount. If you move the head this way, you'll roll over. If you move it that way, no rolling. Etc. And parents provide a different kind of guidance. We react differently to different sounds, actions, movements of child. Even if these are not "truth" philosophically, they represent for the child emotional truth. I'm just a guy who reads things about AI on the Internet, but I have to say it sometimes seems to me like AI folks have funny ideas of what "intelligence" is, and how it develops. The description of "unsupervised" learning above doesn't comport with any human experience I recognize. IME, humans receive constant immediate feedback on everything we do in life.

aaronblohowiak 9y ago

I wager it has something to do with prediction and rules

sillysaurus3 9y ago

You need to learn without someone telling you what the answer is. Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. There is: Death, without reproducing. If an organism dies quickly but left children, they were biologically successful. If you didn't, then your evolutionary branch goes away. I'm not sure this can be simulated, but it might be worth trying. Biological brains are the product of a billion years of the above strategy, so hopefully it's feasible to do the same thing. Don't people already do that? Well, no. The strategy implies some sort of physics simulation with general rules, not a specific ruleset tuned to your domain.

striking 9y ago
>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. [emphasis mine] > There is: Death, without reproducing. The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulate many possible configurations of actors until the end of their lifetimes.
- sillysaurus3 9y ago
  Yet it's the only strategy that we're absolutely certain produces intelligent life.
  - striking 9y ago
    Are you absolutely certain we qualify as intelligent life? edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.
    - sillysaurus3 9y ago
      An interesting question, but if we make a program that makes wry jokes about what it means to be alive or argues persuasively about its own freedoms, the world will notice.
    - TheOtherHobbes 9y ago
      It's a good question, and I think the answer is - up to a point. We're biased towards individualism and the illusion of personal intelligence making clever decisions using its own resources, but in fact our real intelligence is collective. Individually, most humans are a little bit smarter than other animals, but not much. Humans are also excellent mimics. We learn by copying, and a lot of what we consider intelligence is copied behaviour and copied beliefs - often applied unconsciously. So we've created systems - spoken language, writing, science, technology - that can externalise, collect, and preserve the best of our learning, so the species as a whole, including average humans, can get the benefit of it. This works fine as far as it does, but our political and economic systems are still founded on animal logic (i.e. personal advantage no matter the cumulative long-term cost) instead of conscious collective intelligence amplification and species optimisation. Based on this, AI won't get far unless someone invents an analogous way to externalise and abstract learning, so different AI projects can bootstrap each other, and there are common systems for abstraction and generalisation. Unsupervised learning isn't going to be all that interesting unless it can do that. It'll be a fun toy for certain classes of problems, but it won't be anywhere close to an AGI.

GuiA 9y ago

Well, the algorithm is death and reproduction, isn't it? If the output of your networks performs well enough when it confronts the real world, you get to reproduce before you die, and some of the essence of your network gets to live for another generation. If not, you die, and your network doesn't get to contribute to the next generation.

eanzenberg 9y ago

Actually: >>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning. Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.

31reasons 9y ago
I think innate intelligence/ animal intelligence is developed by evolutionary algorithm. You don't need to do back prop for it. If the network is wrong, the animal just dies.
- fngl51 9y ago
  I would agree with this approach. In evolution, it would be the equivalent to a conditional lethal mutation. In humans and even non-humans, may behaviors are learned through a form of adaptive behavior that oftentimes becomes a form of abductive reasoning. What is learned is "good enough" to serve as ground truth until there is evidence to contradict.
- eanzenberg 9y ago
  What you call "evolutionary algorithm" is just a pre-trained model your DNA created. That took 3.5 billion years to get to this point.
iandanforth 9y ago
Can we go with 'incomplete' rather than 'false'? :) One of the most interesting areas of study for me is early childhood motor learning which is a fascinating combination of unsupervised, reinforced, and imitation learning. There are even significant intrinsic knowledge (instinctual/reflexive/morphological) effects. Also I really like to encompass non-human learning when talking about this stuff to remind myself of all the work nature did before we got to cultural transmission of knowledge.
- eanzenberg 9y ago

phkahler 9y ago

>> Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life to judge actions/decisions against. Is it possible that your conscious self is the oracle that provides feedback to subconscious networks?

tree_of_item 9y ago
No, why would "your conscious self" know any of the answers? The problems biological organisms face aren't even well defined enough to have "answers" generally.
- phkahler 9y ago
  People learn many things by being told a set of rules and/or objectives. Over time they can self-evaluate their performance against those rules and objective. I suppose an animal may have a need - it feel hungry. If it can satisfy that need then it should reinforce whatever it did to get not-hungry. I suppose that doesn't require conscious awareness. But at some level of complexity we have to be able to plan more complex actions (hunting, tracking) and be able to self-evaluate performance on the sub-tasks used to get food.

Tenobrus 9y ago

What you've described for unsupervised learning is really closer to reinforcement learning, where you get rewards without any information. True unsupervised learning is more like "figure out the clusters of these n-dimensional vectors".

mhermher 9y ago

Pain and pleasure are the oracles. Maybe not at every moment, but within a given lag.

sjg007 9y ago

This is codifying "instinct" which presumably has already been done at least once (via DNA in some species).

samfisher83 9y ago

Well your teachers, parents, society tell you what is right and wrong so we really aren't learning unsupervised.

eanzenberg 9y ago
Very little of what human's learn is done "unsupervised".
dtech 9y ago
Also things like pain, hunger and pleasure also are for the most part generated outside of the "learning" parts of the brain, so similar to a pre-defined utility function/oracle.
- makapuf 9y ago
  Yes a 1yo child has a good way of getting feedback from the environment when learning to walk. They fall and it hurts !
axontaxi 9y ago
But who teaches the previous generations and how? Chicken or egg first? I think the origin is communication and memory. When a being (X) signals to another being (Y) the origins of its pain (signal of increased death probability) and X and Y can remember this cause, they can begin to teach.

skybrian 9y ago

Biological brains try to predict the immediate future: what they will sense next. That's the ground truth.

yters 9y ago

Solomonoff induction captures the algorithm, but it cannot be implemented. Perhaps that means the mind is non algorithmic.

snowwrestler 9y ago

Having just raised a human from newborn to walking, talking kid, I have to say that her life so far has been chock full of oracles guiding her learning. Long before language, babies spend hours every day just interacting with the physical environment, which is "true": the toy is here, not there. The body weighs a certain amount. If you move the head this way, you'll roll over. If you move it that way, no rolling. Etc. And parents provide a different kind of guidance. We react differently to different sounds, actions, movements of child. Even if these are not "truth" philosophically, they represent for the child emotional truth. I'm just a guy who reads things about AI on the Internet, but I have to say it sometimes seems to me like AI folks have funny ideas of what "intelligence" is, and how it develops. The description of "unsupervised" learning above doesn't comport with any human experience I recognize. IME, humans receive constant immediate feedback on everything we do in life.