5 ms·

Can someone more knowledgeable explain why biological systems are considered unsupervised instead of reinforcement based systems? While it seems intuitive that

• rm999 11y ago
  The brain needs to form structure before any sort of reinforcement learning is even possible. This structure comes from an "unsupervised" process we don't understand well.
  • justicezyx 11y ago
    Isn't it from millions year of natural selection? The reason why a human brain can think, is just by chance.
    • daveguy 11y ago
      Personally I think a lot of it has to do with evolution. We certainly have different sections of the brains that are more utilized in different tasks, auditory, visual, motor, etc. I think it is likely a lot of the initial feedback is preset -- the instinctual positive and negative responses of certain stimulus.
    • conceit 11y ago
      As much as it doesn't help to call it intelligent design, chance is not any good either, if asking for a reason. Random chance is as good as no reason at all.
      • justicezyx 11y ago
        Mixing natural selection and intelligent design is a problem, but that's not a problem for natural selection.
        • conceit 11y ago
          What's the context? The GP probably meant that intelligence is an emergent property not directly coded in DNA. Natural selection is not pure chance. At least seeing it that way doesn't yield anything except maybe lowered expectations. You want some expectations. Hiding you theory behind Nature is not any less religious. Nature is pretty much synonym with living things, existence in general, and actually stems from to be born. Nature as the reason for things being born is thus kind of true by definition. The rational behind the generational evolution is the exact opposite of pure chance, though. Chance is only applicable if there are two possible outcomes. In hindsight, only one outcome was possible, so there are no chances, just facts. The argument I am getting at is, self similarity seems to be such a common property, that it's maybe inevitable somewhere in a complex system. From there to self awareness and critical thinking is a long way to go, but what you call millions of years is a blink of an eye on a cosmological scale. The idea of complexity includes enough variety to appear random to us, so we might agree that it's just a difficult to express concept and this is only semantic quibbling.
• baobabaobab 11y ago
  >While it seems intuitive that most individual "intelligent" systems in animals can be seen as unsupervised, isn't life itself driven in a reinforced manner? I think apparently unsupervised systems could be explained by models that predict a future input given their current state and input. Correct predictions are reinforced. An RNN-like model. (If the network is small, it should learn some compressed representation, which can be used as an input to a more abstract layer that makes more general predictions over a longer time period)
• nabla9 11y ago
  There is no enough reinforcement volume to learn anything complex. If child gets external reward for every waking moment until she is 12 years old, it's just 4.2 million signals. Reinforcement learning works for fine motor control and other tasks where the feedback loop is tight and immediate. Reinforcement and conditioning can also modulate high level cognition and behavior, but it's not the secret sauce of learning.
  • sabertoothed 11y ago
    I could not follow your argument. Could you elaborate? (Honest question)
    • deleted 11y ago
      [deleted]
    • nabla9 11y ago
      Reinforcement learning is learning by interacting with an environment. RL agent learns from the consequences of its actions. Biological system don't live long enough to get enough feedback to learn complex behavior trough consequences. Animal or human must be able to generalize and categorize what they have learned correctly without external feedback teaching it how to derive the function that's doing it. For example, if you want to learn how to tie a complex knot and learn it trough trial and error you might have try it million times if you improve your behavior mainly trough consequences of your actions. In practice you probably try only 5-10 times before you learn to do it and it involves pausing and looking at the problem. There is some kind of unsupervised model building happening that is not involving external input.
      • sabertoothed 11y ago
        Ok, I see what you mean now. But it seems like (or one could misunderstand you in way that) you see those concepts as mutually exclusive. I would assume a combination of reinforcement learning and unsupervised (and supervised) learning. Rats have been trained to detect landmines and then go back to their trainers and show them the mine. This is complex behaviour that was taught using reinforcement (at least on a top level). There will be some unsupervised learning going on in the rat's brain on a lower level. But it is complex behaviour and it's been reinforcement learnt.
• sabertoothed 11y ago
  I would agree. Could one not argue that even unsupervised learning is kind of reinforced by let's say the emotion you obtain from successfully performing a learning task? Then the reward signal does not come from the external environment but from resolving cognitive dissonance in the brain. Happy to be proved wrong by experts here.
• daveguy 11y ago
  In AI the definition of reinforcement learning vs unsupervised is that reinforcement requires external identification of the correct result to compare with the given result. If going through life you had artificial reality display that flashed "apple" in front of every apple and "orange" in front of every orange you would have reinforcement learning. There is certainly a major component of reinforcement in teaching, but that does not account for all of learning. With humans we are able to transfer concepts to different situations without examples and there are a lot of categories that are identified without any explicitly reinforced examples.
  • zardo 11y ago
    You're describing supervised learning. Reinforcement learning is more like you stick random things in your mouth, and if one tastes good, you learn to stick that kind of thing in your mouth more often.
    • daveguy 11y ago
      Well, yes, I was describing supervised learning (as what doesn't happen) where labels are not present is unsupervised (which makes things much more difficult). Reinforcement is more like a special case between supervised and unsupervised. Difficult like unsupervised because the labels aren't known, but with some feedback that is not a direct answer. Sorry for the confusion. There is very little in the way of purely unsupervised learning. I think the distinction between unsupervised and reinforcement is not very well defined. I would argue that even a "pure" unsupervised algorithm like k-means has a kind of reinforcement in the group means.
• daveguy 11y ago
  In AI the definition of reinforcement learning vs unsupervised is that reinforcement requires external identification of the correct result to compare with the given result. If going through life you had artificial reality display that flashed "apple" in front of every apple and "orange" in front of every orange you would have reinforcement learning. There is certainly a major component of reinforcement in teaching, but that does not account for all of learning. There is a lot of transfer of concepts to different situations and there are a lot of categories that are identified without any explicitly reinforced examples.