7 ms·

I recently had a child. Watching her learn inspires a sense of amazement. It starts slow with basic grasping, then shaking, then hitting, then tracking, then po

I recently had a child. Watching her learn inspires a sense of amazement. It starts slow with basic grasping, then shaking, then hitting, then tracking, then pointing. You really see how human's learn from first principles. One difference I perceive between a child an a neural network is that it appears a child does not start from a blank slate. The brain of a baby seems to come hard-wired with some basic instinctual functionality like grabbing and object tracking.

EGreg 4y ago
Chomsky is that you?
- thomassmith65 4y ago
  That joke probably is going over a lot of heads. People associate Chomsky with Manufacturing Consent now. https://en.wikipedia.org/wiki/Verbal_Behavior#Chomsky's_review_and_replies https://en.wikipedia.org/wiki/Verbal_Behavior#Chomsky's_revi...
- random314 4y ago
staticassertion 4y ago
> One difference I perceive between a child an a neural network is that it appears a child does not start from a blank slate. The brain of a baby seems to come hard-wired with some basic instinctual functionality like grabbing and object tracking. I think that's likely, but also babies have ~9 months of development, at least some of that time (3-6 months) they're moving around/ have a brain.
- deltree7 4y ago
  Babies have Millions of years of development. I still can't believe we start with Babies as base point to start learning. When the babies are born it is almost on auto-pilot (Nature > Nurture)
- dmarchand90 4y ago
  Babies seem to have most of the major emotion expressions and smiling appears to be universal https://www.galvnews.com/health/free/article_c0d6e928-4122-5337-9c26-7f4e357dbd3b.html https://www.galvnews.com/health/free/article_c0d6e928-4122-5...
bmitc 4y ago
> The brain of a baby seems to come hard-wired with some basic instinctual functionality like grabbing and object tracking. My first thought is, what else are the hands and eyes supposed to do? Haha. They're evolved to do those exact things.
- tsimionescu 4y ago
  Object tracking is actually a very complex behavior, requiring significant computational power. The eye has little to do with this, except for providing the raw sensory information. Basically the brain has to interpret each single image, then stitch together multiple images, understand that they are related, recognize objects within them, then start interpreting differences between them as movement etc.
  - bmitc 4y ago
    But the serious part of my point is that the eye didn't evolve in a vacuum. It isn't "just a sensor" in an evolutionary sense because it evolved along with the brain and other senses as part of the holistic vision system. It's not like the brain is born with a blank slate and the eye as a sensor. It comes prebuilt with firmware and drivers, so to speak. That's then catalyzed by experience. I think this is part of the problem in computer vision. Everyone just takes cameras and stitches them into a computer, hoping to be able to figure it out. It basically doesn't work. I think there needs to be embodied vision. I'd love to see any research that explores this. I have an intuitive suspicion that our vision system and processing is tightly bound to our body, in particular our neck and hands and legs. (This is unfounded, since I'm not a biologist or cognitive scientist, and is just my naive hunch. I'd be surprised if it's far off though.)
    - tsimionescu 4y ago
      That makes a lot of sense, yes. Related to your hunch, I'm also speculating, but I'd think at the very least our eye muscles are an integral part of our vision system, it's not just the retina itself.
    - tibbydudeza 4y ago
exodust 4y ago
I wonder how much the child's sensory reward system leads to memories and associations. How can artificial neural networks ever "feel" good about anything, and use that to learn? A child learns that petting a cat is better than hitting the cat. Petting feels better, and the cat will stick around for longer. But how does AI learn the same thing without any pleasurable sense the petting provides? Also will we see robots throwing tantrums? Children learn this tactic early, and it works to achieve desired outcomes. Later they progress to crafty manipulation tactics, such as carefully executed tone when asking for treats etc.
- atwood22 4y ago
  Neural networks have loss functions. In a sense, feeling good and bad can be looked at as outcomes of a loss function.
- hndamien 4y ago
  We have a cost function that could loosely be correlated to a pain/sadness function. Perhaps it makes sense to also include a pleasure function to give them some kind of autonomous direction. Perhaps it is this part of AI that will ultimately lead to trouble. Is pleasure the inverse of pain?
  - exodust 4y ago
    I think the inverse of pain is just comfortable or satisfied. Like a cat sitting on the mat. If the cat finds an electric heated blanket nearby and moves from mat to warm blanket, we are now in pleasure zone. Cat now has more than it needs. AI could use a pleasure function, but I'm not sure how it could justify seeking or getting more than it needs. The "pleasure" would need to be linked to a measurable benefit for its hardware or systems, and energy would need to be used to achieve that. I guess it could happen, but then we'll need greed functions to cap the pursuit of pleasure!
- littlestymaar 4y ago
  > Also will we see robots throwing tantrums? Children learn this tactic early, and it works to achieve desired outcomes. Sigh. Tantrum isn't a “learnt tactic” (it's the child's inability to control his frustration / other emotion) and it doesn't work to achieve desired outcome which is why they stop when the child is mature enough to have better emotional control (unless you're trying to raise a tyrannical child, in which case giving your child what he wants when he throws a tantrum is a good way to succeed…)
spaceman_2020 4y ago
My toddler's default mode of learning seems to be "let's try to eat it and see what happens" :)
- t0mas88 4y ago
  They all do it, apparently it's because their motor skills initially aren't developed enough to use their fingers to learn texture. Putting things in your mouth provides more data so to speak. Mine also really liked eating sand for a while :-)
  - thegabriele 4y ago
    I believe it's much simplier: since eating is primary for any living species, how would you differentiate food from non-food while understanding the world (and still need to eat)?
    - bearmode 4y ago
      The problem with that is that if it isn't food they usually don't try to eat it. They just put the object in their mouths and kind of feel it around. There's no attempt to consume it.
- dachryn 4y ago
  in contrast to my cat: let's try to smell it
t0mas88 4y ago
Mine is now 1.5 and the amazement continues. Hearing them learn to speak is also at a whole different level than any algorithm or adult does. They use some rudimentary logic even at that age. There is a thing that's in English called an "egg cup" (according to Google Translate :-)), there is one word for that thing in Dutch: "eierdop". My daughter, just having heard the word "egg" for the first time that day looked at an egg cup and decided that it was "ei-bakje" in Dutch, which roughly translates to "small container for egg". Only took maybe 2 seconds. I think that ability to find relationships between things and extrapolate very rapidly from them is what makes human kids so fast at learning. Which may have something to do with the fact that human memory is very associative, almost like what near 100% data locality would be in CPU design with many many cores.
- d1sxeyes 4y ago
  Dutch is weird. I was interested to see whether 'dop' or 'erdop' meant something similar to cup, and the question is really just one of spacing. I discovered actually 'dop' means something like 'shell'. So APPARENTLY eierdop all as one word means egg cup, but eier dop as two separate words means egg shell?
  - tibbydudeza 4y ago
    "eierdop" for me means the shell of the egg (Afr) ???.
  - t0mas88 4y ago
    Dutch has less spaces. Generally if it's one object, it's one word. This causes a typical spelling mistake native Dutch speakers make in English: we combine words into one that should be two words. In Dutch "eier dop" does not exist. Could be "ei dop" and then you're taking about two separate objects, an egg and a cap (as in bottle cap). And that's also immediately an example, I tried to type that as bottlecap and my phone corrected it with an extra space ;-)
- stevenjgarner 4y ago
  Thanks so much for sharing. Refreshes my own experience. I was so clear what I wanted to do with my life - so many goals and aspirations. Then I remember holding my first-borne in one hand. Less than an hour old she peered up to me almost inquisitively as if to say "I hope you got this?" At that moment I realized what my life was about. Can barely remember those goals and aspirations now. As they say in Spanish, my first borne now has 42 years. Brought 6 kids and 5 grandchildren into this world. As I age I am reminded by the same process you talk about but in reverse now. Watch the Robin Williams movie "Awakenings" [1]. Really portrays it. We come into this world absolutely unable to do much of anything. We fade in. Then ... trust me ... we fade out. But in between, if we're really lucky we get to shine. I wonder if a neural network will ever do that? [1] https://www.imdb.com/title/tt0099077/ https://www.imdb.com/title/tt0099077/
icare_1er 4y ago
Observing how children need a fairly limited amount of "learning examples" or "training sets" to learn that a cat is a cat, for instance, remains one of the most solid argument convincing me that AI fundamentally is not reproducing what actually happens in human brains.
- dachryn 4y ago
  actually it's closer than you might think. If you consider the eye and visual cortex to be the first layers of a deep learning network, we are actually getting a relatively similar system. However it has limitations further down the line in information representation stuff. The eye is genetically programmed to detect lines, hard edges, colours, and more complex patterns. This is exactly the same as the first layers of a neural net. However we then humanly assemble stuff like cat = eyes+fur+pointy+cute+small+meowing. A neural net has no way yet to represent information at that level, but we are getting closer there than going beyond dumb numbers. A neural net has implicitly the concept, but is not yet doing the 3d representation inside itself, which makes it harder to define something like 'ears' and 'tail'
  - tibbydudeza 4y ago
    Yep recognizing a cat from a hamster - is it just pixels or broken down into higher level concepts like head, body, legs and tail.
tominous 4y ago
In animals some things are definitely hardwired. I had a pet rabbit and made the mistake of watching a nature documentary about snakes when she was in the room. The mere sound of a rattle snake on TV caused the rabbit to stamp a few times, spray urine, and run away to her hutch (skidding on the floor as the left). I'm sure humans have similar instincts. For example, the tendency of young kids to avoid plants until they are shown that they are edible - possibly leading to the general preference of artificial foods over anything that might be found in a garden. [1] I've also noticed that toddlers don't gravitate towards knives the same way they do everything else. I certainly wouldn't rely on this behaviour, but I've noticed it in my own children and others. [1] https://www.theatlantic.com/health/archive/2013/12/why-babies-fear-plants/281954/ https://www.theatlantic.com/health/archive/2013/12/why-babie...

EGreg 4y ago

Chomsky is that you?

thomassmith65 4y ago
That joke probably is going over a lot of heads. People associate Chomsky with Manufacturing Consent now. https://en.wikipedia.org/wiki/Verbal_Behavior#Chomsky's_review_and_replies https://en.wikipedia.org/wiki/Verbal_Behavior#Chomsky's_revi...
random314 4y ago

staticassertion 4y ago

> One difference I perceive between a child an a neural network is that it appears a child does not start from a blank slate. The brain of a baby seems to come hard-wired with some basic instinctual functionality like grabbing and object tracking. I think that's likely, but also babies have ~9 months of development, at least some of that time (3-6 months) they're moving around/ have a brain.

deltree7 4y ago
Babies have Millions of years of development. I still can't believe we start with Babies as base point to start learning. When the babies are born it is almost on auto-pilot (Nature > Nurture)
dmarchand90 4y ago
Babies seem to have most of the major emotion expressions and smiling appears to be universal https://www.galvnews.com/health/free/article_c0d6e928-4122-5337-9c26-7f4e357dbd3b.html https://www.galvnews.com/health/free/article_c0d6e928-4122-5...

bmitc 4y ago

> The brain of a baby seems to come hard-wired with some basic instinctual functionality like grabbing and object tracking. My first thought is, what else are the hands and eyes supposed to do? Haha. They're evolved to do those exact things.

tsimionescu 4y ago
Object tracking is actually a very complex behavior, requiring significant computational power. The eye has little to do with this, except for providing the raw sensory information. Basically the brain has to interpret each single image, then stitch together multiple images, understand that they are related, recognize objects within them, then start interpreting differences between them as movement etc.
- bmitc 4y ago
  But the serious part of my point is that the eye didn't evolve in a vacuum. It isn't "just a sensor" in an evolutionary sense because it evolved along with the brain and other senses as part of the holistic vision system. It's not like the brain is born with a blank slate and the eye as a sensor. It comes prebuilt with firmware and drivers, so to speak. That's then catalyzed by experience. I think this is part of the problem in computer vision. Everyone just takes cameras and stitches them into a computer, hoping to be able to figure it out. It basically doesn't work. I think there needs to be embodied vision. I'd love to see any research that explores this. I have an intuitive suspicion that our vision system and processing is tightly bound to our body, in particular our neck and hands and legs. (This is unfounded, since I'm not a biologist or cognitive scientist, and is just my naive hunch. I'd be surprised if it's far off though.)
  - tsimionescu 4y ago
    That makes a lot of sense, yes. Related to your hunch, I'm also speculating, but I'd think at the very least our eye muscles are an integral part of our vision system, it's not just the retina itself.
  - tibbydudeza 4y ago

exodust 4y ago

I wonder how much the child's sensory reward system leads to memories and associations. How can artificial neural networks ever "feel" good about anything, and use that to learn? A child learns that petting a cat is better than hitting the cat. Petting feels better, and the cat will stick around for longer. But how does AI learn the same thing without any pleasurable sense the petting provides? Also will we see robots throwing tantrums? Children learn this tactic early, and it works to achieve desired outcomes. Later they progress to crafty manipulation tactics, such as carefully executed tone when asking for treats etc.

atwood22 4y ago
Neural networks have loss functions. In a sense, feeling good and bad can be looked at as outcomes of a loss function.
hndamien 4y ago
We have a cost function that could loosely be correlated to a pain/sadness function. Perhaps it makes sense to also include a pleasure function to give them some kind of autonomous direction. Perhaps it is this part of AI that will ultimately lead to trouble. Is pleasure the inverse of pain?
- exodust 4y ago
  I think the inverse of pain is just comfortable or satisfied. Like a cat sitting on the mat. If the cat finds an electric heated blanket nearby and moves from mat to warm blanket, we are now in pleasure zone. Cat now has more than it needs. AI could use a pleasure function, but I'm not sure how it could justify seeking or getting more than it needs. The "pleasure" would need to be linked to a measurable benefit for its hardware or systems, and energy would need to be used to achieve that. I guess it could happen, but then we'll need greed functions to cap the pursuit of pleasure!
littlestymaar 4y ago
> Also will we see robots throwing tantrums? Children learn this tactic early, and it works to achieve desired outcomes. Sigh. Tantrum isn't a “learnt tactic” (it's the child's inability to control his frustration / other emotion) and it doesn't work to achieve desired outcome which is why they stop when the child is mature enough to have better emotional control (unless you're trying to raise a tyrannical child, in which case giving your child what he wants when he throws a tantrum is a good way to succeed…)

spaceman_2020 4y ago

My toddler's default mode of learning seems to be "let's try to eat it and see what happens" :)

t0mas88 4y ago
They all do it, apparently it's because their motor skills initially aren't developed enough to use their fingers to learn texture. Putting things in your mouth provides more data so to speak. Mine also really liked eating sand for a while :-)
- thegabriele 4y ago
  I believe it's much simplier: since eating is primary for any living species, how would you differentiate food from non-food while understanding the world (and still need to eat)?
  - bearmode 4y ago
    The problem with that is that if it isn't food they usually don't try to eat it. They just put the object in their mouths and kind of feel it around. There's no attempt to consume it.
dachryn 4y ago
in contrast to my cat: let's try to smell it

t0mas88 4y ago

Mine is now 1.5 and the amazement continues. Hearing them learn to speak is also at a whole different level than any algorithm or adult does. They use some rudimentary logic even at that age. There is a thing that's in English called an "egg cup" (according to Google Translate :-)), there is one word for that thing in Dutch: "eierdop". My daughter, just having heard the word "egg" for the first time that day looked at an egg cup and decided that it was "ei-bakje" in Dutch, which roughly translates to "small container for egg". Only took maybe 2 seconds. I think that ability to find relationships between things and extrapolate very rapidly from them is what makes human kids so fast at learning. Which may have something to do with the fact that human memory is very associative, almost like what near 100% data locality would be in CPU design with many many cores.

d1sxeyes 4y ago
Dutch is weird. I was interested to see whether 'dop' or 'erdop' meant something similar to cup, and the question is really just one of spacing. I discovered actually 'dop' means something like 'shell'. So APPARENTLY eierdop all as one word means egg cup, but eier dop as two separate words means egg shell?
- tibbydudeza 4y ago
  "eierdop" for me means the shell of the egg (Afr) ???.
- t0mas88 4y ago
  Dutch has less spaces. Generally if it's one object, it's one word. This causes a typical spelling mistake native Dutch speakers make in English: we combine words into one that should be two words. In Dutch "eier dop" does not exist. Could be "ei dop" and then you're taking about two separate objects, an egg and a cap (as in bottle cap). And that's also immediately an example, I tried to type that as bottlecap and my phone corrected it with an extra space ;-)
stevenjgarner 4y ago
Thanks so much for sharing. Refreshes my own experience. I was so clear what I wanted to do with my life - so many goals and aspirations. Then I remember holding my first-borne in one hand. Less than an hour old she peered up to me almost inquisitively as if to say "I hope you got this?" At that moment I realized what my life was about. Can barely remember those goals and aspirations now. As they say in Spanish, my first borne now has 42 years. Brought 6 kids and 5 grandchildren into this world. As I age I am reminded by the same process you talk about but in reverse now. Watch the Robin Williams movie "Awakenings" [1]. Really portrays it. We come into this world absolutely unable to do much of anything. We fade in. Then ... trust me ... we fade out. But in between, if we're really lucky we get to shine. I wonder if a neural network will ever do that? [1] https://www.imdb.com/title/tt0099077/ https://www.imdb.com/title/tt0099077/

icare_1er 4y ago

Observing how children need a fairly limited amount of "learning examples" or "training sets" to learn that a cat is a cat, for instance, remains one of the most solid argument convincing me that AI fundamentally is not reproducing what actually happens in human brains.

dachryn 4y ago
actually it's closer than you might think. If you consider the eye and visual cortex to be the first layers of a deep learning network, we are actually getting a relatively similar system. However it has limitations further down the line in information representation stuff. The eye is genetically programmed to detect lines, hard edges, colours, and more complex patterns. This is exactly the same as the first layers of a neural net. However we then humanly assemble stuff like cat = eyes+fur+pointy+cute+small+meowing. A neural net has no way yet to represent information at that level, but we are getting closer there than going beyond dumb numbers. A neural net has implicitly the concept, but is not yet doing the 3d representation inside itself, which makes it harder to define something like 'ears' and 'tail'
- tibbydudeza 4y ago
  Yep recognizing a cat from a hamster - is it just pixels or broken down into higher level concepts like head, body, legs and tail.

tominous 4y ago

In animals some things are definitely hardwired. I had a pet rabbit and made the mistake of watching a nature documentary about snakes when she was in the room. The mere sound of a rattle snake on TV caused the rabbit to stamp a few times, spray urine, and run away to her hutch (skidding on the floor as the left). I'm sure humans have similar instincts. For example, the tendency of young kids to avoid plants until they are shown that they are edible - possibly leading to the general preference of artificial foods over anything that might be found in a garden. [1] I've also noticed that toddlers don't gravitate towards knives the same way they do everything else. I certainly wouldn't rely on this behaviour, but I've noticed it in my own children and others. [1] https://www.theatlantic.com/health/archive/2013/12/why-babies-fear-plants/281954/ https://www.theatlantic.com/health/archive/2013/12/why-babie...