4 ms·

At the risk of nitpicking, I'm going to specifically address your statement of "It feels like an ostrich" To my mind, that's EXACTLY how humans do it. A baby

At the risk of nitpicking, I'm going to specifically address your statement of "It feels like an ostrich" To my mind, that's EXACTLY how humans do it. A baby is not instructed, "it has 2 legs, feathers, and is tall, therefore it is an ostrich", there's quite a bit more "LOOK OSTRICH /present input of ostrich/" prior to the point of being able to generate any justification. As sister posts have pointed out, I tend to believe that the ability to justify the classification is another learned skillset that comes later developmentally, the ability for higher primates (and babies) to perform classification without the (as far as I know) ability to do the higher order reasoning lends itself to this theory.

andrewljohnson 11y ago
I find how my two-year old classifies birds to be amusing: 1) He called all birds ducks for the longest time, presumably based on being exposed to a bath duck, and this weird Youtube video about ducks swimming in the water. 2) Then he called some of them bird, based on being exposed to a bath toy bird. 3) Then he split them up a bit more... parrot, chicken, rooster, etc. He started seeing them in books and real life. 4) At no point could he explain the difference. Still can't. 5) And he definitely thinks the only difference between a rooster and a chicken is colorful feathers. 6) And he's not quite sure about ducks and geese, but usually gets it right.
- swombat 11y ago
  I guess he still needs to develop the advanced bullshitting (sorry, I meant "rationalisation") skills required to say, "Well, I think it's a duck, because it looks like my internal picture of what ducks are supposed to look like, dear sir!"
white-flame 11y ago
I agree that the initial human classifier is the "general feel". My distinguishing point is in the retraining. We can form conceptual constructions of what we see to resolve training correction, and once that is bootstrapped, carry the formed concepts and the meta-cognitive capability of concept creation over into new contexts. If we've only seen automobiles, then all of a sudden are introduced to trains, 18-wheelers, motorcycles, etc, we are challenged to conceptually organize and discover the differences between these classifications. There's still a fair amount of "feel" in weird overlap cases, but the conceptual distinction is created after the fact. There is no mechanism for that in NNs.
pokkanome 11y ago
Using the word "feels" (or "thinks") is a too much of a personification of a neural network. You cannot forget that under the hood, it is cranking through a bunch of math functions. You always get the same output from the same input. That function is not going to change on its own, and it is not going to suddenly gain new abilities to express itself in words. It is not a human, so there is no point in assuming it will behave or grow like a human.
bobcostas55 11y ago
I think one of the major differences is that humans use multiple independent algorithms when it comes to deciding if there's an ostrich in the picture. And most of these are far more sophisticated than the NNs. So in practice the way we solve classification problems is fundamentally different, even though it may seem/feel somewhat similar. Most prominently there's object recognition (telling things apart from the background, recognizing parts of objects, etc.), size/distance determination (which uses over a dozen separate algorithms: https://en.wikipedia.org/wiki/Depth_perception https://en.wikipedia.org/wiki/Depth_perception). We can also reconstruct a 3d model from a 2d image and compare that to our idea of what ostriches look like in 3 dimensions. So even if it is done unconsciously, we recognize an ostrich because (among other things) it's the size of an ostrich. The NN has no idea what size is, let alone the size of an ostrich relative to other things. I think the next big breakthrough in machine learning is in ensembles containing NNs trained for completely different things, but acting in a complementary fashion.

andrewljohnson 11y ago

I find how my two-year old classifies birds to be amusing: 1) He called all birds ducks for the longest time, presumably based on being exposed to a bath duck, and this weird Youtube video about ducks swimming in the water. 2) Then he called some of them bird, based on being exposed to a bath toy bird. 3) Then he split them up a bit more... parrot, chicken, rooster, etc. He started seeing them in books and real life. 4) At no point could he explain the difference. Still can't. 5) And he definitely thinks the only difference between a rooster and a chicken is colorful feathers. 6) And he's not quite sure about ducks and geese, but usually gets it right.

swombat 11y ago
I guess he still needs to develop the advanced bullshitting (sorry, I meant "rationalisation") skills required to say, "Well, I think it's a duck, because it looks like my internal picture of what ducks are supposed to look like, dear sir!"

white-flame 11y ago

I agree that the initial human classifier is the "general feel". My distinguishing point is in the retraining. We can form conceptual constructions of what we see to resolve training correction, and once that is bootstrapped, carry the formed concepts and the meta-cognitive capability of concept creation over into new contexts. If we've only seen automobiles, then all of a sudden are introduced to trains, 18-wheelers, motorcycles, etc, we are challenged to conceptually organize and discover the differences between these classifications. There's still a fair amount of "feel" in weird overlap cases, but the conceptual distinction is created after the fact. There is no mechanism for that in NNs.

pokkanome 11y ago

Using the word "feels" (or "thinks") is a too much of a personification of a neural network. You cannot forget that under the hood, it is cranking through a bunch of math functions. You always get the same output from the same input. That function is not going to change on its own, and it is not going to suddenly gain new abilities to express itself in words. It is not a human, so there is no point in assuming it will behave or grow like a human.

bobcostas55 11y ago

I think one of the major differences is that humans use multiple independent algorithms when it comes to deciding if there's an ostrich in the picture. And most of these are far more sophisticated than the NNs. So in practice the way we solve classification problems is fundamentally different, even though it may seem/feel somewhat similar. Most prominently there's object recognition (telling things apart from the background, recognizing parts of objects, etc.), size/distance determination (which uses over a dozen separate algorithms: https://en.wikipedia.org/wiki/Depth_perception https://en.wikipedia.org/wiki/Depth_perception). We can also reconstruct a 3d model from a 2d image and compare that to our idea of what ostriches look like in 3 dimensions. So even if it is done unconsciously, we recognize an ostrich because (among other things) it's the size of an ostrich. The NN has no idea what size is, let alone the size of an ostrich relative to other things. I think the next big breakthrough in machine learning is in ensembles containing NNs trained for completely different things, but acting in a complementary fashion.