4 ms·

Yes in the most general sense, a "data set" could be anything, but I'm talking about the highly curated and labeled data sets that are used to train contemporar

Yes in the most general sense, a "data set" could be anything, but I'm talking about the highly curated and labeled data sets that are used to train contemporary ML systems. You can feed a million images that are labeled "cat" or "no cat" to one of these systems and it can achieve a human-level proficiency at identifying cats in images. But, it won't be able to do anything other than identify cats, it's far too narrow to be considered intelligent in any way. If you can feed a series of timestamped photos to a system, basically an unlabeled arbitrary video stream, and you can demonstrate that it formed some notion of what a cat is, that would be very interesting indeed.

rjtavares 9y ago
Have you ever seen a child learn how to speak? He also needs loads of "labelled" data to initially learn about concepts.
- resu_nimda 9y ago
  The data is not labelled in the same way. The child has still acquired their knowledge solely from sensory experience. How do they know to apply the spoken label "mom" to the recurring pattern in their visual data stream? With modern data sets and labelling, so much of the problem domain is deeply hard-coded into the system. It doesn't have to learn what letters and words mean and how to identify them in a totally arbitrary* visual or audio stream. It just gets a relatively minute amount of structured data that it has an embedded understanding of what to do with. Sure there are things like OCR and speech-to-text, but I don't think you could just run your streams through those, there's just so much subtle information loss. In order to make AI that really has a chance of reaching what humans would call intelligence, I firmly believe it has to make meaning out of some kind of raw sensory experience analogous to ours. *Ok, not totally arbitrary, a human would not learn language from a video of a forest, and that's where the "labelling" comes in, from observing other people using language, but a child can still learn any language from just that, and the labels are often vague, inaccurate, contradictory, complex, abstract, etc. There's no master training set with the right answers, you have to decide for yourself. And humans were also capable of bootstrapping language from nothing. I just don't see modern supervised learning systems ever doing things like that.
  - IanCal 9y ago
    Actually you can learn a lot of stuff unsupervised, and then just use supervised learning to extract the information from within the net. So you can just throw huge amounts of letters at a deep net then later look at the activation at the top and say "Ok, so when neurons X, Y and Z are on it's when it's looking at a 1". I'm pretty sure you can just connect two at the top and have it say the number it sees. > In order to make AI that really has a chance of reaching what humans would call intelligence, I firmly believe it has to make meaning out of some kind of raw sensory experience analogous to ours. And why must it be able to do this from scratch? Why must it be hampered with the same limitations we have?
  - nl 9y ago
    Unlabelled datasets are valuable in themselves. Self-Taught Learning: Transfer Learning from Unlabeled Data http://www.andrewng.org/portfolio/self-taught-learning-transfer-learning-from-unlabeled-data/ http://www.andrewng.org/portfolio/self-taught-learning-trans...
- mongodude 9y ago
  I would disagree on this point, humans unlike current AI systems can learn from one or two data points, especially at easier tasks like identifying cats. Current AI algorithms need huge labeled data sets for solving narrow problems so one needs to build more generalization ability to our current AI systems.
  - Tenoke 9y ago
    That's because we are already VERY pretrained on very similar tasks. You should look into transfer learning and one shot learning for examples of techniques that in practice do something very similar.
    - mongodude 9y ago
      Yes, I have heard of transfer learning and used them in practice. Very powerful but still primitive. One-shot learning techniques are still to mature but I agree with you that technologies like these will reduce our reliance on datasets and make the AI algorithms learn in more human like way.
- korzun 9y ago
  Are you comparing a child learning to speak with a set of conditional statements?

rjtavares 9y ago

Have you ever seen a child learn how to speak? He also needs loads of "labelled" data to initially learn about concepts.

resu_nimda 9y ago
The data is not labelled in the same way. The child has still acquired their knowledge solely from sensory experience. How do they know to apply the spoken label "mom" to the recurring pattern in their visual data stream? With modern data sets and labelling, so much of the problem domain is deeply hard-coded into the system. It doesn't have to learn what letters and words mean and how to identify them in a totally arbitrary* visual or audio stream. It just gets a relatively minute amount of structured data that it has an embedded understanding of what to do with. Sure there are things like OCR and speech-to-text, but I don't think you could just run your streams through those, there's just so much subtle information loss. In order to make AI that really has a chance of reaching what humans would call intelligence, I firmly believe it has to make meaning out of some kind of raw sensory experience analogous to ours. *Ok, not totally arbitrary, a human would not learn language from a video of a forest, and that's where the "labelling" comes in, from observing other people using language, but a child can still learn any language from just that, and the labels are often vague, inaccurate, contradictory, complex, abstract, etc. There's no master training set with the right answers, you have to decide for yourself. And humans were also capable of bootstrapping language from nothing. I just don't see modern supervised learning systems ever doing things like that.
- IanCal 9y ago
  Actually you can learn a lot of stuff unsupervised, and then just use supervised learning to extract the information from within the net. So you can just throw huge amounts of letters at a deep net then later look at the activation at the top and say "Ok, so when neurons X, Y and Z are on it's when it's looking at a 1". I'm pretty sure you can just connect two at the top and have it say the number it sees. > In order to make AI that really has a chance of reaching what humans would call intelligence, I firmly believe it has to make meaning out of some kind of raw sensory experience analogous to ours. And why must it be able to do this from scratch? Why must it be hampered with the same limitations we have?
- nl 9y ago
  Unlabelled datasets are valuable in themselves. Self-Taught Learning: Transfer Learning from Unlabeled Data http://www.andrewng.org/portfolio/self-taught-learning-transfer-learning-from-unlabeled-data/ http://www.andrewng.org/portfolio/self-taught-learning-trans...
mongodude 9y ago
I would disagree on this point, humans unlike current AI systems can learn from one or two data points, especially at easier tasks like identifying cats. Current AI algorithms need huge labeled data sets for solving narrow problems so one needs to build more generalization ability to our current AI systems.
- Tenoke 9y ago
  That's because we are already VERY pretrained on very similar tasks. You should look into transfer learning and one shot learning for examples of techniques that in practice do something very similar.
  - mongodude 9y ago
    Yes, I have heard of transfer learning and used them in practice. Very powerful but still primitive. One-shot learning techniques are still to mature but I agree with you that technologies like these will reduce our reliance on datasets and make the AI algorithms learn in more human like way.
korzun 9y ago
Are you comparing a child learning to speak with a set of conditional statements?