3 ms·

Unsupervised learning is the dark matter of AI, as LeCun said. We cannot rely on labeled data, which is expensive and scarce. The first company to figure out h

• ninjin 8y ago
  > Models requiring labeled data or only working with toy data (even if unlabeled) will soon become irrelevant. Care to present evidence for that conclusion or better define the value of “soon”? Most work I am aware of in my own field still requires copious amounts of training data – even with the progress that you reference – and generalisation is still a big challenge.
• Tenoke 8y ago
  >Unsupervised learning is the dark matter of AI, as LeCun said. We cannot rely on labeled data, which is expensive and scarce. This sounds very extreme to me. Sure, unsupervised learning is already bearing fruit (most obviously in NLP) but there are domains where labeled data is unlikely to disappear soon - in many cases semi-supervised learning is a more reasonable goal which will reduce the need for labeling (or labeling-related tasks). >The first company to figure out how to leverage the massive unlabeled data that is available will win the AI arms race. This is an even stronger claim. Why are you so sure only one actor will be able to leverage it? Yes, it will be a huge boon but claiming that the first to crack it will 'win' seems very farfetched.
• bjoernbu 8y ago
  > The first company to figure out how to leverage the massive unlabeled data that is available will win the AI arms race. I would expect that this will remain a per-task question for a very long time, maybe even forever. For example, I was amazed at how elegant https://openai.com/blog/unsupervised-sentiment-neuron/ https://openai.com/blog/unsupervised-sentiment-neuron/ is for an unsupervised model for sentiment analysis. I think there are numerous similar application in text understanding and I would not be suprised to eventually seen many sota systems learn in an unsupervised fashion. Still, there remain many other tasks (in general, but also for text understanding) where this and similar strategies do not seem applicable to me -- or at least where I cannot directly imagine how they should be. I would not be suprised if we end up with many, many successful strategies to apply unsupervised learning -- many completely different, and thousands of problems still being used with the help of supervised learning. Not with a "company to figure out" a general case
• barkingcat 8y ago
  unlabelled data is the death of humans. Let AI train on unlabelled data, and it will soon learn that humans are a net drain and need to be killed off.
• mijail 8y ago
  What's the current sentiment on synthetic data? It may be a self serving question since I work at synthetic data company but I'm curious for fear of being stuck in an echo chamber.
  • paol 8y ago
    It's been very productive for us (in machine vision).
  • ska 8y ago
    It's no magic bullet. I think it can help in particular instances but see a lot of people chasing their own tails.
• albertzeyer 8y ago
  > The first company to figure out how to leverage the massive unlabeled data that is available will win the AI arms race. Well, in many tasks, this is already solved. E.g. speech recognition or translation. Or just think about language modeling. The main problem is mostly just computing time and/or computing resources. We have enough supervised training data to train our model for weeks. We basically have an almost infinite amount of unsupervised data. So yes, we could in theory make the model bigger and then train it for months or years. But that's not really practical currently.