6 ms·

Are there any (unsupervise or min supervised) methods yet that learn on non repeating datasets? As humans we don’t really learn by rereading the same text 35 ti

Are there any (unsupervise or min supervised) methods yet that learn on non repeating datasets? As humans we don’t really learn by rereading the same text 35 times, or stear at the same picture continuously, and so why are all these algorithm research papers keep focusing on methods that are basically trying to fit a function over a static known and labeled data?

IanCal 8y ago
To be fair, we do spend a pretty major chunk of our lives looking at repetition to learn, particularly when younger. The term you may be looking for though is "one shot learning".
- ericd 8y ago
  Yeah, one shot learning in humans is built on top of enormous model representations of the world built up over many years, so I wouldn’t be surprised if we’re a bit premature in trying to pull it off in ML other than in very simple cases.
emcq 8y ago
Yes there are certainly datasets you can train on a dataset effectively without repetition; this is more common in areas like reinforcement learning where you are simulating every training point. While theoretically possible you have the same random seed to produce the same training point, in practice that wont happen without bugs but some training data will be similar. Augmentation can also ensure you dont see the exact training data more than one. However, this doesn't guarantee you will be able to train an amazing model. You are limited by properties like model capacity, label noise, poor feature representation, optimizer+learning rate, etc. In the limit empirically there will be some dataset size after which you get diminishing returns on improving model performance.
dahart 8y ago
> As humans we don’t really learn by rereading the same text 35 times, or stear at the same picture continuously Well we do learn language that way, by hearing the same sounds over and over until we understand words and associations. There are lots of other examples of needing to have repetition for learning to sink in as well, from music to math to writing.
krona 8y ago
Our genes have spent millions of years preconditioning us to expect the world to appear to us in a certain way. We then spend a lifetime refining an emergent cognitive model so that we can navigate it.

IanCal 8y ago

To be fair, we do spend a pretty major chunk of our lives looking at repetition to learn, particularly when younger. The term you may be looking for though is "one shot learning".

ericd 8y ago
Yeah, one shot learning in humans is built on top of enormous model representations of the world built up over many years, so I wouldn’t be surprised if we’re a bit premature in trying to pull it off in ML other than in very simple cases.

emcq 8y ago

Yes there are certainly datasets you can train on a dataset effectively without repetition; this is more common in areas like reinforcement learning where you are simulating every training point. While theoretically possible you have the same random seed to produce the same training point, in practice that wont happen without bugs but some training data will be similar. Augmentation can also ensure you dont see the exact training data more than one. However, this doesn't guarantee you will be able to train an amazing model. You are limited by properties like model capacity, label noise, poor feature representation, optimizer+learning rate, etc. In the limit empirically there will be some dataset size after which you get diminishing returns on improving model performance.

dahart 8y ago

> As humans we don’t really learn by rereading the same text 35 times, or stear at the same picture continuously Well we do learn language that way, by hearing the same sounds over and over until we understand words and associations. There are lots of other examples of needing to have repetition for learning to sink in as well, from music to math to writing.

krona 8y ago

Our genes have spent millions of years preconditioning us to expect the world to appear to us in a certain way. We then spend a lifetime refining an emergent cognitive model so that we can navigate it.