3 ms·

I'm not a CNN guy, but your question can't really make sense given my understanding of neural networks. In short, the output step of a NN has to include a class

I'm not a CNN guy, but your question can't really make sense given my understanding of neural networks. In short, the output step of a NN has to include a classifier. That is, there's a matrix of probabilities that the NN generates. The higher a given probability, the more likely the input is to match the category associated with that value. For example, a NN trained to distinguish "light" from "dark" may output a matrix with value [0.3, 0.7]. To train a CNN on "everything" means you have to have an arbitrarily large output matrix. Can you list every possible category of everything? Probably not. Even if you could take a swing at it, it's hard to get enough data (and time!) to train the net on each category. Small datasets result in overfitting of the training data, and poor overall performance. How big a data set would you need to properly train a sufficiently-sized CNN with an arbitrarily-sized classifier?

nl 11y ago
The question makes perfect sense. Word2Vec does a pretty good job of encoding the entire English language. Autocoders are a thing, and back in 2012 Google trained one against YouTube and it learnt to recognise cats. The state of the art has progressed a lot since then. See https://googleblog.blogspot.com.au/2012/06/using-large-scale-brain-simulations-for.html https://googleblog.blogspot.com.au/2012/06/using-large-scale...
pigscantfly 11y ago
I think I understand what you're getting at - that network architectures have a fixed size output - but you're incorrect in saying that the final layer must be a classifier. In general, you can optimize any differentiable function with gradient descent, and the output does not have to be a probability distribution. The original poster's question does make sense; he's asking what would happen if you trained the network on something like the ILSVRC dataset.

nl 11y ago

The question makes perfect sense. Word2Vec does a pretty good job of encoding the entire English language. Autocoders are a thing, and back in 2012 Google trained one against YouTube and it learnt to recognise cats. The state of the art has progressed a lot since then. See https://googleblog.blogspot.com.au/2012/06/using-large-scale-brain-simulations-for.html https://googleblog.blogspot.com.au/2012/06/using-large-scale...

pigscantfly 11y ago

I think I understand what you're getting at - that network architectures have a fixed size output - but you're incorrect in saying that the final layer must be a classifier. In general, you can optimize any differentiable function with gradient descent, and the output does not have to be a probability distribution. The original poster's question does make sense; he's asking what would happen if you trained the network on something like the ILSVRC dataset.