Yes, capacity is intimately tied to VC dimension; in particular, VC dimension is one way to measure capacity. See the Wikipedia article for more information: ht

Yes, capacity is intimately tied to VC dimension; in particular, VC dimension is one way to measure capacity. See the Wikipedia article for more information: http://en.wikipedia.org/wiki/Vc_dimension http://en.wikipedia.org/wiki/Vc_dimension I'm not an expert on deep learning (although I generally understand how they work on vision problems), so I'm not sure if you can precisely measure the capacity of deep networks. Informally, the primary number that seems to matter is the number of parameters in the network that have to be learned. This paper quotes that at "more than 120 million". SVMs, in contrast, typically work with feature dimensionalities (i.e., # of parameters) that are on the order of 1,000 - 100,000. You can't directly compare these numbers because there are various non-linearities involved, but this deep learning network is definitely much higher capacity than an SVM would be with normal feature dimensionalities.