> I thought the main selling point of deep learning that it finds non-linear connections in the data. Agreed. Yet there are unknown feature spaces in which the

> I thought the main selling point of deep learning that it finds non-linear connections in the data. Agreed. Yet there are unknown feature spaces in which the deep nets separate the data linearly; the mapping learnt by the deep nets from the data to this space is likely highly nonlinear. For example, a binary logistic regression can be thought as mapping data to a 1D space ([0, 1]), and separates the two classes at say 1/2 linearly.