4 ms·
> Feature engineering is a mechanism of creating new levels of abstraction in knowledge representation because each (non-trivial) feature extract and makes expl
by ericand 8y ago
> Feature engineering is a mechanism of creating new levels of abstraction in knowledge representation because each (non-trivial) feature extract and makes explicit some piece of knowledge hidden in the data. It is almost precisely what deep learning is intended for. In this sense, feature engineering does what hidden layers of a neural network do or what the convolutional layer of a neural network does
Very intriguing and thoughtful statement. I hadn't ever thought of it that way.
- tvladeck 8y agoThis is essentially repeating your quote, but an "aha moment" clicked for me when I read that what successful neural networks are basically doing is such good feature learning that the problem can be solved by a simple linear model in the end. E.g. if you have an N-layer neural network, N-1 layers are doing feature learning, and the Nth layer is a simple {logistic, multinomial/softmax, gaussian, poisson, ...} model
- halflings 8y agoRelated:"the kernel trick" [1]. "The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary." (what powers Support Vector Machines, the neural networks of the 90s, and still alive and kicking today) [1] https://en.wikipedia.org/wiki/Kernel_method#Mathematics:_the_kernel_trick https://en.wikipedia.org/wiki/Kernel_method#Mathematics:_the...
- jacquesm 8y agoA quick - and not 100% correct but it will do - way of looking at deep learning is to think of it as automated feature detection and weighing.
- brylie 8y agoHave no worry. Little, if anything, is 100% correct. We sometimes get hung up on correcting and contradicting people, often missing a deeper truth. It takes skill to find the grain of truth and build on it :-)