3 ms·
The first few layer of CNNs are basically learned wavelets. I've had the thought to use basis functions, based on wavelets and/os Legendre polynomials, to perfo
by kortex 5y ago
The first few layer of CNNs are basically learned wavelets. I've had the thought to use basis functions, based on wavelets and/os Legendre polynomials, to perform the first few layers of image processing (more regular and don't need backprop). Haven't been in that space in a while though and don't have much time to mess with it.
CNNs have gotten so good though it seems a little moot.
- SubiculumCode 5y agoNot in the field, but that seems to me a potentially interesting intuition.
- tubby12345 5y agoit's almost universally known that the whole point of NNs is that they've obviated the need for feature engineering (for some tasks)
- kortex 5y agoRight, exactly. This is manually engineering the lower features, which CNNs just learn. The chief advantage of this technique is a) certain regular wavelets have performance benefits over convolutions, especially when talking low level/FPGA/ASIC space b) being not learned can be beneficial, as these is nothing to overfit. I can see it being handy for embedded/rasppi like applications. In particular, you can festoon a crude face detector with a dozen Haar filters. End to end training is just soooo convenient though.