3 ms·
You might want to just make the first layer a DCT of the input vector which will split them into decorrelated components.
by duped 4y ago
You might want to just make the first layer a DCT of the input vector which will split them into decorrelated components.
- wardedVibe 4y agoI see we are re-deriving conv nets
- duped 4y agoNot really, the first principle of not-too-bad ANNs is to use some kind of preprocessed, hopefully decorrellated feature vectors as the inputs. Conceptually that is way easier to implement with intuition and know-how over blindly training an arbitrary structure to do what you want. Ideally the training just derives non-intuitive relationships between these decorrelated input features to give you something useful.