3 ms·
A non-linearity that works much better than ReLUs
- seesawtron 6y agoCould one benefit from it by replacing ReLU with sine activation functions in all DNN (conv.) architectures? In this paper they only show that it works better in MLPs based (non convolutional) deep neural architectures, if I understood it correctly? Traditional ReLU-CNNs do perform better than Sine-MLPs in one of the tasks(Supp. Table 4).
- ai_ja_nai 6y agoSounds very interesting, although it has huge prerequites on the kind of problems they are addressing