28 ms·
Mish is generally superior to RadamW https://lessw.medium.com/meet-mish-new-state-of-the-art-ai-activation-function-the-successor-to-relu-846a6d93471f https://l
by The_rationalist 6y ago
Mish is generally superior to RadamW https://lessw.medium.com/meet-mish-new-state-of-the-art-ai-activation-function-the-successor-to-relu-846a6d93471f https://lessw.medium.com/meet-mish-new-state-of-the-art-ai-a...
- dron57 6y agoThat's cool, but Mish is an activation function while SGD and AdamW are optimizers. Apples and oranges.