3 ms·
Without a non-linear activation function, it wouldn't be an ANN, because multiple linear layers are equivalent to a single layer applying the composition of the
by ovi256 8y ago
Without a non-linear activation function, it wouldn't be an ANN, because multiple linear layers are equivalent to a single layer applying the composition of their transforms.
The article gives no clue about this part. I have no idea if the optical domain can compute non-linear functions.
- jf- 8y agoIf you want a non-linear function that can be produced optically, consider an evanescent wave[1], as used in TIRF microscopy. Whether this is suitable in practise as an activation function I couldn’t say, though the curve does to my eye look like something that could be used in place of relu. [1] https://en.m.wikipedia.org/wiki/Evanescent_field https://en.m.wikipedia.org/wiki/Evanescent_field