If you want a non-linear function that can be produced optically, consider an evanescent wave[1], as used in TIRF microscopy. Whether this is suitable in practi

If you want a non-linear function that can be produced optically, consider an evanescent wave[1], as used in TIRF microscopy. Whether this is suitable in practise as an activation function I couldn’t say, though the curve does to my eye look like something that could be used in place of relu. [1] https://en.m.wikipedia.org/wiki/Evanescent_field https://en.m.wikipedia.org/wiki/Evanescent_field