3 ms·
It's in TensorFlow: https://www.tensorflow.org/api_docs/python/tf/nn/relu6 https://www.tensorflow.org/api_docs/python/tf/nn/relu6 And even points to a paper: h
by mcintyre1994 9y ago
It's in TensorFlow: https://www.tensorflow.org/api_docs/python/tf/nn/relu6 https://www.tensorflow.org/api_docs/python/tf/nn/relu6
And even points to a paper: http://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf http://www.cs.utoronto.ca/~kriz/conv-cifar10-aug2010.pdf
I'm not sure why they didn't make it a variable maximum though, or what the advantage is over multiplying a sigmoid by 6.
- clarle 9y agoThere's an explanation here: https://stackoverflow.com/questions/47220595/why-the-6-in-relu6 https://stackoverflow.com/questions/47220595/why-the-6-in-re... It's an arbitrary value, but 6 fits best into 8 bits.