3 ms·
A relatively small lookup table for the sigmoid function can also work well. Here are the various sigmoid approximations that Theano (a library used for deep le
by lightcatcher 13y ago
A relatively small lookup table for the sigmoid function can also work well. Here are the various sigmoid approximations that Theano (a library used for deep learning research among other things) offers: http://deeplearning.net/software/theano/library/tensor/nnet/nnet.html#tensor.nnet.sigmoid http://deeplearning.net/software/theano/library/tensor/nnet/...
- gamegoblin 13y agoI usually use an array with a few thousand entries. In C this gives me a 2.5x speedup over the exact function with no important decrease in accuracy.
- dfrodriguez143 13y agoVery interesting and simple improvement. Definitely will take a look at that.
- mistercow 13y agoI wonder though if you might actually do better overall with a smaller lookup table and interpolation (or even just a polynomial approximation, which can be evaluated without branching), since large lookup tables can lead to bad cache behavior.