4 ms·

The video says that it's really a logistic curve. Which is also the activation function (sigmoid) in a few machine learning articles that I've been reading. h

The video says that it's really a logistic curve. Which is also the activation function (sigmoid) in a few machine learning articles that I've been reading. https://en.wikipedia.org/wiki/Sigmoid_function https://en.wikipedia.org/wiki/Sigmoid_function https://en.wikipedia.org/wiki/Logistic_function https://en.wikipedia.org/wiki/Logistic_function Can anyone recommend anything to read to add more clarity? Why does this appear in deep learning too?

andrewprock 7y ago
It's used in algorithms to provide a continuously differentiable function, which is much easier analyze than a step function.
fsh 7y ago
For neural networks to work, the activation function has to be nonlinear and monotonic. In order to be able to apply the chain rule for training the weights by backpropagation, the function also has to be differentiable. The logistic function is one obvious choice, but other functions such as tanh, arctan or relu are popular as well.
bno1 7y ago
In machine learning it's used as an activation function. Activation functions are used to add non-linearity to the model. I think sigmoid functions are inspired from neurons, but I'm not sure exactly how. Nowadays, ReLU activations are used more often than sigmoidals because they are quicker to compute.
- bno1 7y ago
  >I think sigmoid functions are inspired from neurons I mean the use of sigmoids as activations in NNs
- cmarschner 7y ago
  ReLu are mainly used because in practice they show better results than sigmoids in many areas. The fact that they are also faster to compute is just a nice side effect.
Florin_Andrei 7y ago
> The video says that it's really a logistic curve. I mean, of course it's not a true, complete exponential, or else it would take over the universe in 2.3 years or something. All exponentials run out of something eventually. The question is, how soon does that happen.

andrewprock 7y ago

It's used in algorithms to provide a continuously differentiable function, which is much easier analyze than a step function.

fsh 7y ago

For neural networks to work, the activation function has to be nonlinear and monotonic. In order to be able to apply the chain rule for training the weights by backpropagation, the function also has to be differentiable. The logistic function is one obvious choice, but other functions such as tanh, arctan or relu are popular as well.

bno1 7y ago

In machine learning it's used as an activation function. Activation functions are used to add non-linearity to the model. I think sigmoid functions are inspired from neurons, but I'm not sure exactly how. Nowadays, ReLU activations are used more often than sigmoidals because they are quicker to compute.

bno1 7y ago
>I think sigmoid functions are inspired from neurons I mean the use of sigmoids as activations in NNs
cmarschner 7y ago
ReLu are mainly used because in practice they show better results than sigmoids in many areas. The fact that they are also faster to compute is just a nice side effect.

Florin_Andrei 7y ago

> The video says that it's really a logistic curve. I mean, of course it's not a true, complete exponential, or else it would take over the universe in 2.3 years or something. All exponentials run out of something eventually. The question is, how soon does that happen.