4 ms·
But can't you typically do the activation as a separate step? Where you first incorporate all of the inputs, then decide activation?
by taeric 8y ago
But can't you typically do the activation as a separate step? Where you first incorporate all of the inputs, then decide activation?
- abecedarius 8y agoI'm not sure quite what you mean, but you can do the linear part of one layer all together, then the nonlinear part, etc.: for weight,bias in layers: a = sigmoid(np.dot(weight, a) + bias)
- taeric 8y agoSo my question is how large is the linear part? Granted, I was hoping it was big enough to bring these algorithms into consideration. Looks like I'm right in that they are larger than 2x2. However, they are still not 8kx8k. :(
- abecedarius 8y agoThey can be pretty big! I don't know what's normal these days, though.