3 ms·
Right. But without an activation function, the matrix multiplications would just compose into a single matrix for the whole net. I guess the floating-point roun
by abecedarius 8y ago
Right. But without an activation function, the matrix multiplications would just compose into a single matrix for the whole net. I guess the floating-point rounding brought up above is supposed to serve as an activation function for this purpose, though I haven't looked into it.
- taeric 8y agoBut can't you typically do the activation as a separate step? Where you first incorporate all of the inputs, then decide activation?
- abecedarius 8y agoI'm not sure quite what you mean, but you can do the linear part of one layer all together, then the nonlinear part, etc.: for weight,bias in layers: a = sigmoid(np.dot(weight, a) + bias)
- taeric 8y agoSo my question is how large is the linear part? Granted, I was hoping it was big enough to bring these algorithms into consideration. Looks like I'm right in that they are larger than 2x2. However, they are still not 8kx8k. :(
- abecedarius 8y agoThey can be pretty big! I don't know what's normal these days, though.