4 ms·
How would one extract an algebraic function from the network’s weights? Why would the function the network approximates necessarily be algebraic?
by ludwigschubert 7y ago
How would one extract an algebraic function from the network’s weights? Why would the function the network approximates necessarily be algebraic?
- Plough_Jogger 7y agoThe function would simply be a chain of multiplications and (typically) non-linear transformations. For classification problems, there is often a final non-linear transformation, like a softmax. Example with ReLu activation: Output_i = max((input_i * weight_i),0)
- jawarner 7y agoI don't understand. The example you gave is not an algebraic function.
- Plough_Jogger 7y agoYou are correct. It is possible to construct a linear neural network, where an algebraic function could be extracted, but in practice, almost all networks use non-linear activation functions. In the case of a linear network, the function would be a dot product between the input and the weights: 𝑥1𝑤1+𝑥2𝑤2+𝑥3𝑤3 ... for all inputs (xi) and weights (wi)