4 ms·

Anyone have information about understanding how inputs affect outputs in a trained ANN? Specifically to better understand the system, which inputs have the most

Anyone have information about understanding how inputs affect outputs in a trained ANN? Specifically to better understand the system, which inputs have the most impact and how they interact with out inputs.

TrainedMonkey 13y ago
I would check Neural Network class on Coursera. Subject is so complex that you need to put time to get anywhere.
- chmike 13y ago
  What is so complex ? Neurons compute a sum of its input. Neurons are connected by synapse. A synapse has a oefficient that weights the input. If the synapse coefficitent is the real value c and the value of the input neuron is i, then The sum will be computed with c*i. One can represent this by vector and matrix multiplication. In the most simple model, this sum is the output. In more sofisticated models the result of the sum is modified by a function with a sigmoid shape. One of these sigmoid function is infinitly derivable. This allows to fit any continuous function with a sum of these sigmoids. Different strategies have been suggested to do the fit. But all these ANN don't explain the biological neural networks found in our cortex.
  - TrainedMonkey 13y ago
    It is complex as soon as you try to do anything but simplest things with Neural network. What I really liked about Coursera's NN class is theory of how RBM's can be used to model associative memory.
    - chmike 13y ago
      Ah yes. I agree. Doing things with these simple neurons is complex. On the other hand a 3 year old kid can sing and dance at the same time. How can it be so complex? ;) It is a fascinating problem which is still lacking an answer.
tlarkworthy 13y ago
Its called the Jacobian. The differential of the outputs by the inputs. Tells you how each input affects the output vector at a specific location in input space. You can calculate it manually. Use inbuilt Matlab functions or finite difference it.
- foobarqux 13y ago
  So what is all the talk about how ANNs yield solutions that don't have a straightforward interpretation unlike regression?
  - tlarkworthy 13y ago
    Well the jacobian is a numerical matrix unique at each point in the input space. So its kinda hard to visualize the changing Jacobian matrix over the input space. People do interpret ANNs. Normally by visualising the weight matrices on the input layer, which have a 1-1 mapping to input vector attributes (so you can label them). It gets a bit hard on subsequent layers though. decision trees are an order of magnitude easier to interpret though, they are very compact and consist of a number of binary decision in the input space. For a given classification you only need to look at log(d) decisions to work out how it got to its conclusion. Its not usually that hard to how it arrived at the tree from training. The relationship between the Jacobian and the training is fairly convoluted, given back propagation and cross validation. PS so back-prorogation uses weights differentiated against outputs. Jacobian is input differentiated against outputs. You can do smart things with Jacobian aware neural architectures. See "Forward models: Supervised learning with a distal teacher" which trains two networks in parallel, one a forward model, one an inverse model, and uses the Jacobian to circumnavigate the problematic non-convexity of the inverse model.
    - foobarqux 13y ago
      > Well the jacobian is a numerical matrix unique at each point in the input space. So its kinda hard to visualize the changing Jacobian matrix over the input space. Yeah it doesn't seem like it would be easy to gain much insight since the partial derivatives are a function of the (other) inputs. > People do interpret ANNs. Normally by visualising the weight matrices on the input layer, which have a 1-1 mapping to input vector attributes (so you can label them). What do the first-layer weights tell me?

TrainedMonkey 13y ago

I would check Neural Network class on Coursera. Subject is so complex that you need to put time to get anywhere.

chmike 13y ago
What is so complex ? Neurons compute a sum of its input. Neurons are connected by synapse. A synapse has a oefficient that weights the input. If the synapse coefficitent is the real value c and the value of the input neuron is i, then The sum will be computed with c*i. One can represent this by vector and matrix multiplication. In the most simple model, this sum is the output. In more sofisticated models the result of the sum is modified by a function with a sigmoid shape. One of these sigmoid function is infinitly derivable. This allows to fit any continuous function with a sum of these sigmoids. Different strategies have been suggested to do the fit. But all these ANN don't explain the biological neural networks found in our cortex.
- TrainedMonkey 13y ago
  It is complex as soon as you try to do anything but simplest things with Neural network. What I really liked about Coursera's NN class is theory of how RBM's can be used to model associative memory.
  - chmike 13y ago
    Ah yes. I agree. Doing things with these simple neurons is complex. On the other hand a 3 year old kid can sing and dance at the same time. How can it be so complex? ;) It is a fascinating problem which is still lacking an answer.

tlarkworthy 13y ago

Its called the Jacobian. The differential of the outputs by the inputs. Tells you how each input affects the output vector at a specific location in input space. You can calculate it manually. Use inbuilt Matlab functions or finite difference it.

foobarqux 13y ago
So what is all the talk about how ANNs yield solutions that don't have a straightforward interpretation unlike regression?
- tlarkworthy 13y ago
  Well the jacobian is a numerical matrix unique at each point in the input space. So its kinda hard to visualize the changing Jacobian matrix over the input space. People do interpret ANNs. Normally by visualising the weight matrices on the input layer, which have a 1-1 mapping to input vector attributes (so you can label them). It gets a bit hard on subsequent layers though. decision trees are an order of magnitude easier to interpret though, they are very compact and consist of a number of binary decision in the input space. For a given classification you only need to look at log(d) decisions to work out how it got to its conclusion. Its not usually that hard to how it arrived at the tree from training. The relationship between the Jacobian and the training is fairly convoluted, given back propagation and cross validation. PS so back-prorogation uses weights differentiated against outputs. Jacobian is input differentiated against outputs. You can do smart things with Jacobian aware neural architectures. See "Forward models: Supervised learning with a distal teacher" which trains two networks in parallel, one a forward model, one an inverse model, and uses the Jacobian to circumnavigate the problematic non-convexity of the inverse model.
  - foobarqux 13y ago
    > Well the jacobian is a numerical matrix unique at each point in the input space. So its kinda hard to visualize the changing Jacobian matrix over the input space. Yeah it doesn't seem like it would be easy to gain much insight since the partial derivatives are a function of the (other) inputs. > People do interpret ANNs. Normally by visualising the weight matrices on the input layer, which have a 1-1 mapping to input vector attributes (so you can label them). What do the first-layer weights tell me?