3 ms·

Can you clarify what you mean by 'non-differentiable outcomes'?

• ArnoVW 5y ago
  My guess: a function that you can evaluate but that you can not differentiate. Differentiate being "know the slope". Differentiable functions are great because you can run gradient descent on them in a very optimized way. Example: if your objective is to have a very high value, search in the direction that has a mounting slope. Though maybe I'm missing something 'cos it seems to me you can run gradient descend on non-differentiable functions. It just requires more evaluations.
  • LeegleechN 5y ago
    Neural network models have millions of parameters, which means you are trying to optimize in spaces with millions of dimensions. If you can't use differentiation you get hit really hard with the curse of dimensionality.
  • jeeeb 5y ago
    > Though maybe I'm missing something 'cos it seems to me you can run gradient descend on non-differentiable functions Isn't that exactly the point though? If you don't have an analytical solution for the gradient of the loss (reward) wrt the parameters - yes - you could brute force a numerical solution but as the number of parameters grows that quickly becomes infeasible. Approaches such as RL and GA provide a more intelligent way to search the parameter space.