We should move this conversation to email (as I will check that more frequently and will be more likely to get back to it). See email in my profile. Active su

We should move this conversation to email (as I will check that more frequently and will be more likely to get back to it). See email in my profile. Active subspace comes from the uncertainty quantification community. If you assume all your parameters are Gaussian, then the sensitivity matrix is directly correlated to the probability density functions. I find it easier to think in terms of the sensitivity matrix, but useful to realize the sensitivity matrix to approximate (complex) probability distributions. My though was that if you were optimizing have a huge parameter space theta = [theta1, ... thetam] then you could reduce the parameter space by only looking at theta_reduce = [theta | d loss/d theta > threshold] or you could look at active subspaces and change the parameter space to xi = [xi1, ... xi_m] where x_i = SUM a_j theta_j. xi_i could be given by the largest eigenvectors of the sensitivity matrix S_ij = d^2 loss/dtheta_i dtheta_j Wouldn't it be nice if hacker news supported latex. I haven't done any work here, but I suspect I will be doing some of this towards the end of summer.