4 ms·
This does not solve the issue. You still need to invert this sparse matrix for inference. The inverse of a sparse covariance matrix is dense.
by mrfox321 7y ago
This does not solve the issue. You still need to invert this sparse matrix for inference. The inverse of a sparse covariance matrix is dense.
- nestorD 7y agoThe raw mathematical formula uses an inversion but what you truly need is to solve a linear system (which is why most implementations store a cholesky decomposition of the covariance matrix) which can be done without transforming the matrix into a dense matrix.
- clarkeni 7y agoWhat would you use to solve the system? Iterative methods or gradient descent even?
- lp251 7y agoyes
- plus 7y agoSomething like MINRES or GMRES or CGSTAB. Correct me if I'm wrong, but the covariance matrix in GPR should be diagonally dominant, so the diagonal of the matrix can be used as a somewhat effective preconditioner.
- nestorD 7y agoWith most kernels the matrix is symmetric and positive-definite so I would probably use a conjugate gradient: https://en.wikipedia.org/wiki/Conjugate_gradient_method https://en.wikipedia.org/wiki/Conjugate_gradient_method
- bhl 7y ago+1 This is how I solved for the optimal parameter: instead of inverting the matrix, which is memory-heavy and computationally expensive, I used the solve function from scipy.linalg. I looked into numpy and scipy methods for symmetry and sparsity, but my kernel matrix is still too dense.