4 ms·
Another easy solution, if your kernel is appropriate (goes to 0 at infinity such as a Gaussian, Matèrn or Exponential kernel), is to set a threshold under which
by nestorD 7y ago
Another easy solution, if your kernel is appropriate (goes to 0 at infinity such as a Gaussian, Matèrn or Exponential kernel), is to set a threshold under which your covariance is set to zero and to use a sparse representation for the covariance matrix (you can even be more extreme and set a fixed number of entry to be non-zero giving you a fixed memory use).
- mrfox321 7y agoThis does not solve the issue. You still need to invert this sparse matrix for inference. The inverse of a sparse covariance matrix is dense.
- nestorD 7y agoThe raw mathematical formula uses an inversion but what you truly need is to solve a linear system (which is why most implementations store a cholesky decomposition of the covariance matrix) which can be done without transforming the matrix into a dense matrix.
- clarkeni 7y agoWhat would you use to solve the system? Iterative methods or gradient descent even?
- lp251 7y agoyes
- plus 7y agoSomething like MINRES or GMRES or CGSTAB. Correct me if I'm wrong, but the covariance matrix in GPR should be diagonally dominant, so the diagonal of the matrix can be used as a somewhat effective preconditioner.
- nestorD 7y agoWith most kernels the matrix is symmetric and positive-definite so I would probably use a conjugate gradient: https://en.wikipedia.org/wiki/Conjugate_gradient_method https://en.wikipedia.org/wiki/Conjugate_gradient_method
- bhl 7y ago+1 This is how I solved for the optimal parameter: instead of inverting the matrix, which is memory-heavy and computationally expensive, I used the solve function from scipy.linalg. I looked into numpy and scipy methods for symmetry and sparsity, but my kernel matrix is still too dense.