4 ms·

Higher level BLAS operations, such as solving a linear system, or computing svd/eigen, cannot be done in-place the way matrix multiplication can, and require ad

Higher level BLAS operations, such as solving a linear system, or computing svd/eigen, cannot be done in-place the way matrix multiplication can, and require additional memory of a predetermined, fixed size, called the work array. This cannot be pre-allocated in Julia, as there is no interface to do so in LinearAlgebra, so these BLAS calls will always allocate memory.

DNF2 6y ago
Have you looked thoroughly over https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/ https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/ ? There's a ton of in-place operations there, including in-place solving of linear systems, and at least some stuff related to svd (though the names are pretty un-intuitive, being wrappers for BLAS and LAPACK functions).
- cbkeller 6y ago
  Hah, you were not kidding about the names being unintuitive. For anyone else interested there is a large list starting here [1]. The in-place versions will all end with `!`. [1] https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.LAPACK https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#Linea...
- spacedome 6y ago
  Yes, many of these are "in-place" but will still allocate. I typically use the geev!/ggev!/geevx! routines, if you look at the source code you will see that the work arrays are still allocated inside the call. The in-place here (unfortunately) means only that the input is overwritten, not that there is no allocation.
  - DNF2 6y ago
    The purpose of in-place operations is to avoid allocations, so this seems like an oversight. Is this a matter of simply wrapping the 'right' LAPACK routines, or is there something missing in the interfaces that could be trivially added in principle?
    - spacedome 6y ago
      It is entirely possible, but is not trivial, especially for the user who then needs to know "arcane knowledge" of BLAS/LAPACK work array sizes and flags. There was some discussion about this on github, but it sort of trailed off without a real resolution. I think it is considered too complicated/niche to be in base, and was recommended to be an external library, but nobody (myself included) seems particularly interested in what amounts to maintaining a fork of the entire BLAS package. The base devs would have more insight, but this is my view from the outside at least.

DNF2 6y ago

Have you looked thoroughly over https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/ https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/ ? There's a ton of in-place operations there, including in-place solving of linear systems, and at least some stuff related to svd (though the names are pretty un-intuitive, being wrappers for BLAS and LAPACK functions).

cbkeller 6y ago
Hah, you were not kidding about the names being unintuitive. For anyone else interested there is a large list starting here [1]. The in-place versions will all end with `!`. [1] https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#LinearAlgebra.LAPACK https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#Linea...
spacedome 6y ago
Yes, many of these are "in-place" but will still allocate. I typically use the geev!/ggev!/geevx! routines, if you look at the source code you will see that the work arrays are still allocated inside the call. The in-place here (unfortunately) means only that the input is overwritten, not that there is no allocation.
- DNF2 6y ago
  The purpose of in-place operations is to avoid allocations, so this seems like an oversight. Is this a matter of simply wrapping the 'right' LAPACK routines, or is there something missing in the interfaces that could be trivially added in principle?
  - spacedome 6y ago
    It is entirely possible, but is not trivial, especially for the user who then needs to know "arcane knowledge" of BLAS/LAPACK work array sizes and flags. There was some discussion about this on github, but it sort of trailed off without a real resolution. I think it is considered too complicated/niche to be in base, and was recommended to be an external library, but nobody (myself included) seems particularly interested in what amounts to maintaining a fork of the entire BLAS package. The base devs would have more insight, but this is my view from the outside at least.