4 ms·
The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It
by yorwba 8y ago
The problem with einsum is that you have to explicitly specify the mapping between dimensions and indices every time, without any way to enforce consistency. It would be more ergonomic if each tensor had labeled dimensions. That would prevent the kind of silly mistake where you mix up the ordering of dimensions and only notice it when you later change the shape of the tensors so the different dimensions no longer match up.
- sischoel 8y agoI think, this is something that the community is now slowly realizing. I've must have read about that multiple times recently. Some packages that seem to provide tensors with labeled dimensions.(I haven't personally worked with them): xarray (Python): https://github.com/pydata/xarray https://github.com/pydata/xarray NamedArrays.jl (Julia): https://github.com/davidavdav/NamedArrays.jl https://github.com/davidavdav/NamedArrays.jl
- adamnemecek 8y agohttp://nlp.seas.harvard.edu/NamedTensor http://nlp.seas.harvard.edu/NamedTensor
- deleted 8y ago[deleted]
- claudius 8y agoITensor [1] has somewhat pioneered this concept in tensor networks for condensed matter physics. If I understand correctly, Uni10 [2], a similar project, is even working on a graphical interface for such networks so that you can "draw" the network and have the computer figure out the best contraction order. In my own code I’ve recently also implemented such named indices (only had "einstein summation"-like contraction specifiers before) and they make tensor contractions so much simpler to write, especially since you can simply overload operator * and have it figure out which legs need to go together. As a side effect, it also enforces your algorithms to make sense because you can't simply add two tensors living on different (but equal-dimensional) vector spaces together anymore. [1] https://itensor.org https://itensor.org [2] https://uni10.gitlab.io/ https://uni10.gitlab.io/
- mcabbott 8y agoI made a thing to audit that you are consistent with your indices [1] as an alternative to explicit named-tensor objects. And found, but have not used, another package in a similar spirit [2]. Although to be honest, simply writing A[n,μ,ν,c] etc. (using different letters for different spaces) makes it pretty easy to visually check that you are getting this right. This is one of the attractions of the notation, even on paper. Unfortunately np.einsum's string notation makes this harder to see, as the indices aren't adjacent to the variable name. [1] https://github.com/mcabbott/TensorCast.jl#checking https://github.com/mcabbott/TensorCast.jl#checking (Julia) [2] https://github.com/ofnote/tsalib https://github.com/ofnote/tsalib (Python)
- claudius 8y ago> I made a thing to audit that you are consistent with your indices [1] as an alternative to explicit named-tensor objects. And found, but have not used, another package in a similar spirit [2]. Cool, I will have to check this out! > Although to be honest, simply writing A[n,μ,ν,c] etc. (using different letters for different spaces) makes it pretty easy to visually check that you are getting this right. This is one of the attractions of the notation, even on paper. Unfortunately np.einsum's string notation makes this harder to see, as the indices aren't adjacent to the variable name. The problem is not so much the letter-space association but also handling the ordering of the spaces inside the tensor. For example in my code, to do a contraction over two indices, you could do prod<2>(a, b, "tlx,tr,p1,p2|tlx,p1,tl|tl,p2,tr") where the result would then have index order tl,p2,tr. The problem was then that changing the index order in one place (e.g. for performance reasons) meant having to re-check all other places where this is used. If you want to contract a tensor network like (d) in [1], this quickly gets complicated. With named indices, the above becomes a * b and if I change the index order in any place, it gets automatically changed there, too. [1] https://journals.aps.org/prb/article/10.1103/PhysRevB.81.165104/figures/8/medium https://journals.aps.org/prb/article/10.1103/PhysRevB.81.165...
- mcabbott 8y agoRight, changing the order would be a pain. Although if the reason for doing so is memory layout (for speed), then a lazy permutedims(A) would decouple index order from this. Perhaps when creating a tensor for the first time there ought to be a way to specify the layout? Haven't thought much but something like A[n^4, μ,ν, c^1] := .... would not be hard to do. In your prod<2>(a, b, ... example, if p1 and p2 are in the same space, how would a*b know which one to contract? Or do they have different names from when a was created?
- CuriousSkeptic 8y agoWhy is it that mathematicians are so comfortable with using ordering to encode semantics? Evident from this thread it seems many developer recognize the need to use proper labels to simplify reasoning. Doesn’t mathematicians have similar desires?
- improbable22 8y agoWell, many different notations are convenient in different contexts. The compiler they target is a bit more flexible than developers', so they are free to just describe what they are doing on page 1. Human languages face the same trade-off: in some word order is very important, in others less so, but they need to compensate with some kind of case labels.
- curiousmiro 8y agoSee HarvardNLP's namedtensor which is under active development! disclaimer: i'm a contributor