4 ms·

The tensor product, demystified (2018)

94 points by RevoGen 3y ago

uptownfunk 3y ago
Very well written. The author should write textbooks.
sigmoid10 3y ago
A nice, high school level writeup of how to calculate this product with ordinary vectors, but it totally drops the ball on the necessity and its use in physics. It would probably be best to ignore the entire last paragraph and instead read up on pure vs. mixed quantum states if you actually care about that.
- hackandthink 3y ago
  It's in the text: "The tensor product itself captures all ways that basic things can 'interact' with each other!" Tensor Product is also the way to go when combining classical probabilistic systems. And you need the tensor product already for pure states in QM. (mixed states need density matrices)
  - sigmoid10 3y ago
    >The tensor product itself captures all ways that basic things can 'interact' with each other! In quantum physics, "interacting" usually has a different meaning. So one should use these terms more carefully. >And you need the tensor product already for pure states in QM. No. Pure states are just vectors (or more precise: rays) in Hilbert space. The usual inner product is sufficient to work with them. An outer (=tensor) product of these states will just give you a density matrix with tr(ρ^2)=1.
    - hackandthink 3y ago
      "Product states are multipartite quantum states that can be written as a tensor product" "In the special case of pure states the definition simplifies: a pure state is separable if and only if it is a product state." https://en.wikipedia.org/wiki/Separable_state https://en.wikipedia.org/wiki/Separable_state
      - sigmoid10 3y ago
        Keyword is can. There's a reason why most university level physics curricula defer quantum density matrices (and by extension tensor algebra) to more advanced classes. There's a lot of mathematical legwork required before you can actually make use of that.
akasakahakada 3y ago
Tensor product is basically the mathematical way to express if condition. Say you have 2x2 matrix A, B, C. Any arbitrary component within the Tensor(A, B, C) is: if A =(a1,a2) and B=(b1,b2) and C=(c1,c2): this value The coordination is concatenation of matching dimension: (a1b1c1, a2b2c2) While in many computational natural science, people using tensor product to store and manipulate data. Also it is how the mathematical equations being written on paper. But in computer science's perspective, dealing with tensor matrix is simply a waste of memory since 90% of the time people are dealing with sparse system. System that their matrix is dominated by zero. Also it would be super clear if people just write if-then pseudocode instead of cryptic half-bake tensor expressions. People tend to invent their own notation while writting paper.
- consilient 3y ago
  > Tensor product is basically the mathematical way to express if condition. Not really, no. The way mathematicians actually express if conditions is with the word "if". The obvious pointlessly formal way to do it is with a pair of functions `ThingConditionedOn -> {0, 1}` and `{0, 1} -> Result`, but why would you? > dealing with tensor matrix is simply a waste of memory since 90% of the time people are dealing with sparse system. Tensors are not their components, any more than locations are their coordinates. Whether you choose a sparse or dense (or symbolic) encoding does not change the object being encoded.
  - akasakahakada 3y ago
    > encoding does not change the object being encoded Here I am talking about real memory space. There exist more efficient isometrical representation of the same object but since mathematicians don't usually know computer, inefficient tensor matrix is by default. That's the problem.
tempodox 3y ago
If only those squiggles were readable.
Y_Y 3y ago
It was a bit annoying that the first few examples give a flattened tensor product. It's explained at the end, but the dimensions are at least as important as the values and it feels wrong to skip that (isomorphism notwithstanding).
ubavic 3y ago
Hmm, I always disliked coordinate approach to (multi)linear algebra. To me, the notion of a tensor product clicked with the following definition: Tensor product v₁⊗v₂ of vectors form V is bilinear form on the dual space V* defined by v₁⊗v₂(ω₁,ω₂) = v₁(ω₁)v₂(ω₂). Tensor product V⊗V is space of all bilinear forms on V*, This can easily be generalized to products of forms (covariant tensors), and mixed tensor products. Getting coordinate representation in basis (E₁ ... Eₙ) is trivial: (v₁⊗v₂)ₖ,ₗ = v₁⊗v₂(εₖ,εₗ) where εₖ is dual to Eₖ...
- howling 3y ago
  I always dislike the unnecessary use of dual of dual. We should define tensor product as a quotient space of all linear combinations of v₁⊗v₂ as in https://en.wikipedia.org/wiki/Tensor_product#As_a_quotient_space https://en.wikipedia.org/wiki/Tensor_product#As_a_quotient_s... The advantage of not using dual of dual is that it generalizes correctly to infinite dimensional vector spaces and modules.
- consilient 3y ago
  > Tensor product V⊗V is space of all bilinear forms on V* This is only true for finite-dimensional V. In general bilinear forms on V* are linear maps `V*⊗V* -> F` (where F is the base field) which is isomorphic to `(V*⊗V*)*`. For infinite-dimensional V this is much larger than `V⊗V`. The general characterization is that `V⊗W` is the vector space (and associated linear map `⊗: V x W -> V⊗W`) such that for every bilinear map `h: V x W -> T` there's a unique linear g such that `h(v,w) = g(v⊗w)`. Intuitively, it adds "just enough" elements to `V x W` to encode arbitrary bilinear maps as linear ones.
saberience 3y ago
This didn't demystify anything to me, I left it more mystified it than when I entered.
hackandthink 3y ago
Not relevant for tensor product: The author seems to be related to https://www.sandboxaq.com/ https://www.sandboxaq.com/. "SandboxAQ leverages the compound effects of AI and Quantum technologies (AQ) to solve hard challenges impacting society." This reads like a scam (don't know anything about it).
mbivert 3y ago
Some additional ressources on tensors I've found useful: 1) A video[0] by Michael Penn exposing this idea of "tensor product of vector spaces." It's close to what is presented in this blog post, but more rigorous/complete. 2) Two videos[1][2] by Frederic Schuller. They are each from of bigger courses (resp. ~Differential Geometry for Physics and Quantum Mechanics), but I think they are self-contained enough to be intelligible. They both present tensors in different settings, so one will have to work a little to unify all this. I like in particular how in [1] he really takes the time to first distinguish between all the different tensor product (of space, of vectors, of operators): the usual notation/terminology can be needlessly confusing for beginners. [0]: https://www.youtube.com/watch?v=K7f2pCQ3p3U https://www.youtube.com/watch?v=K7f2pCQ3p3U [1]: https://www.youtube.com/watch?v=OZ1WCyJmjgo https://www.youtube.com/watch?v=OZ1WCyJmjgo [2]: https://www.youtube.com/watch?v=4l-qzZOZt50 https://www.youtube.com/watch?v=4l-qzZOZt50
lisper 3y ago
> But why the tensor product? Why is it that this construction—out of all things—describes the interactions within a quantum system so well, so naturally? I don’t know the answer That's an odd thing for the author to say, because s/he gives the answer later in the very same passage: > but perhaps the appropriateness of tensor products shouldn't be too surprising. The tensor product itself captures all ways that basic things can "interact" with each other! That is the answer. It's the tensor product because there are logically no other possibilities. The tensor product says everything you can possibly say about the interactions of two systems whose states are described by a (possibly infinite) set of numbers and whose interactions correspond to some basic constraints, like being time-reversible. It just so happens that nature behaves according to those constraints, and that is why the tensor product describes the behavior of nature.
- codethief 3y ago
  > It's the tensor product because there are logically no other possibilities. There are many other possibilities, unless you can provide satisfactory answers (from first principles) to the following questions: Why would we expect superpositions of quantum states to be encoded as a vector sum of the individual state vectors? Why is time evolution in quantum mechanics a linear operation on those state vectors? If those things weren't true, tensor products would be utterly useless to describe product states.
  - lisper 3y ago
    We don't expect these things, we merely observe them. Indeed, the fact that QM is linear and hence time-reversible is violently at odds with everyday experience, so it is emphatically not the case that we "expect" these things. This just turns out to be how they are. The tensor product is merely the most compact description of these observations, kind of like how untyped lambda calculus turns out to be a compact description of universal computation (which is also not a thing that one would a priori expect).
    - codethief 3y ago
      > We don't expect these things, we merely observe them. Sure, but your original claim was that > It's the tensor product because there are logically no other possibilities. The tensor product says everything you can possibly say about the interactions of two systems whose states are described by a (possibly infinite) set of numbers and whose interactions correspond to some basic constraints, like being time-reversible. I merely wanted to point out that your claim sounded quite broad and you need to assume many things about the mathematical structure of QM here (based on established observations of course). So, unless you simply take those for granted, you would have to come up with an explanation for them in order for there to be > logically no other possibilities In this case, though (if you take all those observations for granted), your claim becomes almost tautological IMO.
ziofill 3y ago
Very nice, but there’s a technical error in the section “in quantum physics”. A density matrix as described does not represent the state of two systems, just one system. I think the author is confusing a vector in H⊗H with a bounded operator on H.

94 points by RevoGen 3y ago

Very well written. The author should write textbooks.

A nice, high school level writeup of how to calculate this product with ordinary vectors, but it totally drops the ball on the necessity and its use in physics. It would probably be best to ignore the entire last paragraph and instead read up on pure vs. mixed quantum states if you actually care about that.

hackandthink 3y ago
It's in the text: "The tensor product itself captures all ways that basic things can 'interact' with each other!" Tensor Product is also the way to go when combining classical probabilistic systems. And you need the tensor product already for pure states in QM. (mixed states need density matrices)
- sigmoid10 3y ago
  >The tensor product itself captures all ways that basic things can 'interact' with each other! In quantum physics, "interacting" usually has a different meaning. So one should use these terms more carefully. >And you need the tensor product already for pure states in QM. No. Pure states are just vectors (or more precise: rays) in Hilbert space. The usual inner product is sufficient to work with them. An outer (=tensor) product of these states will just give you a density matrix with tr(ρ^2)=1.
  - hackandthink 3y ago
    "Product states are multipartite quantum states that can be written as a tensor product" "In the special case of pure states the definition simplifies: a pure state is separable if and only if it is a product state." https://en.wikipedia.org/wiki/Separable_state https://en.wikipedia.org/wiki/Separable_state
    - sigmoid10 3y ago
      Keyword is can. There's a reason why most university level physics curricula defer quantum density matrices (and by extension tensor algebra) to more advanced classes. There's a lot of mathematical legwork required before you can actually make use of that.

akasakahakada 3y ago

Tensor product is basically the mathematical way to express if condition. Say you have 2x2 matrix A, B, C. Any arbitrary component within the Tensor(A, B, C) is: if A =(a1,a2) and B=(b1,b2) and C=(c1,c2): this value The coordination is concatenation of matching dimension: (a1b1c1, a2b2c2) While in many computational natural science, people using tensor product to store and manipulate data. Also it is how the mathematical equations being written on paper. But in computer science's perspective, dealing with tensor matrix is simply a waste of memory since 90% of the time people are dealing with sparse system. System that their matrix is dominated by zero. Also it would be super clear if people just write if-then pseudocode instead of cryptic half-bake tensor expressions. People tend to invent their own notation while writting paper.

consilient 3y ago
> Tensor product is basically the mathematical way to express if condition. Not really, no. The way mathematicians actually express if conditions is with the word "if". The obvious pointlessly formal way to do it is with a pair of functions `ThingConditionedOn -> {0, 1}` and `{0, 1} -> Result`, but why would you? > dealing with tensor matrix is simply a waste of memory since 90% of the time people are dealing with sparse system. Tensors are not their components, any more than locations are their coordinates. Whether you choose a sparse or dense (or symbolic) encoding does not change the object being encoded.
- akasakahakada 3y ago
  > encoding does not change the object being encoded Here I am talking about real memory space. There exist more efficient isometrical representation of the same object but since mathematicians don't usually know computer, inefficient tensor matrix is by default. That's the problem.

tempodox 3y ago

If only those squiggles were readable.

Y_Y 3y ago

It was a bit annoying that the first few examples give a flattened tensor product. It's explained at the end, but the dimensions are at least as important as the values and it feels wrong to skip that (isomorphism notwithstanding).

ubavic 3y ago

Hmm, I always disliked coordinate approach to (multi)linear algebra. To me, the notion of a tensor product clicked with the following definition: Tensor product v₁⊗v₂ of vectors form V is bilinear form on the dual space V* defined by v₁⊗v₂(ω₁,ω₂) = v₁(ω₁)v₂(ω₂). Tensor product V⊗V is space of all bilinear forms on V*, This can easily be generalized to products of forms (covariant tensors), and mixed tensor products. Getting coordinate representation in basis (E₁ ... Eₙ) is trivial: (v₁⊗v₂)ₖ,ₗ = v₁⊗v₂(εₖ,εₗ) where εₖ is dual to Eₖ...

howling 3y ago
I always dislike the unnecessary use of dual of dual. We should define tensor product as a quotient space of all linear combinations of v₁⊗v₂ as in https://en.wikipedia.org/wiki/Tensor_product#As_a_quotient_space https://en.wikipedia.org/wiki/Tensor_product#As_a_quotient_s... The advantage of not using dual of dual is that it generalizes correctly to infinite dimensional vector spaces and modules.
consilient 3y ago
> Tensor product V⊗V is space of all bilinear forms on V* This is only true for finite-dimensional V. In general bilinear forms on V* are linear maps `V*⊗V* -> F` (where F is the base field) which is isomorphic to `(V*⊗V*)*`. For infinite-dimensional V this is much larger than `V⊗V`. The general characterization is that `V⊗W` is the vector space (and associated linear map `⊗: V x W -> V⊗W`) such that for every bilinear map `h: V x W -> T` there's a unique linear g such that `h(v,w) = g(v⊗w)`. Intuitively, it adds "just enough" elements to `V x W` to encode arbitrary bilinear maps as linear ones.

saberience 3y ago

This didn't demystify anything to me, I left it more mystified it than when I entered.

hackandthink 3y ago

Not relevant for tensor product: The author seems to be related to https://www.sandboxaq.com/ https://www.sandboxaq.com/. "SandboxAQ leverages the compound effects of AI and Quantum technologies (AQ) to solve hard challenges impacting society." This reads like a scam (don't know anything about it).

mbivert 3y ago

Some additional ressources on tensors I've found useful: 1) A video[0] by Michael Penn exposing this idea of "tensor product of vector spaces." It's close to what is presented in this blog post, but more rigorous/complete. 2) Two videos[1][2] by Frederic Schuller. They are each from of bigger courses (resp. ~Differential Geometry for Physics and Quantum Mechanics), but I think they are self-contained enough to be intelligible. They both present tensors in different settings, so one will have to work a little to unify all this. I like in particular how in [1] he really takes the time to first distinguish between all the different tensor product (of space, of vectors, of operators): the usual notation/terminology can be needlessly confusing for beginners. [0]: https://www.youtube.com/watch?v=K7f2pCQ3p3U https://www.youtube.com/watch?v=K7f2pCQ3p3U [1]: https://www.youtube.com/watch?v=OZ1WCyJmjgo https://www.youtube.com/watch?v=OZ1WCyJmjgo [2]: https://www.youtube.com/watch?v=4l-qzZOZt50 https://www.youtube.com/watch?v=4l-qzZOZt50

lisper 3y ago

> But why the tensor product? Why is it that this construction—out of all things—describes the interactions within a quantum system so well, so naturally? I don’t know the answer That's an odd thing for the author to say, because s/he gives the answer later in the very same passage: > but perhaps the appropriateness of tensor products shouldn't be too surprising. The tensor product itself captures all ways that basic things can "interact" with each other! That is the answer. It's the tensor product because there are logically no other possibilities. The tensor product says everything you can possibly say about the interactions of two systems whose states are described by a (possibly infinite) set of numbers and whose interactions correspond to some basic constraints, like being time-reversible. It just so happens that nature behaves according to those constraints, and that is why the tensor product describes the behavior of nature.

codethief 3y ago
> It's the tensor product because there are logically no other possibilities. There are many other possibilities, unless you can provide satisfactory answers (from first principles) to the following questions: Why would we expect superpositions of quantum states to be encoded as a vector sum of the individual state vectors? Why is time evolution in quantum mechanics a linear operation on those state vectors? If those things weren't true, tensor products would be utterly useless to describe product states.
- lisper 3y ago
  We don't expect these things, we merely observe them. Indeed, the fact that QM is linear and hence time-reversible is violently at odds with everyday experience, so it is emphatically not the case that we "expect" these things. This just turns out to be how they are. The tensor product is merely the most compact description of these observations, kind of like how untyped lambda calculus turns out to be a compact description of universal computation (which is also not a thing that one would a priori expect).
  - codethief 3y ago
    > We don't expect these things, we merely observe them. Sure, but your original claim was that > It's the tensor product because there are logically no other possibilities. The tensor product says everything you can possibly say about the interactions of two systems whose states are described by a (possibly infinite) set of numbers and whose interactions correspond to some basic constraints, like being time-reversible. I merely wanted to point out that your claim sounded quite broad and you need to assume many things about the mathematical structure of QM here (based on established observations of course). So, unless you simply take those for granted, you would have to come up with an explanation for them in order for there to be > logically no other possibilities In this case, though (if you take all those observations for granted), your claim becomes almost tautological IMO.

ziofill 3y ago

Very nice, but there’s a technical error in the section “in quantum physics”. A density matrix as described does not represent the state of two systems, just one system. I think the author is confusing a vector in H⊗H with a bounded operator on H.