7 ms·

Sören Laue, creator and first author of this work [1] gave a nice talk [2,3] about it at our reading group not long ago. He also has written about how automatic

Sören Laue, creator and first author of this work [1] gave a nice talk [2,3] about it at our reading group not long ago. He also has written about how automatic and symbolic differentiation are essentially equivalent. [4] [1]: https://papers.nips.cc/paper/2018/file/0a1bf96b7165e962e90cb14648c9462d-Paper.pdf https://papers.nips.cc/paper/2018/file/0a1bf96b7165e962e90cb... [2]: https://www.youtube.com/watch?v=IbTRRlPZwgc https://www.youtube.com/watch?v=IbTRRlPZwgc [3]: https://compcalc.github.io/public/laue/tensor_derivatives.pdf https://compcalc.github.io/public/laue/tensor_derivatives.pd... [4]: https://arxiv.org/pdf/1904.02990.pdf https://arxiv.org/pdf/1904.02990.pdf

newsoul 4y ago
Do you know any books which teach matrix calculus?
- 0xcafefood 4y ago
  https://en.wikipedia.org/wiki/Tensor_calculus https://en.wikipedia.org/wiki/Tensor_calculus is a term you're much likelier to find as the subject of a book. Matrices are just specific special cases of tensors, after all, and mathematicians like to generalize. A good book on differential geometry will also probably start with an overview of tensor calculus.
  - hgsgm 4y ago
    And multivariable calculus is the intro to that.
    - sn9 4y ago
      Hubbard and Hubbard's Vector Calculus is a great introduction to multivariable calculus and linear algebra that ends with an introduction to differential forms. And it has a complete solutions manual you can purchase as well. https://matrixeditions.com/5thUnifiedApproach.html https://matrixeditions.com/5thUnifiedApproach.html
- metanonsense 4y ago
  I liked the following book very much: https://arxiv.org/abs/1802.01528 https://arxiv.org/abs/1802.01528
breck 4y ago
Interesting! Thanks for the [4] link.
davnn 4y ago
I hope that something like his „generic tensor multiplication operator“ [1], a slightly adapted Einstein notation, becomes the standard notation in the future. The traditional linear algebra notation with it‘s ugly transpose formalism is nothing I would ever want to use. [1] https://ojs.aaai.org/index.php/AAAI/article/download/5881/5737 https://ojs.aaai.org/index.php/AAAI/article/download/5881/57...
- soeren_l 4y ago
  I fully agree with you on this one. It took us ages to figure out that linear algebra notation it not the right way. Using tensor notation and einsum, tensor and matrix derivatives become almost trivial. And no need to use ugly transposes, Kronecker symbols, etc.. Thats why we wrote this article and created the website. (Sören Laue, creator of this website and my personal biased view)
  - cookieperson 4y ago
    Thanks for sharing these tools. Do you have any blogs or open access journals sharing this notation? It seems very simple, I'd love to use it

newsoul 4y ago

Do you know any books which teach matrix calculus?

0xcafefood 4y ago
https://en.wikipedia.org/wiki/Tensor_calculus https://en.wikipedia.org/wiki/Tensor_calculus is a term you're much likelier to find as the subject of a book. Matrices are just specific special cases of tensors, after all, and mathematicians like to generalize. A good book on differential geometry will also probably start with an overview of tensor calculus.
- hgsgm 4y ago
  And multivariable calculus is the intro to that.
  - sn9 4y ago
    Hubbard and Hubbard's Vector Calculus is a great introduction to multivariable calculus and linear algebra that ends with an introduction to differential forms. And it has a complete solutions manual you can purchase as well. https://matrixeditions.com/5thUnifiedApproach.html https://matrixeditions.com/5thUnifiedApproach.html
metanonsense 4y ago
I liked the following book very much: https://arxiv.org/abs/1802.01528 https://arxiv.org/abs/1802.01528

breck 4y ago

Interesting! Thanks for the [4] link.

davnn 4y ago

I hope that something like his „generic tensor multiplication operator“ [1], a slightly adapted Einstein notation, becomes the standard notation in the future. The traditional linear algebra notation with it‘s ugly transpose formalism is nothing I would ever want to use. [1] https://ojs.aaai.org/index.php/AAAI/article/download/5881/5737 https://ojs.aaai.org/index.php/AAAI/article/download/5881/57...

soeren_l 4y ago
I fully agree with you on this one. It took us ages to figure out that linear algebra notation it not the right way. Using tensor notation and einsum, tensor and matrix derivatives become almost trivial. And no need to use ugly transposes, Kronecker symbols, etc.. Thats why we wrote this article and created the website. (Sören Laue, creator of this website and my personal biased view)
- cookieperson 4y ago
  Thanks for sharing these tools. Do you have any blogs or open access journals sharing this notation? It seems very simple, I'd love to use it