3 ms·
Way back when, I did a masters in physics. I learned a lot of math: vectors, a ton of linear algebra, thermodynamics (aka entropy), multi-variable and then tens
by libraryofbabel 1y ago
Way back when, I did a masters in physics. I learned a lot of math: vectors, a ton of linear algebra, thermodynamics (aka entropy), multi-variable and then tensor calculus.
This all turned out to be mostly irrelevant in my subsequent programming career.
Then LLMs came along and I wanted to learn how they work. Suddenly the physics training is directly useful again! Backprop is one big tensor calculus calculation, minimizing… entropy! Everything is matrix multiplications. Things are actually differentiable, unlike most of the rest of computer science.
It’s fun using this stuff again. All but the tensor calculus on curved spacetime, I haven’t had to reach for that yet.
- psb217 1y agoThat past work will pay off even more when you start looking into diffusion and flow-based models for generating images, videos, and sometimes text.
- pornel 1y agoBreakthrough in image generation speed literally came from applying better differential equations for diffusion taken from statistical mechanics physics papers: https://youtu.be/iv-5mZ_9CPY https://youtu.be/iv-5mZ_9CPY
- JBits 1y agoFor me, it's the very basics of general relativity which made the distinction between the cotangent and tangents space click. Optimisation on Riemannian manifolds might give an opportunity to apply more interesting tensor calculus with a non-trivial metric.
- alguerythme 1y agoWell, calculus on curved space, please let me introduce you to: https://arxiv.org/abs/2505.18230 https://arxiv.org/abs/2505.18230 (This is self advertising) If you know how to incorporate time into that, I am interested.
- CrossVR 1y agoAny reason you didn't pick up computer graphics before? Everything is linear algebra and there's even actual physics involved.
- ls-a 1y agoIs that you Will Hunting
- jwar767 1y agoI have the same experience but with a masters in control theory. Suddenly all the linear algebra and differential equations are super useful in understanding this.
- r-bryan 1y agoCheck out this 156-page tome: https://arxiv.org/abs/2104.13478 https://arxiv.org/abs/2104.13478: "Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges" The intro says that it "...serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented." Working all the way through that, besides relearning a lot of my undergrad EE math (some time in the previous century), I learned a whole new bunch of differential geometry that will help next time I open a General Relativity book for fun.
- minhaz23 1y agoI have very little formal education in advanced maths, but I’m highly motivated to learn the math needed to understand AI. Should i take a stab at parsing through and trying to understand this paper (maybe even using AI to help, heh) or would that be counter-productive from the get-go and I'm better off spending my time following some structured courses in pre-requisite maths before trying to understand these research papers? Thank you for sharing this paper!
- mixmastamyk 1y agoYou might take a course on linear-algebra, at K.A. for example: https://www.khanacademy.org/math/linear-algebra https://www.khanacademy.org/math/linear-algebra And any prereqs you need. I also find the math-is-fun site to be excellent when I need to brush up on something from long ago and want a concise explanation. i.e. A 10 minute review, more than a few pithy sentences, yet less than a dozen-hour diatribe. https://www.mathsisfun.com/ https://www.mathsisfun.com/
- minhaz23 1y agoThank you for sharing this!
- Quizzical4230 1y ago
- lazarus01 1y agoModern numeric compute frameworks provide automatic differentiation to calculate derivatives, including Tensorflow and Jax.
- 3abiton 1y agoThe funny thing about physics maths, is that we didn't have to learn the intuition behind it, it was a mean to an end. Going through undergrad mathematically blind was a right of passage.
- Mallowram 1y ago[dead]