5 ms·
Physics-Based Deep Learning Book
- gillesjacobs 5y ago"Physics-based" Deep Learning seems like a misnomer. From the abstract "Deep Learning Applications for Physics" sounds more apt. There definitely is value in transferring standard terminology and methods from physics to deep learning. But from the preview it's unclear if that is the focus.
- AbrahamParangi 5y agoIndeed, “Deep Learning Based Physics” seems a little more correct.
- amcoastal 5y agoThe common terminology is Physics Informed Neural Networks (PINNs)
- ngcc_hk 5y agoThe question of that is it target the network. But if it is for physics … should it be neural network using physics for physics (by physics scientist).
- westurner 5y ago"Physics-informed neural networks" https://en.wikipedia.org/wiki/Physics-informed_neural_networks https://en.wikipedia.org/wiki/Physics-informed_neural_networ... But what about statistical thermodynamics and information theory? What about thin film? What are some applications for PINNs and for {DL, RL,} in physics?
- nerdponx 5y agoI definitely thought it was the opposite from the title, applying physics concepts/techniques to general deep learning problems.
- omrjml 5y agoThis does not appear to be the usual approach of training the neural network on tons of data from the physics simulation. Instead they use the actual physics equations to form the loss function, which is a far more robust way of creating such an emulator. So physics based deep learning title is appropriate.
- kkylin 5y agoIndeed. I see where everyone on HN is coming from. But if you're a physicist, and you've come across a lot of "deep learning applied to physics but the model has no physics in it" (and there's plenty of that), then the title may make perfect sense.
- lambdatronics 5y agoYeah, many scientists in my field have been justifiably skeptical of black-box machine/deep learning applications -- just sounded like the latest meaningless buzzword. I think this approach potentially is a big deal. [Edit: by "this approach" I mean what the article is calling "differentiable physics" -- but I don't love that moniker. The "physics informed neural network" approach doesn't seem that great to me. It's much slower than doing an actual simulation, the resulting errors are larger, and you can't re-use results -- it's a one-off solution. The fact that you can use it to interpolate isn't that much of a selling point. The only nice thing is that you can throw any system of equations you want at it without having to design a numerical solver.]
- jessriedel 5y agoEh, they may put more emphasis on that technique, but its only a subset of the scope: > This document contains a practical and comprehensive introduction of everything related to deep learning in the context of physical simulations. ... Beyond standard supervised learning from data, we’ll look at physical loss constraints, more tightly coupled learning algorithms with differentiable simulations,...
- deleted 5y ago[deleted]
- Archit3ch 5y agoWhy python? Doesn't Julia compose traditional ODE solvers with differential programming better?
- UncleOxidant 5y agoFound a couple of resources for what they call Physics-Informed Neural Networks in Julia: https://mitmath.github.io/18337/lecture3/sciml.html https://mitmath.github.io/18337/lecture3/sciml.html https://diffeqflux.sciml.ai/dev/ https://diffeqflux.sciml.ai/dev/ https://www.youtube.com/watch?v=HKJB0Bjo6tQ https://www.youtube.com/watch?v=HKJB0Bjo6tQ (Interpretable Deep Learning for Physics - I don't think there's any Julia in the video itself, but Miles Cranmer uses Julia for this work - he created SymbolicRegression.jl)
- atty 5y agoBecause the python/numpy/tensorflow/Pytorch ecosystem is the deep learning ecosystem. You write a book on what you know, and what the audience wants. And everything they do here has tools that work just fine in python, no need to switch to Julia. And as someone who does this sort of work as my job, I can tell you the tool chains we use for what could be considered “industrial-scale” applications do not yet have appropriate Julia-based replacements.
- amkkma 5y agoThat's because it's keeping things fairly simple. Implementing the more involved/advanced techniques would quickly outpace what's possible in python, now and for the foreseeable future (see https://www.stochasticlifestyle.com/useful-algorithms-that-are-not-optimized-by-jax-pytorch-or-tensorflow/ https://www.stochasticlifestyle.com/useful-algorithms-that-a... but also other reasons such as composability )
- tubby12345 5y agois this the "why not rewrite it in rust" of the DL space?
- nerdponx 5y ago
- s1291 5y agoWhat a coincidence! just before I come across this post, I was reading this book this morning.
- sampo 5y agoMore like Deep Learning -based physics. Or more accurately: Deep Learning -assisted physics simulations.
- lambdatronics 5y agoYeah, the Julia-language equivalent is referred to as "scientific machine learning" but I don't like that either. I think "simulation learning" would be snappier and maybe more appropriate.
- ivan_ah 5y agometa comment: the website you're viewing when you click that link is generated using Jupyter Book , which is like the best thing ever https://jupyterbook.org/intro.html https://jupyterbook.org/intro.html Imagine all the beauty of .Rmd (easily generate books by combining markdown explanations with executable cells), but adapted for the Python ecosystem (jupyter notebooks). Jupyuter-book is a really well thought-out project. You create the book with a _toc.yml file: https://github.com/tum-pbs/pbdl-book/blob/main/_toc.yml https://github.com/tum-pbs/pbdl-book/blob/main/_toc.yml and all the config is in one file: https://github.com/tum-pbs/pbdl-book/blob/main/_config.yml https://github.com/tum-pbs/pbdl-book/blob/main/_config.yml (the build system leverages Sphinx which is the docs workhorse in the Python world) One of the coolest things is the "Launcher" option which gives readers the options to "run" any notebook interactively (using the rocket button in the top right). It's a one-line config https://github.com/tum-pbs/pbdl-book/blob/main/_config.yml#L10-L12 https://github.com/tum-pbs/pbdl-book/blob/main/_config.yml#L... A similar config would enable the "Launch in Pybinder" option which is a free ephemeral jupyter provider, see https://mybinder.org/ https://mybinder.org/ This "execute anywhere" option is nicely abstracted away as the `thiebe` library, and there is even POC work to run a pyodide kernel (https://github.com/executablebooks/thebe/issues/465 https://github.com/executablebooks/thebe/issues/465) so soon all of this goodness will work offline in your browser! As an educator, it's hard not to get excited about the future, given the pace at which learning/teaching tooling is developing!
- ngcc_hk 5y agoNeed another thread on this. It is a major learning tool when I first encountered Jupyter.
- RavlaAlvar 5y agoI was doing a project that simulate a projectile motion with drag and Magnus force. This has no analytical solutions and I currently solve it with a numeric solver. Does anyone know if I could approximate a closed form solution with a simple MLP network?
- omrjml 5y agoNot MLP, but PySindy[0] might be a way to find a closed form solution. A simple MLP may not be best way to achieve what are you are after. [0] https://github.com/dynamicslab/pysindy https://github.com/dynamicslab/pysindy