3 ms·
It is a little jarring to hear "data-driven" and "nonperturbative" in the same sentence. It sounds a little bit like saying you designed a boat with a better li
by scythe 1y ago
It is a little jarring to hear "data-driven" and "nonperturbative" in the same sentence. It sounds a little bit like saying you designed a boat with a better lift-to-drag ratio. "Wait, is it a boat or a plane?". So, I opened the paper fully expecting to not understand anything, and I was pleasantly surprised.
> First we deduce formally-exact non-perturbative guiding center
equations of motion assuming a hidden symmetry with associated conserved quantity J. We refer to J as the non-perturbative adiabatic invariant.
Simply: this is not just some kind of unsupervised ML black-box magic. There is a formal mathematical solution to something, but it has a certain gap, namely precisely what quantity is conserved and how to calculate it.
> Then we describe a data-driven method for learning J from a dataset of full-orbit α-particle trajectories. [...] Our proposed method for learning J applies on a per-magnetic field basis; changing B requires re-training. This makes it well-suited to stellarator design assessment tasks, such as α-loss fraction uncertainty quantification.
With the formal simplification of the dynamics in hand, the researchers believe that a trained model can then give a useful approximation of the invariant, which allows the formal model, with its unknown parameters now filled in, to be used to model the dynamics.
In a crude way, I think I have a napkin-level sketch of what they're doing here. Suppose we are modeling a projectile, and we know nothing of kinematics. They have determined that the projectile has a parabolic trajectory (the formal part) and then they are using data analysis to find the g coefficient that represents gravitational acceleration (the data-driven part). Obviously, you would never need machine learning in such a very simple case as I have described, but I think it approximates the main idea.
- ChrisMarshallNY 1y agoOne of the nice things about LLMs/ML, is that they can pound away at something for a billion cycles, and do exactly the same things that you or I would do. for _ in 0..<1000000000000 { do_something_complicated() }
- kjkjadksj 1y agoIsn’t that one of the nice things of computers in general not a feature of llm?
- ChrisMarshallNY 1y agoThe difference is the complexity of the repeated task
- BandButcher 1y agoBut don't those complexities still boil down to machine level instructions?? Or can/do llms operate outside of a CPU? Thanks
- ChrisMarshallNY 1y agoI’m not getting into angels and pinheads, but modern ML has the ability to perform “fuzzy analysis,” and interpret results in a far more flexible manner, than ever before. They may not be able to match an MIT Ph.D, at analyzing experimental feedback, but they can probably match a lot of research assistants. It’s like having a billion RAs, all running experiments, and triaging the results. I understand that is how they have made such good progress on medicines, with AI. > “I have not failed. I've just found 10,000 ways that won't work.” -Attributed to Thomas Edison
- kjkjadksj 1y agoWith machine learning, not with llm.
- ChrisMarshallNY 1y agoThis is true.
- Timwi 1y agoThe CPU/GPU is to an LLM kinda like axons and dendrites are to the human brain: just a low-level implementation detail. The main crux of an LLM is what happens at a higher level. The machine-level instructions being executed are just matrix multiplications. Billions of them. The complexity of LLM behavior is emergent from that.
- elcritch 1y agoOften in physics the equations are already known or can be derived. However, taking a formula, generally a PDE, and solving it efficiently is the real trick. Also as you point out, formulating the equation in terms of core invariants you wish to hold, plays an important part. Finding simplified easy to solve solutions and using them to estimate solutions and using adjustments to get closer to the real solution is a baselime technique. That's the core of the pertubative approach in physics which uses : https://en.wikipedia.org/wiki/Perturbation_theory#:~:text=Perturbation%20theory%20develops%20an%20expression,%22first%2Dorder%22%20perturbative%20correction https://en.wikipedia.org/wiki/Perturbation_theory#:~:text=Pe... However, now it's possible to train AI models to learn much more complex approximations that allow them to run much quicker and more accurately. A prime example is DeepMinds AlphaFold, IMHO. I haven't read up on the research to much, but I'd place firm bets that AI models will be critical in controlling any viable fusion technology.