7 ms·
‘Machine Scientists’ Distill the Laws of Physics from Raw Data
- hollerith 4y agoWhat could go wrong?
- pmoriarty 4y ago"They started by downloading all the equations in Wikipedia. They then statistically analyzed those equations to see what types are most common. This allowed them to ensure that the algorithm’s initial guesses would be straightforward — making it more likely to try out a plus sign than a hyperbolic cosine, for instance. The algorithm then generated variations of the equations using a random sampling method that is mathematically proven to explore every nook and cranny in the mathematical landscape." Wikipedia contains equations from many different branches of math. Aren't mathematical equations written in all sorts of notations (really obscure notations sometimes, for obscure branches of math), depending on the branch of math they're used in? How would this program even be able to use equations written in a notation it doesn't understand? Even if it somehow understood the notation, that doesn't mean the algorithm understood the branch of math the equation was for. This all sounds unworkable to me, unless they're somehow limiting the equations they use to some branch of math they already understand.
- nh23423fefe 4y agoIf you imagine it's nonsense, then you probably don't understand it. You deduced a false conclusion from assumptions you've inserted.
- pmoriarty 4y agoCould you elaborate on this? What assumptions do you object to? Are you saying that different branches of math don't have their own special notations? Sure, they have notation in common, but they have their own notation too, to express objects, relations, or operations that are of special relevance to them. That's not to mention that any given symbol could mean different things depending on the branch of math it's used in.
- kloch 4y agoThis article seems to be about ML but good old fashioned brute force searches are also possible. It's trivial to find formulas that fit data to within the variable's uncertainties. You can easily drown in such output. There are two main challenges with this approach: 1. make sure you have the right "ingredients" (factors) for the search, and 2. filter the output so that human reviewers don't waste time on obvious nonsense equations. Filtering output is not terribly difficult. Some assessment of complexity is required and depending on the formula a symmetry score might be useful. Some actual physics equations are quite complex so you have to be expecting a simple equation or filtering output will be less effective. Making sure you have all of the right potential factors available is very hard. Expanding the input factor set extends the search space and time.
- ChrisRackauckas 4y agoBrute force costs grow exponentially (well, combinatorically) with the number of terms. It's viable for the small cases, but not for the larger cases. There's a nice paper from 2016 which demonstrates this very directly (http://www.cogsys.org/proceedings/2016/paper-2016-2.pdf http://www.cogsys.org/proceedings/2016/paper-2016-2.pdf), though I'm sure there's some stuff from the 70's on it as well.
- YeGoblynQueenne 4y agoAw, no mention of the two Robot Scientists, Adam and Eve, who don't just invent theories but also design and run their own experiments to verify their theories? Wikipedia page: https://en.wikipedia.org/wiki/Robot_Scientist https://en.wikipedia.org/wiki/Robot_Scientist Nature publication: https://www.nature.com/articles/nature02236 https://www.nature.com/articles/nature02236 I guess US media don't realise there's science and technology outside of the Americas eh?
- compacct27 4y agoThe people in the article worked in Catalan, so I guess maybe they do.
- djdsol 4y agoThis is also how a few hedge funds ( WorldQuant and spinoffs ) get alpha signals/factors for their long/short portfolios. They get 95% of their aggregate alpha/returns from Genetic Programming and not human quant researchers.
- programmer_dude 4y agoBrilliant! I am inspired to build my own machine scientists.
- Maursault 4y agoIt's pretty straight forward. You have to build your own computer scientist, first. Then make them cool. Voila!
- tony_cannistra 4y agoI found it curious that one of the implementations of symbolic regression (the "machine scientist" referenced in the article) is a Python wrapper on Julia: https://github.com/MilesCranmer/PySR https://github.com/MilesCranmer/PySR I don't think I've seen a Python wrapper on Julia code before.
- adgjlsfhk1 4y agoThere's also https://github.com/SciML/diffeqpy https://github.com/SciML/diffeqpy which wraps Julia diffeq solvers for python.
- amelius 4y agoGiven that Python's ecosystem is enormous, I would be surprised if everything was duplicated in Julia code. It would also be quite wasteful.
- fennecs 4y agoBasically approximating a solution, and then finding an equation from some data. Nothing wrong with writing down an approximation to some problem. An approximation can be very useful. As long as you know it’s limitations it’s fine to answer questions where it applies. But this can not replace deeper understanding. It’s all good to write down approximate solutions to some hard n body problem or non linear pde, just don’t look too far ahead in time, or update your model with fresh data if you need to.
- riskneutral 4y ago> It’s all good to write down approximate solutions to some hard n body problem or non linear pde, just don’t look too far ahead in time, or update your model with fresh data if you need to. It sounds like this symbolic regression approach can potentially find correct PDE solutions directly from data: "In February, they fed their system 30 years’ worth of real positions of the solar system’s planets and moons in the sky. The algorithm skipped Kepler’s laws altogether, directly inferring Newton’s law of gravitation and the masses of the planets and moons to boot." Note that Newton’s law of gravitation can also be derived from a differential equation. Also note that simply knowing the exact solution to the PDE does not mean that the PDE model itself is an exact representation of reality. For example, we know from Einstein's laws that Newton's laws are not correct at astronomical scales. In the symbolic regression approach, the solution would be limited by the number of different observable variables available to train the model on. Many PDEs in real world problems also do not have any known exact symbolic solution, so approximate symbolic formulas are used routinely anyway but are just painstakingly discovered and derived by humans with "deeper understanding" (actually, just hoards of PhD students and their advisors throwing everything they can think of at the problem until one of them lands on a unusable formula).
- mrow84 4y ago> In the symbolic regression approach, the solution would be limited by the number of different observable variables available to train the model on. What do you mean here? Aren't all approaches limited by the number of observables available (except speculatively)?
- bnewbold 4y agoI'm really curious to see if efforts like this, and from computational augmentation and automation more generally, will yield progress on symbolic modeling of large natural systems. There is a segment in Adam Curtis's "All Watched Over By Machines of Loving Grace" which describes scientists trying to model the ecology of a prairie. As described by Curtis, the more data they collected and more complex their model, the worse the predictions of the model became. It feels like the lessons of the past couple generations (many decades) have been that symbolic models don't compose together easily; symbolic analysis only works in simple systems and has hit diminishing returns; numeric and ML methods work well enough; etc. I'm curious if better tooling (augmentation and/or automation) can push through some of these challenges and yield real understanding. Or if there are some fundamental scaling problems that can not be overcome.
- ThePhysicist 4y agoI worked on genetic programming for my MSc in econometrics, used it to build human-readable risk models for time series data. It worked really well. The article seems to call these methods genetic algorithms, though Koza himself (the "godfather" of genetic algorithms & genetic programming) was pretty keen on distinguishing the two. The main difference is that genetic algorithms represent solutions as linear strings / vectors whereas genetic programming works on tree structures. Maybe just splitting hairs here though.
- pmoriarty 4y ago"The main difference is that genetic algorithms represent solutions as linear strings / vectors whereas genetic programming works on tree structures." Yes, that and that the tree data structure used by genetic programming tends to have functions in every non-leaf node, and static values or variables in the leaf nodes.
- optimalsolver 4y agoIt should also be emphasized that genetic programming is just one approach to program synthesis, i.e. automatically deriving computer programs from data. You don't have to use genetic/evolutionary algorithms to search the space of programs, it's just the most popular method. You can even try pure random search if you're feeling particularly lucky: https://github.com/pySRURGS/pySRURGS https://github.com/pySRURGS/pySRURGS
- nyc111 4y ago> A constant implied that it had identified two proportional quantities — in this case, period squared and distance cubed. In other words, it stopped when it found an equation. Very interesting article. Thanks for posting. I've been thinking about the physics equation and its relation to proportionality. Not all physics equations are proportionalities because in physics the equality sign is loaded. To me, proportionality is fundamental not the physics equation. So I would have written the last sentence of the quote as "...it stopped when it found [a proportionality].
- lagrange77 4y agoAlso have a look at SINDy (Sparse Identification of Nonlinear Dynamics) by Steve Brunton et al. https://youtu.be/NxAn0oglMVw https://youtu.be/NxAn0oglMVw
- amelius 4y agoDo they apply Occam's razor, where simpler equations are favored over more complicated ones?
- franknstein 4y ago"One might define simplicity as the length of the equation, say, and accuracy as how close the curve gets to each point in the data set, but those are just two definitions from a smorgasbord of options." "..the algorithm evaluated candidate equations in terms of how well they could compress a data set. A random smattering of points, for example, can’t be compressed at all; you need to know the position of every dot. But if 1,000 dots fall along a straight line, they can be compressed into just two numbers (the line’s slope and height). The degree of compression, the couple found, gave a unique and unassailable way to compare candidate equations."
- serenitylater 4y ago
- 7thaccount 4y agoYes. The article states something about statistics and Bayesian and other terms I only barely understand. It uses "+" over "hyperbolic cosine" for example (to quote the article).
- mensetmanusman 4y ago“ These algorithms resemble supercharged versions of Excel’s curve-fitting function, except they look not just for lines or parabolas to fit a set of data points, but billions of formulas of all sorts. In this way, the machine scientist could give the humans insight into why cells divide, whereas a neural network could only predict when they do.” It’s very cool witnessing the consequences of Moore’s law; as the cost of billions of calculations goes to pennies, this becomes easier and easier. Since the complexity of nature is unbounded, I wonder if the slow down of Moore’s law will represent a plateau of what can be symbolically discovered. For example, will the next set of scientistic discoveries require quintillions of combinatorial checks that can’t be accomplished in a human lifetime?
- jas- 4y agoWouldn't this be a "what we know about physics" distillation? I am basing this assumption on the "Raw data" as being that of already performed experimentation etc.
- plaguepilled 4y agoFor those interested, this is the relevant article (I think): https://www.science.org/doi/10.1126/sciadv.aav6971 https://www.science.org/doi/10.1126/sciadv.aav6971 - I'm partway through it now and am having a blast.
- eggy 4y agoI am hoping this also opens up more opportunity to leverage Lisp's symbolic powers. I had great fun with Structure and Interpretation of Classical Mechanics (SICM), and recently a paper on analyzing music using symbolic regression [0], and another on symbolic regression with Lisp [1]. Julia and CL seem perfect for this with Mathematica as another option for quickly playing with ideas. [0] https://www.researchgate.net/publication/286905402_Symbolic_regression_as_a_computer-aided_music_tool_for_analysis_and_composition https://www.researchgate.net/publication/286905402_Symbolic_... [1] https://towardsdatascience.com/symbolic-regression-the-forgotten-machine-learning-method-ac50365a7d95 https://towardsdatascience.com/symbolic-regression-the-forgo...
- ChrisRackauckas 4y agoYeah, Julia does great in this domain. DataDrivenDiffEq.jl (https://datadriven.sciml.ai/ https://datadriven.sciml.ai/) is a comprehensive symbolic regression package out there, which pulls together all of what's mentioned in the article (SINDy and variants, EQSearch, SymbolicRegression.jl (which is the core of PySR), OccamNet, etc.) all into a single API to allow comparisons between each of the methods. The Julia automated discovery of physical equations work showcases these symbolic regression techniques quite heavily, for example in the State of SciML talk (https://www.youtube.com/watch?v=eSeY4K4bITI https://www.youtube.com/watch?v=eSeY4K4bITI) we discuss how this has been used now across hundreds of different scientific use cases. The universal differential equations paper which describes the Julia SciML organization (https://arxiv.org/abs/2001.04385 https://arxiv.org/abs/2001.04385) demonstrates these cases where neural networks in differential equations are mixed with symbolic regression to allow for mixing prior physical knowledge with data for discovering just the unknown higher order physical equations. And there's a lot more directions we're going next.
- eggy 4y agoChris, the work of you and others on Julia is nothing short of amazing. Thanks! I tried Julia when it first came out, and then picked it up again to replace my MATLAB habit, and I have not looked back. I can't wait until the sims.jl package and others can be used like Simulink/Modelica.