9 ms·
Machine Learning’s ‘Amazing’ Ability to Predict Chaos
- dangerlibrary 9y agoI was really hoping this was about weather forecasting.
- joshuamorton 9y ago>when the powerful algorithm known as “deep learning” Quanta is normally better than this (and the rest of the article is decent). But still :(
- sgillen 9y agoVery cool! This looks like a very promising way to improve weather forecasts, predict wildfire movements (maybe?), and can maybe be used in control for chaotic systems? (Among other uses of course). It feels like every day I'm seeing deep learning make more mathematical tools obsolete. It's amazing how useful this tool has been.
- freeone3000 9y agoWhen we started saving layers and gained nonlinear functions, it gained the ability to approximate any function, given enough time and data. These are big caveats. RL can function with a lot less data. SVMs can run with a lot less time and space. Partial function application and expert modelling reduce data needed - and some of the best results are from ensemble suites. It's not obsoleting, it's another tool in the box.
- skywhopper 9y agoDon't be too seduced by the enticing ideas at the end of the article. The disconnect here is that success in learning how to predict the results of an algorithmic simulation is not really indicative of how it would perform with the decidedly non-algorithmic natural behavior of weather or earthquakes, phenomena which don't operate in a closed system with predefined limits and parameters. It sounds like the next step, but even if weather were reducible to machine-discoverable patterns, you first must face the need to collect an immense amount of high-resolution condition data from around the globe on an ongoing basis. I'm guessing our current mesh of satellites and weather stations, as impressive as it is, comes nowhere close to enough raw data to sufficiently feed the beast in this case. And even that assumes that we _know about_ and can measure all of the factors that go into weather patterns. I sincerely doubt that's the case.
- IAmEveryone 9y ago(Part of) the idea seems to be synthesizing missing data from later observations of the known data, i. e. "to arrive at this state (A_t,C_t,E_t), the initial state must have been...(A_t-1,B_t-1,C_t-1,E_t-1)..." I have my doubts that this can just overcome the fundamental problem of chaos, but it doesn't sound impossible.
- heavenlyblue 9y agoWell, but then this is just a fancy name for combinatorial optimisation.
- contravariant 9y agoIt's actually kind of unclear what it means to predict results from an algorithmic simulation. Surely that's just trying to use a different method to simulate the process? I also don't see them mentioning anything about noise, so what exactly are they trying to achieve? Surely perfect prediction should already be possible, otherwise what are they comparing the output to?
- joe_the_user 9y ago
- XnoiVeX 9y agoIsn't deterministic-chaos an oxymoron?
- nabla9 9y agoNo. Chaotic dynamical system can be completely deterministic.
- throwawayjava 9y agoThe fact that the answer to this is "no" has fed entire subfields of mathematics for a generation.
- imh 9y agoThe specific kind of chaos here implies (roughly) that a system starting in state X and a system starting arbitrarily close to X will at some point in the future behave totally differently. Not different as in "they'll drift further apart," but different as in "this one rolls off the cliff and this doesn't." It's a fascinating field of dynamical systems, which is one of the coolest subfields of math (since the world is dynamic). Here's the wikipedia page https://en.wikipedia.org/wiki/Chaos_theory https://en.wikipedia.org/wiki/Chaos_theory
- mljoe 9y agoNot only can chaos be exhibited in fully deterministic systems, it can be exhibited in absurdly simple systems. The logistic map is an incredibly simple iterated function that exhibits highly chaotic behavior. It remains one of most mysterious things in mathematics, as far as I can tell. Studying it has consumed a lot of very smart people's careers.
- chadcmulligan 9y agoThe double pendulum is a nice example https://en.wikipedia.org/wiki/Double_pendulum https://en.wikipedia.org/wiki/Double_pendulum
- jimmytidey 9y agoCan we have more publishing like Quanta magazine? It's just perfect in terms of reporting from the frontier, being easy to unsetstand, and not talking down to you.
- greeneggs 9y agoIt's funded by Jim Simons, of the hedge fund Renaissance Technologies. A mathematician with money, trying to promote mathematics. Quanta magazine is a fantastic service. But the other Renaissance magnate, Bob Mercer, as his own media hobby funded Breitbart News.
- wybiral 9y ago> But the other Renaissance magnate, Bob Mercer, as his own media hobby funded Breitbart News. And was one of the major sources of funds behind Cambridge Analytica.
- internetman55 9y agoYeah it's kind of sad to me. Quanta is amazing but I'm not sure if I could reasonably expect to exist without Simons or someone like him privately funding it
- mkempe 9y agoWhy do you feel sad about the fact that an individual who values science is "privately funding" a great publication? surely you don't object to private wealth and/or activity?
- plg 9y agoeven after cross-validation?
- shas3 9y agoNovel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonlinear systems, including human perception. Given this, I think the OP paper is a low-hanging fruit. It was only a matter of time before someone figured out a way to learn specific chaotic nonlinear dynamical systems. Nice work nonetheless.
- IAmEveryone 9y agoI'm not sure if human perception is a chaotic system. Chaos is defined as "small change in input -> large change in output". Perception is actually the opposite, with small input changes (changes in light, different angles,...) leading to fundamentally unchanged perception ("It's a tree").
- jessriedel 9y agoFirst, sensitivity to initial conditions is a necessary but not sufficient property for chaos. Some sort of folding/mixing is also necessary, which can be gauranteed by a bounded state space. Second, it's definitely true that information processing systems, if they are to be reliable enough to be useful, are not going to be chaotic throughout their state space. They need to return the same output given a certain input. But I'd imagine there are also at least a few noisy regions.
- perl4ever 9y agoI always thought that a perception like "it's a tree" that remains stable could possibly be an attractor in a chaotic system. If you look at the trajectory of a particle around an attractor, its position is very unpredictable after a while, but which attractor it is orbiting is not so random or unstable. Thinking of the visual of the Lorenz attractor.
- selimthegrim 9y agoYou need also ergodicity and mixing, as jessriedel states, meaning respectively that you explore every area of your phase space with equal probability, and that two trajectories that begin arbtrarily close together do in fact diverge instead of sticking together, at infinite time
- deleted 9y ago[deleted]
- lewis500 9y agoI want to know how this does with traffic. Pretty remarkable.
- joshuaheard 9y agoI'd like to apply it to the stock market.
- aatchb 9y agoErgodicity becomes a problem here. The nonlinear systems typically studied by complexity and chaos theorists have strong fixed rules that do not change in time. Markets have systematic and structural changes which can make prior observations completely irrelevant.
- leot 9y agoCan anyone speak to whether PRNGs might be affected?
- folli 9y agoIf you know the seed and the used algorithm, you are already able to "predict" the pseudo-random sequence perfectly.
- deleted 9y ago[deleted]
- downandout 9y agoIt would be fascinating to apply this technique to card shuffling. Shuffling techniques are not even close to random...casinos hope for chaos at best. It would be interesting to see if a machine learning algorithm could come up with an approach that could be carried out by humans to, for example, predict whether or not the next shoe of a hand-shuffled blackjack game is going to have a positive or negative expectation for the player.
- curuinor 9y agoYou can card count on apps already. The repeated linear operator + nonlinearity of the ESN (the reservoir) could implement a card-counting memory but, you know, so could a card counting memory in a normal program.
- downandout 9y agoOh I wasn't talking about card counting. I was talking about being able to determine, knowing the order of the cards going into the shuffle, whether or not the next shoe (after the shuffle) would have an overall positive or negative expectation. The theoretical expectation accounting for every single possible order of cards is slightly negative. However, if a massive percentage of the possible orders that the cards can be in is eliminated, the expectation may either be much more positive or negative than the overall theoretical return.
- amelius 9y agoI'm curious, has anyone tried to apply ML to solving large linear systems?
- freeone3000 9y agoYes. Vladmir Vaptnik (1995)'s work on SVMs is probably the most recent, but linear regression is an ML technique as well.
- d--b 9y agoDoesn't that mean that the neural net learned itself a numerical method to compute the solution of the equation, and that it is close enough in terms of approximation up to 7 Lyapukov times, and after that time, the approximation becomes not good enough and the system can't predict? It doesn't sound like too groundbreaking...
- ppod 9y agoI think it's blurring the line between modelling and simulation. You can find an efficient route down a hillside by pouring water down it, or a line of least resistance through a system by passing a current through it. Is the system learning a numerical solution, or performing one? I think this is like building a model of a system that is much closer to the territory than the map compared to a normal model, but still easier to work with than the territory.
- aeorgnoieang 9y ago> Is the system learning a numerical solution, or performing one? Is there a real difference? Any learning must be, fundamentally, an algorithm, so learning is performing.
- ppod 9y agoThis is a deep topic but one good treatment of is in the works of David Deutsch: https://www.cs.indiana.edu/~dgerman/hector/deutsch.pdf https://www.cs.indiana.edu/~dgerman/hector/deutsch.pdf
- jonplackett 9y agoIf there's an algorithm to create this 'randomness' then how can it be all that random? Isn't it just learning to make an approximation of that algorithm, rather than actually predicting chaos?
- empath75 9y agoIt’s working with imperfect information about the current state, which makes the algorithm that generates it useless.
- Swanlyk2 9y agoWhy wouldn't it? A lot of misunderstanding out there in both machine learning and nonlinear dynamics.