6 ms·
They did most of that back in 2015.[1] This is version 2. It's interesting what's happening as the control theory people get into machine learning. The contro
by Animats 7y ago
They did most of that back in 2015.[1] This is version 2.
It's interesting what's happening as the control theory people get into machine learning. The controls people don't typically run a neural net as a controller. They use the trained net as a tool for building a controller with known continuity properties. The trouble with pure neural net controllers is that they sometimes do something totally bogus for some data point within the normal input space. That's not OK in control systems.
This has all the math of machine learning plus the math of control theory and I don't understand it, although I sometimes look at the papers.
[1] https://news.stanford.edu/2015/10/20/marty-autonomous-delorean-102015/ https://news.stanford.edu/2015/10/20/marty-autonomous-delore...
- AndrewKemendo 7y ago>They use the trained net as a tool for building a controller with known continuity properties. This sentence and comment is a revelation to me so thanks! I was very into control theory from 2001-2006 before jumping into Bayes nets and eventually ML. Having not really returned to CT since, it's always been a question in my mind how to use ML for SCADA & PLCs without running into the problems that you describe. As with most problems like this, it seems obvious now that it's described as such - though not actually obvious to those like me outside of it!
- sillysaurusx 7y agoI would highly recommend https://www.youtube.com/watch?v=PYylPRX6z4Q https://www.youtube.com/watch?v=PYylPRX6z4Q as a way of framing the problem. It seems like our options for ML are currently: self-play, dataset fitting, or human feedback. It's possible to solve any problem that can fit into one of those molds. (Unfortunately the opposite is true: if a problem doesn't fit into those categories, it seems very hard to solve with any form of ML.) But, now that I think about it a bit more, the sentence "use a trained net as a tool for building a controller with known continuity properties" seems neither obvious nor straightforward to do. I'm decently well-versed in AI techniques at this point, and I can't really think how I'd sit down and write a program that would generate such a system. Anyone have any leads? Basically, the goal is to make a neural net that helps you design a system that can be predictable. One way would be to write a loss function that penalizes unpredictability, and then let it run until it converges on a predictable control system. But that's a bit like saying "Just draw an owl." Not so easy to think of the actual code to do that.
- Animats 7y agoOne approach is that you have a semi-traditional control system, which has integrators, filters, adders, etc. All these components have tuning parameters, and tuning complex controllers is hard. But once you get them tuned, they behave in a reasonably predictable way. The machine learning part is used to get a predictor. You operate the thing and train a model of what will happen for various inputs. The machine learning system is just an observer in this training phase; it doesn't control anything. You train from recorded data. Then you use the trained model to tune the controller. The model lets you get an output for any set of inputs, so you can now choose input test sets which are suitable for tuning the model, like changing one input at a time and noting the output change. Automatically generating the structure of the controller (how those blocks are connected) is a separate problem. That's called "system identification". See [1]. [1] http://www.mpc.berkeley.edu/research/adaptive-and-learning-predictive-control http://www.mpc.berkeley.edu/research/adaptive-and-learning-p...
- spockz 7y agoSo what is the difference of using this approach with exploring the set of inputs to find the optimal one using a search algorithm like A* or the many different ones? I’ve been struggling to find where machine learning starts and what is just good old statistics (used in data mining for ages) and search algorithms like A*.
- mdiesel 7y agoThey serve the same purpose (optimisation). The advantage of ML is that you don't need to be able to define an objective function instead you just have a data set.
- tcoff91 7y agoPerhaps it could just try and learn the parameters to some kind of pre existing control theory algorithm? Perhaps with a simulator it could fall into the first category you mentioned.
- jsinai 7y ago
- godelski 7y agoAnyone have more technical information on this? This sounds really cool.
- charliepark 7y ago> "The trouble with pure neural net controllers is that they sometimes do something totally bogus for some data point within the normal input space." A friend's theory on Google Maps is that it sometimes intentionally sends you down a non-optimal path in order to gather data on alternate routes. This friend isn't in the ML space, so this is just their anecdotal observation. Are they right, then?
- allovernow 7y agoUnlikely. Usually a suboptimal route is chosen when there's traffic somewhere else. Sometimes traffic might clear up after the path is chosen. Its an appropriate theory though, given that this is Google we're talking about.
- perl4ever 7y agoThere is a class of routes I take, which involve going through a cross street between two east-west roads. Google Maps sometimes directs me to take one that does not end in a traffic light, which would leave me at a stop sign trying to make a left onto a major road across traffic. The last time I was stuck in that position, I basically made a right and then a U-turn.
- wongarsu 7y ago"Alternate routes" are still normal roads people travel on. Since the travel time of a path is just the sum of the travel time of all path segments you only need people traveling every road to estimate the total time of any path. Path finding is well understood with many algorithmic solutions, so there is no reason to use machine learning. Google's implementation is impressive, but I see no reason why it would benefit from people traveling suboptimal paths.
- taneq 7y agoIt doesn't have to be doing gradient descent on a neural network to be machine learning. You could easily generate small perturbations to generated routes if they didn't add much time and took the driver down an under-monitored road.
- allovernow 7y ago>The trouble with pure neural net controllers is that they sometimes do something totally bogus for some data point within the normal input space. That's not OK in control systems. Is that actually true? Everything we do at work with neural nets seems to indicate continuity. We do stuff with mathematical simulations and imagelike processing. Latent space for auto encoders tend to be smoothly varying throughout the range of outputs. I thought the problem is just that it isn't proven that neural nets are sufficiently smooth.
- mjburgess 7y agoIt's not a continuity problem. It's that "bogus" is defined with respect to what it should do, and that criteria requires a working understanding of the environment which isn't present in the model. If you had all possible images of all possible roads in all possible (lighting, weather,..) conditions, etc. then the ideal behaviour is just a function from all inputs to all desired actions. In the absence of this infinity of data you need a model. Any model which has no explicit understanding of the causal behaviour of an environment is going to misbehave for one of those unseen data points (eg. an unseen road in a weather/lighting/etc. condition). "Correct"(/Safe) behaviour is not a function of pixels. No statistical model which associates pixel patterns with action can be correct.
- allovernow 7y ago>Any model which has no explicit understanding of the causal behaviour of an environment is going to misbehave for one of those unseen data points (eg. an unseen road in a weather/lighting/etc. condition I disagree. That sounds like an older interpretation of behavior of simple deep nets like the inception family. My experience with auto encoders and GANs is such that the nets can and unquestionably do interpolate between training data points. What's more, the latent spaces display order reflecting logic - much in the way that you can perform arithmetic on BERT encodings in what amounts to a form of logic. This is the where SOTA is now and I think it's a strong step towards AI, though we're still far from it. The trick is in understanding the boundaries in high dimensional space that your training data represents. So there is some degree of covering all your bases, so to speak, and that requires a new kind of intuition. These are very exciting times in tech/ML.
- pjc50 7y agoThat was roughly the thesis of Fuzzy Logic (Kosko 1993); complex control systems that used neural nets during design to extract "rules" from the system under control.
- elfexec 7y ago> They did most of that back in 2015.[1] Just in time for the 30th anniversary of back to the future. Nice.
- octoboto 7y agoWhat papers are you talking about? Can you please link to them? I don't see anything about neural networks in the papers about this experiment.