4 ms·

This looks neat, but anyone interested should know there's also a whole mature field to solve similar problems. The field of control theory has some really simp

This looks neat, but anyone interested should know there's also a whole mature field to solve similar problems. The field of control theory has some really simple and really robust methods for getting a system from state A to state B. If you aren't trying to learn how the system works simultaneous to your algorithm being deployed, it can be a really a good fit. This book and control theory solve different problems. I'm just commenting because I sometimes see modern ML folks trying to apply super duper general techniques to problems that are easily solved specifically with simple undergrad-level techniques from the specific fields (including this material to control problems). Also, I just think control theory is really cool and perhaps under-discussed in "cool math/eng stuff" circles.

dimatura 6y ago
After the basic PID controller, what would you consider the most useful tools for control?
- tshaddox 6y ago
  I know virtually nothing about control theory, but when I was into RC quadcopters back around 2011 I know Kalman filters got name dropped a lot. https://en.wikipedia.org/wiki/Kalman_filter https://en.wikipedia.org/wiki/Kalman_filter
  - aiprof 6y ago
    Kalman filters and variations are covered in Ch. 19. It can be viewed as a special case of belief updating in a partially observable Markov decision process.
- thx2099100 6y ago
  the next step after PID control is state-space based control. it's not a single tool but more of a framework. it goes beyond frequency based representations and can also model non-linear control problems. the kalman filter is an example of the use of this framework where it's combined with statistics. the problem in control theory though is that once you want to go beyond linear control things get very difficult, most of the literature seems to be about finding clever ways of approximating your problem into something linear.
- 6gvONxR4sf7o 6y ago
  Check out model predictive control. And LQR if you haven’t learned that yet. And the bazillion state estimation techniques that start with Kalman filtering variants (eg UKF) and progressively account more and more for nonlinearities.
ajfriend 6y ago
Can you recommend any good books or other resources?
- jamessb 6y ago
  Ben Recht [0] has worked on control theory and reinforcement learning. See his blog post "What we've Learned to control" [1] and the survey paper "A Tour of Reinforcement Learning: The View from Continuous Control" [2] may be of interest. [0]: https://people.eecs.berkeley.edu/~brecht/ https://people.eecs.berkeley.edu/~brecht/ [1]: https://www.argmin.net/2020/06/29/tour-revisited/ https://www.argmin.net/2020/06/29/tour-revisited/ [2]: https://arxiv.org/abs/1806.09460 https://arxiv.org/abs/1806.09460
- 6gvONxR4sf7o 6y ago
  I learned it from a course where my professor had his own notes printed by the school bookstore for us, and I've held onto those notes like they're gold. So I can't refer you anything by firsthand knowledge, but the internet seems to like Dorf & Bishop's Modern Control Systems, which seems like a decent coverage of the basics: https://smile.amazon.com/Modern-Control-Systems-13th-Richard-dp-0134407628/dp/0134407628/ https://smile.amazon.com/Modern-Control-Systems-13th-Richard...
aiprof 6y ago
You are right that sometimes people use much more sophisticated algorithms than are really required to solve a problem. My experience is that the simplest approach is often the best one in the long term. There are actually deep connections between problems in control theory and the topics in this book (e.g., LQR control), though these different communities often use different notation. The focus of the book is more on higher-level decision making, but the execution of the decisions---such as motor commands to an actuator---are often best done through PID or some other method that can be found in a control theory textbook.
deleted 6y ago
[deleted]
eli_gottlieb 6y ago
Control theory is indeed, quite cool, and under-discussed in most circles. Also, unfortunately, classical control theory is primarily good for linear time-invariant dynamics in the frequency space of the Laplace transform. If you can't locally linearize your model and/or need to learn a model, classical control approaches are underdeveloped, and everyone has switched to optimal control and RL.
- 6gvONxR4sf7o 6y ago
  I think you might mean something different by optimal control than I would. I'd bucket that into control theory, but it seems you're drawing a distinction.
  - eli_gottlieb 6y ago
    I would distinguish between classical control and optimal control. In the former, you don't have a (Hamilton-Jacobi-)Bellman equation to solve, while in the latter, the approximate dynamic programming is The Hard Part.
graycat 6y ago
For control theory from a mathematician, Fleming, long at the Division of Applied Mathematics at Brown University, there is: Wendell H. Fleming and Raymond W. Rishel, Deterministic and Stochastic Optimal Control, ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979. There is more on optimization, especially from the Hahn-Banach theorem, and also Kalman filtering, in David G. Luenberger, Optimization by Vector Space Methods, John Wiley and Sons, Inc., New York, 1969.
somethoughts 6y ago
This is neat - specifically because what you point out is actually true. The primary author is a professor and the secondary author is a Phd graduate - both in the Stanford Aeronautics and Astronautic Department [1][2] - in the Dynamics and Control Group. A brief skim through of the book shows many references to Control Theory texts. [1] https://mykel.kochenderfer.com https://mykel.kochenderfer.com [2] http://timallanwheeler.com/aboutme/cv/Wheeler_CV_201804.pdf http://timallanwheeler.com/aboutme/cv/Wheeler_CV_201804.pdf
- jonathanstrange 6y ago
  That explains why there is not as much traditional Decision Theory in the book as one would expect from the title, such as Peter C. Fishburn's work, issues in MAUT and MCDM, seminal work by Bouyssou, Vincke, Pirlot, Keeney, or the older discussions about SEU vs. EU, evidential vs. causal decision theory, etc. It's a rather technical book, focusing on issues like advanced sensor data fusion, control theory, and optimization. Not that there is anything wrong with that, the title is just slightly misleading.

dimatura 6y ago

After the basic PID controller, what would you consider the most useful tools for control?

tshaddox 6y ago
I know virtually nothing about control theory, but when I was into RC quadcopters back around 2011 I know Kalman filters got name dropped a lot. https://en.wikipedia.org/wiki/Kalman_filter https://en.wikipedia.org/wiki/Kalman_filter
- aiprof 6y ago
  Kalman filters and variations are covered in Ch. 19. It can be viewed as a special case of belief updating in a partially observable Markov decision process.
thx2099100 6y ago
the next step after PID control is state-space based control. it's not a single tool but more of a framework. it goes beyond frequency based representations and can also model non-linear control problems. the kalman filter is an example of the use of this framework where it's combined with statistics. the problem in control theory though is that once you want to go beyond linear control things get very difficult, most of the literature seems to be about finding clever ways of approximating your problem into something linear.
6gvONxR4sf7o 6y ago
Check out model predictive control. And LQR if you haven’t learned that yet. And the bazillion state estimation techniques that start with Kalman filtering variants (eg UKF) and progressively account more and more for nonlinearities.

ajfriend 6y ago

Can you recommend any good books or other resources?

jamessb 6y ago
Ben Recht [0] has worked on control theory and reinforcement learning. See his blog post "What we've Learned to control" [1] and the survey paper "A Tour of Reinforcement Learning: The View from Continuous Control" [2] may be of interest. [0]: https://people.eecs.berkeley.edu/~brecht/ https://people.eecs.berkeley.edu/~brecht/ [1]: https://www.argmin.net/2020/06/29/tour-revisited/ https://www.argmin.net/2020/06/29/tour-revisited/ [2]: https://arxiv.org/abs/1806.09460 https://arxiv.org/abs/1806.09460
6gvONxR4sf7o 6y ago
I learned it from a course where my professor had his own notes printed by the school bookstore for us, and I've held onto those notes like they're gold. So I can't refer you anything by firsthand knowledge, but the internet seems to like Dorf & Bishop's Modern Control Systems, which seems like a decent coverage of the basics: https://smile.amazon.com/Modern-Control-Systems-13th-Richard-dp-0134407628/dp/0134407628/ https://smile.amazon.com/Modern-Control-Systems-13th-Richard...

aiprof 6y ago

You are right that sometimes people use much more sophisticated algorithms than are really required to solve a problem. My experience is that the simplest approach is often the best one in the long term. There are actually deep connections between problems in control theory and the topics in this book (e.g., LQR control), though these different communities often use different notation. The focus of the book is more on higher-level decision making, but the execution of the decisions---such as motor commands to an actuator---are often best done through PID or some other method that can be found in a control theory textbook.

deleted 6y ago

[deleted]

eli_gottlieb 6y ago

Control theory is indeed, quite cool, and under-discussed in most circles. Also, unfortunately, classical control theory is primarily good for linear time-invariant dynamics in the frequency space of the Laplace transform. If you can't locally linearize your model and/or need to learn a model, classical control approaches are underdeveloped, and everyone has switched to optimal control and RL.

6gvONxR4sf7o 6y ago
I think you might mean something different by optimal control than I would. I'd bucket that into control theory, but it seems you're drawing a distinction.
- eli_gottlieb 6y ago
  I would distinguish between classical control and optimal control. In the former, you don't have a (Hamilton-Jacobi-)Bellman equation to solve, while in the latter, the approximate dynamic programming is The Hard Part.

graycat 6y ago

For control theory from a mathematician, Fleming, long at the Division of Applied Mathematics at Brown University, there is: Wendell H. Fleming and Raymond W. Rishel, Deterministic and Stochastic Optimal Control, ISBN 0-387-90155-8, Springer-Verlag, Berlin, 1979. There is more on optimization, especially from the Hahn-Banach theorem, and also Kalman filtering, in David G. Luenberger, Optimization by Vector Space Methods, John Wiley and Sons, Inc., New York, 1969.

somethoughts 6y ago

This is neat - specifically because what you point out is actually true. The primary author is a professor and the secondary author is a Phd graduate - both in the Stanford Aeronautics and Astronautic Department [1][2] - in the Dynamics and Control Group. A brief skim through of the book shows many references to Control Theory texts. [1] https://mykel.kochenderfer.com https://mykel.kochenderfer.com [2] http://timallanwheeler.com/aboutme/cv/Wheeler_CV_201804.pdf http://timallanwheeler.com/aboutme/cv/Wheeler_CV_201804.pdf

jonathanstrange 6y ago
That explains why there is not as much traditional Decision Theory in the book as one would expect from the title, such as Peter C. Fishburn's work, issues in MAUT and MCDM, seminal work by Bouyssou, Vincke, Pirlot, Keeney, or the older discussions about SEU vs. EU, evidential vs. causal decision theory, etc. It's a rather technical book, focusing on issues like advanced sensor data fusion, control theory, and optimization. Not that there is anything wrong with that, the title is just slightly misleading.