4 ms·

This is slightly off-topic, but I noticed that you did work on stochastic optimal control. Do you have any books you would recommend on the subject (either opt

This is slightly off-topic, but I noticed that you did work on stochastic optimal control. Do you have any books you would recommend on the subject (either optimal control or stochastic optimal control)? Ideally as good as Halmos. :) I come from a physics/cs background, and find that standard treatments of control are very much intended for EE folks. But this seems like a historical accident-- the techniques would seem to be very broadly useful in other fields.

NY_USA_Hacker 15y ago
If you want something written as well as Halmos, then start writing, and good luck! "Historical accident": Well, sure, in part nearly all the pure math departments pushed out any such topics! You are correct: Optimal control has been mostly in advanced parts of electrical engineering. So, optimal control in EE was an example doing math outside math departments. Of math done outside math departments, control theory is relatively good mathematically. And, yes, there should be applications elsewhere. For physics, yes, the deterministic theory of optimal control has been seen as replacing the older calculus of variations which goes back to Newton. My references are old. More recent work on stochastic optimal control has been by R. T. Rockafellar at University of Washington. For Stuart E. Dreyfus and Averill M. Law, 'The Art and Theory of Dynamic Programming', ISBN 0-12-221860-4, Academic Press, New York. 'dynamic programming' is a big part of the discrete time versions of optimal control. It can be stochastic or deterministic. The linear-quadratic-Gaussian (linear 'plant' or system, quadratic cost to be minimized, and Gaussian exogenous random variables) has 'deterministic equivalence' -- nice -- and this book treats it. The book is a good, elementary start. Apparently Dreyfus was a R. Bellman student. For L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, 'The Mathematical Theory of Optimal Processes', ISBN 0-470-69381-9, Interscience Publishers, John Wiley & Sons, New York. it was, of course, the main source of the Pontryagin maximum principle, and, thus, a swift kick in the back side for parts of US aerospace in the 1960s. For Michael Athans and Peter L. Falb, 'Optimal Control: An Introduction to the Theory and Its Applications', McGraw-Hill Book Company, New York. Athans was long in EE at MIT and did some military work, e.g., on parts of the C5A airplane. Falb was at Brown's Division of Applied Mathematics. One Athans story was the 'control' for least time to climb to, say, 100,000 feet for an F-4: Go up to a few thousand feet, go into a dive, get supersonic, get the lower drag of a few hundred knots above Mach 1, and then with the lower drag continue supersonic to the final altitude. I was not able to know if that 'control' idea was just intuitive or directly from computation and the Pontryagin maximum principle which is, after all, just a necessary condition, i.e., local optimality. Also in that division at Brown, see the papers of Harold Kushner. As I recall, he wrote out a stochastic version of the Pontryagin maximum principle. For E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin. there are connections with economic planning. For Dimitri P. Bertsekas and Steven E. Shreve, 'Stochastic Optimal Control: The Discrete Time Case', ISBN 0-12-093260-1, Academic Press, New York. the math is done carefully. So there is a lot of attention to measurability. Part of the reason is the issue of 'measurable selection': This can be a deep subject, but often a relatively simple way out is via regular conditional probabilities as in Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM, Philadelphia. For David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York. this has likely the easiest mathematical treatment of deterministic optimal control and also Kalman filtering and also the math needed for 'least action' in physics. For E. B. Lee and L. Markus, 'Foundations of Optimal Control Theory', ISBN 0471-52263-5, John Wiley & Sons, New York. this tried to be clean mathematically when it was written. At the beginning, should know some relatively advanced results in ordinary differential equations, e.g., as in Earl A. Coddington and Norman Levinson, 'Theory of Ordinary Differential Equations', McGraw-Hill, New York. For more, you can consider non-linear filtering and connections with mathematical finance. The above is just from my bookshelf. Likely now a better bibliography could be assembled via the Internet which actually is at least a little larger than my bookshelf!
- lliiffee 15y ago
  Thank you thank you thank you!

NY_USA_Hacker 15y ago

If you want something written as well as Halmos, then start writing, and good luck! "Historical accident": Well, sure, in part nearly all the pure math departments pushed out any such topics! You are correct: Optimal control has been mostly in advanced parts of electrical engineering. So, optimal control in EE was an example doing math outside math departments. Of math done outside math departments, control theory is relatively good mathematically. And, yes, there should be applications elsewhere. For physics, yes, the deterministic theory of optimal control has been seen as replacing the older calculus of variations which goes back to Newton. My references are old. More recent work on stochastic optimal control has been by R. T. Rockafellar at University of Washington. For Stuart E. Dreyfus and Averill M. Law, 'The Art and Theory of Dynamic Programming', ISBN 0-12-221860-4, Academic Press, New York. 'dynamic programming' is a big part of the discrete time versions of optimal control. It can be stochastic or deterministic. The linear-quadratic-Gaussian (linear 'plant' or system, quadratic cost to be minimized, and Gaussian exogenous random variables) has 'deterministic equivalence' -- nice -- and this book treats it. The book is a good, elementary start. Apparently Dreyfus was a R. Bellman student. For L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mischenko, 'The Mathematical Theory of Optimal Processes', ISBN 0-470-69381-9, Interscience Publishers, John Wiley & Sons, New York. it was, of course, the main source of the Pontryagin maximum principle, and, thus, a swift kick in the back side for parts of US aerospace in the 1960s. For Michael Athans and Peter L. Falb, 'Optimal Control: An Introduction to the Theory and Its Applications', McGraw-Hill Book Company, New York. Athans was long in EE at MIT and did some military work, e.g., on parts of the C5A airplane. Falb was at Brown's Division of Applied Mathematics. One Athans story was the 'control' for least time to climb to, say, 100,000 feet for an F-4: Go up to a few thousand feet, go into a dive, get supersonic, get the lower drag of a few hundred knots above Mach 1, and then with the lower drag continue supersonic to the final altitude. I was not able to know if that 'control' idea was just intuitive or directly from computation and the Pontryagin maximum principle which is, after all, just a necessary condition, i.e., local optimality. Also in that division at Brown, see the papers of Harold Kushner. As I recall, he wrote out a stochastic version of the Pontryagin maximum principle. For E. B. Dynkin and A. A. Yushkevich, 'Controlled Markov Processes', ISBN 0-387-90387-9, Springer-Verlag, Berlin. there are connections with economic planning. For Dimitri P. Bertsekas and Steven E. Shreve, 'Stochastic Optimal Control: The Discrete Time Case', ISBN 0-12-093260-1, Academic Press, New York. the math is done carefully. So there is a lot of attention to measurability. Part of the reason is the issue of 'measurable selection': This can be a deep subject, but often a relatively simple way out is via regular conditional probabilities as in Leo Breiman, 'Probability', ISBN 0-89871-296-3, SIAM, Philadelphia. For David G. Luenberger, 'Optimization by Vector Space Methods', John Wiley and Sons, Inc., New York. this has likely the easiest mathematical treatment of deterministic optimal control and also Kalman filtering and also the math needed for 'least action' in physics. For E. B. Lee and L. Markus, 'Foundations of Optimal Control Theory', ISBN 0471-52263-5, John Wiley & Sons, New York. this tried to be clean mathematically when it was written. At the beginning, should know some relatively advanced results in ordinary differential equations, e.g., as in Earl A. Coddington and Norman Levinson, 'Theory of Ordinary Differential Equations', McGraw-Hill, New York. For more, you can consider non-linear filtering and connections with mathematical finance. The above is just from my bookshelf. Likely now a better bibliography could be assembled via the Internet which actually is at least a little larger than my bookshelf!

lliiffee 15y ago
Thank you thank you thank you!