4 ms·

Hm I didn't know different areas of control theory were siloed. Learning about control theory in graduate school was awesome and it seems like a field that woul

Hm I didn't know different areas of control theory were siloed. Learning about control theory in graduate school was awesome and it seems like a field that would benefit from ML a lot. I know they use RL agents for control networks for e.g. cartpole, but I would've thought it would be more widespread! Do you think the development of Differentiable Programming (i.e. the observation of more generality beyond pure ML/DL) was really the missing piece? Also, just curious, what are your studies in?

melony 5y ago
Control theory has a very, very long parallel history alongside ML. ML, specifically probabilistic and reinforcement learning, uses a lot of dynamic programming ideas and Bellman equations in its theoretical modeling. Lookup the term cybernetics, it is an old term in the pre-internet era to mean control theory and optimization. The Soviets even had a grand scheme to build networked factories that could be centrally optimized and resource allocated. Their Slavic communist AWS-meets-Walmart efforts spawned a Nobel laureate; Kantorovich was given the award for inventing linear programming. Unfortunately the CS field is only just rediscovering control theory while it has been a staple of EE for years. However, there haven't been many new innovations in the field until recently when ML became the new hottest thing.
- SleekEagle 5y ago
  This is some insanely cool history! I had no idea the Soviets had such a technical vision, that's actually pretty amazing. I've heard the term "cybernetics" but honestly just thought it was some movie-tech term, lol. It seems really weird that control theory is in EE departments considering it's sooo much more mathematical than most EE subdisciplines except signals processing. I remember a math professor of mine telling us about optimization techniques that control systems practitioners would know more about than applied mathematicians because they were developed specifically for the field, can't remember what the techniques were though ...
  - dreamcompiler 5y ago
    Laplace transforms are one such trick. Given a linear differential equation describing your system, Laplace transforms let you solve it using basic algebra. Unfortunately this doesn't work on nonlinear systems.
    - melony 5y ago
      The reduction of convolution to FFT is beautiful ;)
  - melony 5y ago
    There is this excellent HN-recommended fiction called Red Plenty that dramatised the efforts on the other side of the Atlantic. https://news.ycombinator.com/item?id=8417882 https://news.ycombinator.com/item?id=8417882 > It seems really weird that control theory is in EE departments considering it's sooo much more mathematical than most EE subdisciplines except signals processing. I agree, apparently Bellman's reasoning for calling dynamic programming what it is was because he needed grant funding during the Cold War days and was advised to give his mathematical theories a more "interesting" name. https://en.m.wikipedia.org/wiki/Dynamic_programming#History https://en.m.wikipedia.org/wiki/Dynamic_programming#History The generalised form of the Bellman Equation (co-formulated by Kalman of the Kalman filters fame) to control theory and EE is in some ways what the Maximum Likelihood function is to ML. https://en.m.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%93Bellman_equation https://en.m.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%8...
    - SleekEagle 5y ago
      Looks really cool, added to my amazon cart. Thanks for the rec! That hilarious and sadly insightful. I remember thinking "what the hell is so 'dynamic' about this?" the first time I learned about dynamic programming. Although "memoitative programming" sounds pretty fancy too, lol
- erwincoumans 5y ago
  Interesting, I didn't know about cybernetics. For the ones interested there is a book that discusses both: 'Reinforcement Learning and Optimal Control', by Dimitri P. Bertsekas. It covers exact and approximate Dynamic Programming, finite and infinite horizon problems, deterministic and stochastic models, model-based and model-free optimization. Aside from this book, Ben Recht has some interesting blog about Optimal Control and Reinforcement learning: http://www.argmin.net/2018/06/25/outsider-rl http://www.argmin.net/2018/06/25/outsider-rl

melony 5y ago

Control theory has a very, very long parallel history alongside ML. ML, specifically probabilistic and reinforcement learning, uses a lot of dynamic programming ideas and Bellman equations in its theoretical modeling. Lookup the term cybernetics, it is an old term in the pre-internet era to mean control theory and optimization. The Soviets even had a grand scheme to build networked factories that could be centrally optimized and resource allocated. Their Slavic communist AWS-meets-Walmart efforts spawned a Nobel laureate; Kantorovich was given the award for inventing linear programming. Unfortunately the CS field is only just rediscovering control theory while it has been a staple of EE for years. However, there haven't been many new innovations in the field until recently when ML became the new hottest thing.

SleekEagle 5y ago
This is some insanely cool history! I had no idea the Soviets had such a technical vision, that's actually pretty amazing. I've heard the term "cybernetics" but honestly just thought it was some movie-tech term, lol. It seems really weird that control theory is in EE departments considering it's sooo much more mathematical than most EE subdisciplines except signals processing. I remember a math professor of mine telling us about optimization techniques that control systems practitioners would know more about than applied mathematicians because they were developed specifically for the field, can't remember what the techniques were though ...
- dreamcompiler 5y ago
  Laplace transforms are one such trick. Given a linear differential equation describing your system, Laplace transforms let you solve it using basic algebra. Unfortunately this doesn't work on nonlinear systems.
  - melony 5y ago
    The reduction of convolution to FFT is beautiful ;)
- melony 5y ago
  There is this excellent HN-recommended fiction called Red Plenty that dramatised the efforts on the other side of the Atlantic. https://news.ycombinator.com/item?id=8417882 https://news.ycombinator.com/item?id=8417882 > It seems really weird that control theory is in EE departments considering it's sooo much more mathematical than most EE subdisciplines except signals processing. I agree, apparently Bellman's reasoning for calling dynamic programming what it is was because he needed grant funding during the Cold War days and was advised to give his mathematical theories a more "interesting" name. https://en.m.wikipedia.org/wiki/Dynamic_programming#History https://en.m.wikipedia.org/wiki/Dynamic_programming#History The generalised form of the Bellman Equation (co-formulated by Kalman of the Kalman filters fame) to control theory and EE is in some ways what the Maximum Likelihood function is to ML. https://en.m.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%93Bellman_equation https://en.m.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%8...
  - SleekEagle 5y ago
    Looks really cool, added to my amazon cart. Thanks for the rec! That hilarious and sadly insightful. I remember thinking "what the hell is so 'dynamic' about this?" the first time I learned about dynamic programming. Although "memoitative programming" sounds pretty fancy too, lol
erwincoumans 5y ago
Interesting, I didn't know about cybernetics. For the ones interested there is a book that discusses both: 'Reinforcement Learning and Optimal Control', by Dimitri P. Bertsekas. It covers exact and approximate Dynamic Programming, finite and infinite horizon problems, deterministic and stochastic models, model-based and model-free optimization. Aside from this book, Ben Recht has some interesting blog about Optimal Control and Reinforcement learning: http://www.argmin.net/2018/06/25/outsider-rl http://www.argmin.net/2018/06/25/outsider-rl