6 ms·

Isn't this just optimization of control parameters? What is the comparison with existing control engineering methods like PID tuning, MPC and optimal control?

pidtuner 6y ago
I think you are right, in the case of the trebuchet, it just computes a black box approximation of the inverse of the system. The difference being that an analytic inversion would solve the problem for all wind and target conditions, while the NN solution will only work for for value ranges used to obtain the data that trained the NN. In the case of the inverted pendulum, again the disadvantage of using NN is the black-bock nature of the control algorithm. As a control engineer this gives me the chills, because using black-box algorithms tell you nothing about the robustness of the closed loop system. With model-based control, at least we have strong mathematical tools to guarantee that the closed loop will be robust enough to handle variations outside the data that we used for training (our model). With black-box algorithms like NN you have no guarantees. I would not get into a plane controlled by a NN for sure, look what happened with the 737MAX when software engineers thought they could solve dynamical system problems.
- Libbum 6y ago
  To respond to both the parent question, and this comment: indeed, this is black-box optimal control in essence. However, this method is just one small aspect of the SciML [0] ecosystem now. The article is a little outdated in that sense. Once obtaining your NN control parameter, it's now possible to use Sparse Identification of Nonlinear Dynamics (SINDy) on that parameter to recover equations of motion governing it [1]. The real promise of these methods is to use the universal approximator power of NNs to get around the 'curse of dimensionality' & uncover presently unknown representations of motion within any system. Take a look at [2] for a more detailed description. [0]: https://sciml.ai/ https://sciml.ai/ [1]: https://datadriven.sciml.ai/dev/sparse_identification/sindy/ https://datadriven.sciml.ai/dev/sparse_identification/sindy/ [2]: https://arxiv.org/abs/2001.04385 https://arxiv.org/abs/2001.04385
  - pidtuner 6y ago
    "The real promise of these methods is to use the universal approximator power of NNs...", still if one is to use a grey-box non-linear model dx/dt = F(x, u, t), why use NNs to characterize F? I would be more comfortable using a polynomial to characterize non-linearity than a "deep" black-box. Polynomials are much easier to "train" because it is just one linear regression with no iteration. It has also been hinted that NN are in essence polynomial regressions [0]. Furthermore, most activation functions are base on e^x where the actual implementation of e^x in a computer is again a polynomial! [0] https://arxiv.org/abs/1806.06850 https://arxiv.org/abs/1806.06850
    - Libbum 6y ago
      I'm not entirely confident in answering that directly, so perhaps you can check my reasoning here. If F is completely unknown, perhaps you start training with a 10 dimensional polynomial basis. What is the (computational) cost of obtaining your solution? Once you have it, will this polynomial accurately represent your system in any real world manner? Perhaps higher order parameters are needed to approximate trigonometric functions - are you able to easily add such functions to your training basis? If not - then your basis could be too restrictive to provide you with a minimal implementation of your control variable. It looks like you work with this stuff far more than I have, so perhaps that's not an adequate answer. Another way to look at this though: If you only wanted to characterise your system with polynomials, UODEs + SINDy can do this for you - the NN is simply the optimisation method that's in place of any other optimisation algorithm.
      - pidtuner 6y ago
        The computational cost of "training" a polynomial would be the same as just one iteration of the training algorithm used by typical NNs. When it comes to trig functions, the story is the same as with the exp function e(). When you call the sin() or the cos() functions in your favorite language, in the end it uses taylor series (polynomials) to compute it (plus some hacks to add precision on certain ranges of the function and to overcome some floating point precision limitations). The degree at which a polynomial model would fit the real world system has to be validated against data, just the same as with NNs. What does one do when an NN fit is not good enough or too good (overfitting)? One adds or removes layers. Same with polynomials, one increases or decreases degrees. Sorry for the rant, I am not saying NNs are useless, because I do believe they are super useful for certain problems, specially for categorization. But it seems to me that now a days there is this trend of using NNs as a hammer, and not all problems are nails. Specially when it comes to control, and lives or big economic losses are at stake, it is the responsibility of the engineer to resist the fuzz and craze and use the right tool for the problem.
freemint 6y ago
I assume you are talking about the pendulum. PID tuning is fundamentally restricted to holding set points and can't deal with non-linearites to reach those set points. MPC if solved over a to small time horizon might fail to find a windup trajectory. But that isn't a restriction. However i would consider MPC overblown for this, as MPC is usually a series of optimal control problems. By optimal control i assume you mean things approaches where you optimize a u() so it minimizes some integral. This approach does just that, only that u is not parametrized by polynomials/splines but by a neural network, taking state, time and other things into account.

pidtuner 6y ago

I think you are right, in the case of the trebuchet, it just computes a black box approximation of the inverse of the system. The difference being that an analytic inversion would solve the problem for all wind and target conditions, while the NN solution will only work for for value ranges used to obtain the data that trained the NN. In the case of the inverted pendulum, again the disadvantage of using NN is the black-bock nature of the control algorithm. As a control engineer this gives me the chills, because using black-box algorithms tell you nothing about the robustness of the closed loop system. With model-based control, at least we have strong mathematical tools to guarantee that the closed loop will be robust enough to handle variations outside the data that we used for training (our model). With black-box algorithms like NN you have no guarantees. I would not get into a plane controlled by a NN for sure, look what happened with the 737MAX when software engineers thought they could solve dynamical system problems.

Libbum 6y ago
To respond to both the parent question, and this comment: indeed, this is black-box optimal control in essence. However, this method is just one small aspect of the SciML [0] ecosystem now. The article is a little outdated in that sense. Once obtaining your NN control parameter, it's now possible to use Sparse Identification of Nonlinear Dynamics (SINDy) on that parameter to recover equations of motion governing it [1]. The real promise of these methods is to use the universal approximator power of NNs to get around the 'curse of dimensionality' & uncover presently unknown representations of motion within any system. Take a look at [2] for a more detailed description. [0]: https://sciml.ai/ https://sciml.ai/ [1]: https://datadriven.sciml.ai/dev/sparse_identification/sindy/ https://datadriven.sciml.ai/dev/sparse_identification/sindy/ [2]: https://arxiv.org/abs/2001.04385 https://arxiv.org/abs/2001.04385
- pidtuner 6y ago
  "The real promise of these methods is to use the universal approximator power of NNs...", still if one is to use a grey-box non-linear model dx/dt = F(x, u, t), why use NNs to characterize F? I would be more comfortable using a polynomial to characterize non-linearity than a "deep" black-box. Polynomials are much easier to "train" because it is just one linear regression with no iteration. It has also been hinted that NN are in essence polynomial regressions [0]. Furthermore, most activation functions are base on e^x where the actual implementation of e^x in a computer is again a polynomial! [0] https://arxiv.org/abs/1806.06850 https://arxiv.org/abs/1806.06850
  - Libbum 6y ago
    I'm not entirely confident in answering that directly, so perhaps you can check my reasoning here. If F is completely unknown, perhaps you start training with a 10 dimensional polynomial basis. What is the (computational) cost of obtaining your solution? Once you have it, will this polynomial accurately represent your system in any real world manner? Perhaps higher order parameters are needed to approximate trigonometric functions - are you able to easily add such functions to your training basis? If not - then your basis could be too restrictive to provide you with a minimal implementation of your control variable. It looks like you work with this stuff far more than I have, so perhaps that's not an adequate answer. Another way to look at this though: If you only wanted to characterise your system with polynomials, UODEs + SINDy can do this for you - the NN is simply the optimisation method that's in place of any other optimisation algorithm.
    - pidtuner 6y ago
      The computational cost of "training" a polynomial would be the same as just one iteration of the training algorithm used by typical NNs. When it comes to trig functions, the story is the same as with the exp function e(). When you call the sin() or the cos() functions in your favorite language, in the end it uses taylor series (polynomials) to compute it (plus some hacks to add precision on certain ranges of the function and to overcome some floating point precision limitations). The degree at which a polynomial model would fit the real world system has to be validated against data, just the same as with NNs. What does one do when an NN fit is not good enough or too good (overfitting)? One adds or removes layers. Same with polynomials, one increases or decreases degrees. Sorry for the rant, I am not saying NNs are useless, because I do believe they are super useful for certain problems, specially for categorization. But it seems to me that now a days there is this trend of using NNs as a hammer, and not all problems are nails. Specially when it comes to control, and lives or big economic losses are at stake, it is the responsibility of the engineer to resist the fuzz and craze and use the right tool for the problem.

freemint 6y ago

I assume you are talking about the pendulum. PID tuning is fundamentally restricted to holding set points and can't deal with non-linearites to reach those set points. MPC if solved over a to small time horizon might fail to find a windup trajectory. But that isn't a restriction. However i would consider MPC overblown for this, as MPC is usually a series of optimal control problems. By optimal control i assume you mean things approaches where you optimize a u() so it minimizes some integral. This approach does just that, only that u is not parametrized by polynomials/splines but by a neural network, taking state, time and other things into account.