3 ms·

Yeah, that was my thinking: how is this doing anything other than a PID controller? Glancing at the paper it actually looked like a PD controller, but maybe I m

by tgb 9y ago

Yeah, that was my thinking: how is this doing anything other than a PID controller? Glancing at the paper it actually looked like a PD controller, but maybe I missed some feed back where it could accumulate an integral term as well.

crispyambulance 9y ago
I guess you can think of it as a "self-tuning" PID controller. It "learns" how to ride a bike. If it were really "just" a PID, you would need some arbitrarily complex procedure involving a human for tuning all the parameters?
- andars 9y ago
  It isn't self-tuning, and it doesn't "learn." From the paper: "The three constants c_i need to be set by the implementor" Under "Future Directions": "One obvious thing to do with this system is to have it learn and tune its parameters with experience." The work presented in the paper does not do this. "This network was designed in an ad-hoc, if traditional, way. First, a human tried to control the bicycle with the simulator. After many attempts, the human finally became a somewhat skilled operator of the bicycle ... the human at this point was able to describe the key parameters which were being attended to, and based on this, the two-neuron network was designed." By "designed," the author means "the three constants were selected." This sounds like the "arbitrarily complex procedure" you are looking for.
  - Filligree 9y ago
    So it's a PD controller.
- _dps 9y ago
  There's a whole body of theory on Adaptive Control (what the control systems community has called what we would now called "learning based control"). This is one of the classic texts: https://www.amazon.com/Adaptive-Control-Karl-Johan-Astrom/dp/0201558661 https://www.amazon.com/Adaptive-Control-Karl-Johan-Astrom/dp... In short, many techniques that look a lot like modern machine learning have been in use for decades in control theory for tasks exactly like you describe (e.g. automatic tuning of PIDs). If you want to dig into this a bit deeper you might look up the "MIT Rule", which is a gradient style approach for dynamically tuning (and re-tuning) adaptive control systems, and is fairly common for PID tuning.

by tgb 9y ago

crispyambulance 9y ago

I guess you can think of it as a "self-tuning" PID controller. It "learns" how to ride a bike. If it were really "just" a PID, you would need some arbitrarily complex procedure involving a human for tuning all the parameters?

andars 9y ago
It isn't self-tuning, and it doesn't "learn." From the paper: "The three constants c_i need to be set by the implementor" Under "Future Directions": "One obvious thing to do with this system is to have it learn and tune its parameters with experience." The work presented in the paper does not do this. "This network was designed in an ad-hoc, if traditional, way. First, a human tried to control the bicycle with the simulator. After many attempts, the human finally became a somewhat skilled operator of the bicycle ... the human at this point was able to describe the key parameters which were being attended to, and based on this, the two-neuron network was designed." By "designed," the author means "the three constants were selected." This sounds like the "arbitrarily complex procedure" you are looking for.
- Filligree 9y ago
  So it's a PD controller.
_dps 9y ago
There's a whole body of theory on Adaptive Control (what the control systems community has called what we would now called "learning based control"). This is one of the classic texts: https://www.amazon.com/Adaptive-Control-Karl-Johan-Astrom/dp/0201558661 https://www.amazon.com/Adaptive-Control-Karl-Johan-Astrom/dp... In short, many techniques that look a lot like modern machine learning have been in use for decades in control theory for tasks exactly like you describe (e.g. automatic tuning of PIDs). If you want to dig into this a bit deeper you might look up the "MIT Rule", which is a gradient style approach for dynamically tuning (and re-tuning) adaptive control systems, and is fairly common for PID tuning.