4 ms·

This article described something I've wanted to do for a bit: using PID (and possibly other control algorithms) to control software systems. I've thought about

by pwnna 8y ago

This article described something I've wanted to do for a bit: using PID (and possibly other control algorithms) to control software systems. I've thought about this for a bit and I have some reservations (perhaps unfounded because I'm just not that knowledgeable) actually trying to implement it in live systems: 1. PID tuning is somewhat non-trivial. It's interesting to see that the author was able to tune their PID using a simulation. This is something I've thought about doing for a while but I don't know how I can effectively validate that the simulation approximates reality to an reasonable extent. Sure I could build the simulator and validate it against the live system, but the resource spent doing that might be better off just improving the live system. It would be nice to have some theoretical arguments over why the simulation would be correct. Now this shows my lack of knowledge, so perhaps someone else knows how to deal with this more effectively. 2. Software systems these days are always changing, especially when these things are running on your servers that you're constantly updating. Tuned PID constants are only valid for a particular system configuration (the "plant"). If the plant changes too much, the controller performance can degrade, or worse: it can become unstable and blow up. This makes any system controlled by the PID difficult to change: you want to avoid changes to the system because that means you have to retune the PID, which may be difficult to perform. This effectively becomes a technical debt, especially if the original author of the PID no longer works on that project (and assuming only a few knows how it works to begin with). There are some other thoughts about non-linearity of software systems. Although these are routinely dealt with in the real world, it makes a topic that's already somewhat complex even more complicated, which further adds to the non-maintainable status of the controller. This all makes me feel like PID may not be the best choice in software systems, but I don't have concrete proof either. Sorry if this felt a little bit rambling. It's something that has been on my mind for a bit but I haven't had a chance to formalize.

mirceal 8y ago
there is PID and there is "PID" applying this to a software system is a little bit easier as because you don't have the limitations of a physical systems (the performance of the system are highly dependent on the transducers and the execution units used in the system) there are control algorithms that tune the PID parameters on the fly (google "pid autotuning"). All but the most trivial PID controllers have a certain form for autotuning nowadays (even for physical system as you've pointed out the parameters of the test system do not match the actual systems 100% and also the parameters drift in time due to wear and tear).
anton_ 8y ago
The author is here. Thank you for a thorough reply, these are both valid points. I have been thinking about them before implementing the algo and this is where my thoughts are at the moment: PID tuning is a complicated task, but what I found is that the parameter space where the coefficients are "good enough" is quite big. Certainly much bigger than where the coefficients are "perfect". A lot of times changing the coefficients has some effect on the transient state parameters such as rise[1] and settling[2] times, but they still result in a robust controller that drives the value to the setpoint. I agree, this might not be acceptable for some applications where the transient time is very important, but often times even making a P controller does a fairly good job controlling the system. In this case it means a single coefficient that is relatively straightforward to tune. I’m pretty sure our simulation was not perfect as we scaled down time to speed up the testing cycles, but we didn’t spend too much time on it. It was just a test script to get the initial values of the coefficients. When we got those we could deploy it to staging environment and run a load-testing tool to fine-tune them. Improving the existing system would mean coming up with a way to broadcast the change quickly enough, and that would add more dependencies and synchronization overhead despite being time consuming. Right now everything is done with standalone loosely-coupled services that communicate with each through APIs and cache the results. This is a very good point and I think I’m going to add these details to the post. [1] https://en.wikipedia.org/wiki/Rise_time https://en.wikipedia.org/wiki/Rise_time [2] https://en.wikipedia.org/wiki/Settling_time https://en.wikipedia.org/wiki/Settling_time
anton_ 8y ago
Regarding the point that the software systems being more flexible. A nice thing about controllers such as PID is that they deal well with disturbances and changing environments. In this particular case the PID controller runs alongside the system with a fixed sampling rate. It means that it does not really care about dynamics of the system in control. Say we want to update the caching TTL and that affects how the system behaves. The PID algo only cares about the delta of the error over a fixed period of time (sampling rate) and will adjust accordingly. Also, as mirceal mentioned, there are autotuning algos if it gets really bad. Unfortunately it is a problem if the system changes too much the algo can stop working properly. But that could happen any time you make a major change in any complex system. To me it’s more of a problem of testing and sanity-checking, not of PID or other control algorithms. When you get to a certain point of complexity, testing all possible conditions becomes unrealistic and that might lead to bugs when a big part of the system gets rewritten. On the point that only a few engineers might know how it works. One of the purpose of the article is actually put an idea out there to make more engineers familiar with the concept and collect some feedback. :) I’ll update the comment with “On non-linearity of software systems” when I get a chance. This is actually a really good one and I have a few thoughts about it.
- anton_ 8y ago
  On non-linearity of software systems. Correct me if I’m wrong but I’m assuming you mean it in a sense of being probabilistic and not easily predictable. Which is an interesting point because a lot of modern algorithms are like that. Let’s take neural networks. Being able to explain what the model is doing and how to make sure it’s correct is a whole area of research. For example, the results of a convolutional net can change drastically by altering a single pixel in an image[1]. We also faced with a problem of explaining a machine learning model that was much simpler than a neural net. It is a global problem and unfortunately I don’t have any solution of the top of my head. [1] https://arxiv.org/abs/1710.08864 https://arxiv.org/abs/1710.08864

by pwnna 8y ago

mirceal 8y ago

there is PID and there is "PID" applying this to a software system is a little bit easier as because you don't have the limitations of a physical systems (the performance of the system are highly dependent on the transducers and the execution units used in the system) there are control algorithms that tune the PID parameters on the fly (google "pid autotuning"). All but the most trivial PID controllers have a certain form for autotuning nowadays (even for physical system as you've pointed out the parameters of the test system do not match the actual systems 100% and also the parameters drift in time due to wear and tear).

anton_ 8y ago

The author is here. Thank you for a thorough reply, these are both valid points. I have been thinking about them before implementing the algo and this is where my thoughts are at the moment: PID tuning is a complicated task, but what I found is that the parameter space where the coefficients are "good enough" is quite big. Certainly much bigger than where the coefficients are "perfect". A lot of times changing the coefficients has some effect on the transient state parameters such as rise[1] and settling[2] times, but they still result in a robust controller that drives the value to the setpoint. I agree, this might not be acceptable for some applications where the transient time is very important, but often times even making a P controller does a fairly good job controlling the system. In this case it means a single coefficient that is relatively straightforward to tune. I’m pretty sure our simulation was not perfect as we scaled down time to speed up the testing cycles, but we didn’t spend too much time on it. It was just a test script to get the initial values of the coefficients. When we got those we could deploy it to staging environment and run a load-testing tool to fine-tune them. Improving the existing system would mean coming up with a way to broadcast the change quickly enough, and that would add more dependencies and synchronization overhead despite being time consuming. Right now everything is done with standalone loosely-coupled services that communicate with each through APIs and cache the results. This is a very good point and I think I’m going to add these details to the post. [1] https://en.wikipedia.org/wiki/Rise_time https://en.wikipedia.org/wiki/Rise_time [2] https://en.wikipedia.org/wiki/Settling_time https://en.wikipedia.org/wiki/Settling_time

anton_ 8y ago

Regarding the point that the software systems being more flexible. A nice thing about controllers such as PID is that they deal well with disturbances and changing environments. In this particular case the PID controller runs alongside the system with a fixed sampling rate. It means that it does not really care about dynamics of the system in control. Say we want to update the caching TTL and that affects how the system behaves. The PID algo only cares about the delta of the error over a fixed period of time (sampling rate) and will adjust accordingly. Also, as mirceal mentioned, there are autotuning algos if it gets really bad. Unfortunately it is a problem if the system changes too much the algo can stop working properly. But that could happen any time you make a major change in any complex system. To me it’s more of a problem of testing and sanity-checking, not of PID or other control algorithms. When you get to a certain point of complexity, testing all possible conditions becomes unrealistic and that might lead to bugs when a big part of the system gets rewritten. On the point that only a few engineers might know how it works. One of the purpose of the article is actually put an idea out there to make more engineers familiar with the concept and collect some feedback. :) I’ll update the comment with “On non-linearity of software systems” when I get a chance. This is actually a really good one and I have a few thoughts about it.

anton_ 8y ago
On non-linearity of software systems. Correct me if I’m wrong but I’m assuming you mean it in a sense of being probabilistic and not easily predictable. Which is an interesting point because a lot of modern algorithms are like that. Let’s take neural networks. Being able to explain what the model is doing and how to make sure it’s correct is a whole area of research. For example, the results of a convolutional net can change drastically by altering a single pixel in an image[1]. We also faced with a problem of explaining a machine learning model that was much simpler than a neural net. It is a global problem and unfortunately I don’t have any solution of the top of my head. [1] https://arxiv.org/abs/1710.08864 https://arxiv.org/abs/1710.08864