5 ms·

What Toasters and Distributed Systems Might Have in Common

41 points by twakefield 8y ago

aidenn0 8y ago
http://nikital.github.io/pid/ http://nikital.github.io/pid/ is really neat. I do wish it either had an option for a saturated integral, or had an option for "wind" blowing in a particular direction as any significant change to the setpoint will make large integrals useless and small values have little use absent a force for them to counteract.
- aidenn0 8y ago
  Why complain when it's open source: https://jasom.github.io/pid/index.html https://jasom.github.io/pid/index.html has both improvements I suggested, plus a display of the X and Y error values
  - nikital 8y ago
    Cool, I made this toy some years ago :) Thanks for the additions, merged your code into the original.
ddebernardy 8y ago
Wasteland? [0] [1] [0]: https://en.wikipedia.org/wiki/Wasteland_(video_game) https://en.wikipedia.org/wiki/Wasteland_(video_game) [1]: http://wasteland.wikia.com/wiki/Broken_toaster http://wasteland.wikia.com/wiki/Broken_toaster
pwnna 8y ago
This article described something I've wanted to do for a bit: using PID (and possibly other control algorithms) to control software systems. I've thought about this for a bit and I have some reservations (perhaps unfounded because I'm just not that knowledgeable) actually trying to implement it in live systems: 1. PID tuning is somewhat non-trivial. It's interesting to see that the author was able to tune their PID using a simulation. This is something I've thought about doing for a while but I don't know how I can effectively validate that the simulation approximates reality to an reasonable extent. Sure I could build the simulator and validate it against the live system, but the resource spent doing that might be better off just improving the live system. It would be nice to have some theoretical arguments over why the simulation would be correct. Now this shows my lack of knowledge, so perhaps someone else knows how to deal with this more effectively. 2. Software systems these days are always changing, especially when these things are running on your servers that you're constantly updating. Tuned PID constants are only valid for a particular system configuration (the "plant"). If the plant changes too much, the controller performance can degrade, or worse: it can become unstable and blow up. This makes any system controlled by the PID difficult to change: you want to avoid changes to the system because that means you have to retune the PID, which may be difficult to perform. This effectively becomes a technical debt, especially if the original author of the PID no longer works on that project (and assuming only a few knows how it works to begin with). There are some other thoughts about non-linearity of software systems. Although these are routinely dealt with in the real world, it makes a topic that's already somewhat complex even more complicated, which further adds to the non-maintainable status of the controller. This all makes me feel like PID may not be the best choice in software systems, but I don't have concrete proof either. Sorry if this felt a little bit rambling. It's something that has been on my mind for a bit but I haven't had a chance to formalize.
- mirceal 8y ago
  there is PID and there is "PID" applying this to a software system is a little bit easier as because you don't have the limitations of a physical systems (the performance of the system are highly dependent on the transducers and the execution units used in the system) there are control algorithms that tune the PID parameters on the fly (google "pid autotuning"). All but the most trivial PID controllers have a certain form for autotuning nowadays (even for physical system as you've pointed out the parameters of the test system do not match the actual systems 100% and also the parameters drift in time due to wear and tear).
- anton_ 8y ago
  The author is here. Thank you for a thorough reply, these are both valid points. I have been thinking about them before implementing the algo and this is where my thoughts are at the moment: PID tuning is a complicated task, but what I found is that the parameter space where the coefficients are "good enough" is quite big. Certainly much bigger than where the coefficients are "perfect". A lot of times changing the coefficients has some effect on the transient state parameters such as rise[1] and settling[2] times, but they still result in a robust controller that drives the value to the setpoint. I agree, this might not be acceptable for some applications where the transient time is very important, but often times even making a P controller does a fairly good job controlling the system. In this case it means a single coefficient that is relatively straightforward to tune. I’m pretty sure our simulation was not perfect as we scaled down time to speed up the testing cycles, but we didn’t spend too much time on it. It was just a test script to get the initial values of the coefficients. When we got those we could deploy it to staging environment and run a load-testing tool to fine-tune them. Improving the existing system would mean coming up with a way to broadcast the change quickly enough, and that would add more dependencies and synchronization overhead despite being time consuming. Right now everything is done with standalone loosely-coupled services that communicate with each through APIs and cache the results. This is a very good point and I think I’m going to add these details to the post. [1] https://en.wikipedia.org/wiki/Rise_time https://en.wikipedia.org/wiki/Rise_time [2] https://en.wikipedia.org/wiki/Settling_time https://en.wikipedia.org/wiki/Settling_time
- anton_ 8y ago
stiglitz 8y ago
“IP warmup.” Is this an algorithm to send broadcast emails at the maximum rate that avoids ISP spam detection? I find this concept disturbing. Morals of spam aside, we have two systems clearly working at odds rather than doing productive work.

41 points by twakefield 8y ago

http://nikital.github.io/pid/ http://nikital.github.io/pid/ is really neat. I do wish it either had an option for a saturated integral, or had an option for "wind" blowing in a particular direction as any significant change to the setpoint will make large integrals useless and small values have little use absent a force for them to counteract.

aidenn0 8y ago
Why complain when it's open source: https://jasom.github.io/pid/index.html https://jasom.github.io/pid/index.html has both improvements I suggested, plus a display of the X and Y error values
- nikital 8y ago
  Cool, I made this toy some years ago :) Thanks for the additions, merged your code into the original.

ddebernardy 8y ago

Wasteland? [0] [1] [0]: https://en.wikipedia.org/wiki/Wasteland_(video_game) https://en.wikipedia.org/wiki/Wasteland_(video_game) [1]: http://wasteland.wikia.com/wiki/Broken_toaster http://wasteland.wikia.com/wiki/Broken_toaster

pwnna 8y ago

This article described something I've wanted to do for a bit: using PID (and possibly other control algorithms) to control software systems. I've thought about this for a bit and I have some reservations (perhaps unfounded because I'm just not that knowledgeable) actually trying to implement it in live systems: 1. PID tuning is somewhat non-trivial. It's interesting to see that the author was able to tune their PID using a simulation. This is something I've thought about doing for a while but I don't know how I can effectively validate that the simulation approximates reality to an reasonable extent. Sure I could build the simulator and validate it against the live system, but the resource spent doing that might be better off just improving the live system. It would be nice to have some theoretical arguments over why the simulation would be correct. Now this shows my lack of knowledge, so perhaps someone else knows how to deal with this more effectively. 2. Software systems these days are always changing, especially when these things are running on your servers that you're constantly updating. Tuned PID constants are only valid for a particular system configuration (the "plant"). If the plant changes too much, the controller performance can degrade, or worse: it can become unstable and blow up. This makes any system controlled by the PID difficult to change: you want to avoid changes to the system because that means you have to retune the PID, which may be difficult to perform. This effectively becomes a technical debt, especially if the original author of the PID no longer works on that project (and assuming only a few knows how it works to begin with). There are some other thoughts about non-linearity of software systems. Although these are routinely dealt with in the real world, it makes a topic that's already somewhat complex even more complicated, which further adds to the non-maintainable status of the controller. This all makes me feel like PID may not be the best choice in software systems, but I don't have concrete proof either. Sorry if this felt a little bit rambling. It's something that has been on my mind for a bit but I haven't had a chance to formalize.

mirceal 8y ago
there is PID and there is "PID" applying this to a software system is a little bit easier as because you don't have the limitations of a physical systems (the performance of the system are highly dependent on the transducers and the execution units used in the system) there are control algorithms that tune the PID parameters on the fly (google "pid autotuning"). All but the most trivial PID controllers have a certain form for autotuning nowadays (even for physical system as you've pointed out the parameters of the test system do not match the actual systems 100% and also the parameters drift in time due to wear and tear).
anton_ 8y ago
The author is here. Thank you for a thorough reply, these are both valid points. I have been thinking about them before implementing the algo and this is where my thoughts are at the moment: PID tuning is a complicated task, but what I found is that the parameter space where the coefficients are "good enough" is quite big. Certainly much bigger than where the coefficients are "perfect". A lot of times changing the coefficients has some effect on the transient state parameters such as rise[1] and settling[2] times, but they still result in a robust controller that drives the value to the setpoint. I agree, this might not be acceptable for some applications where the transient time is very important, but often times even making a P controller does a fairly good job controlling the system. In this case it means a single coefficient that is relatively straightforward to tune. I’m pretty sure our simulation was not perfect as we scaled down time to speed up the testing cycles, but we didn’t spend too much time on it. It was just a test script to get the initial values of the coefficients. When we got those we could deploy it to staging environment and run a load-testing tool to fine-tune them. Improving the existing system would mean coming up with a way to broadcast the change quickly enough, and that would add more dependencies and synchronization overhead despite being time consuming. Right now everything is done with standalone loosely-coupled services that communicate with each through APIs and cache the results. This is a very good point and I think I’m going to add these details to the post. [1] https://en.wikipedia.org/wiki/Rise_time https://en.wikipedia.org/wiki/Rise_time [2] https://en.wikipedia.org/wiki/Settling_time https://en.wikipedia.org/wiki/Settling_time
anton_ 8y ago

stiglitz 8y ago

“IP warmup.” Is this an algorithm to send broadcast emails at the maximum rate that avoids ISP spam detection? I find this concept disturbing. Morals of spam aside, we have two systems clearly working at odds rather than doing productive work.