Hi OP here. To answer why PID, basically somebody I follow asked this question on twitter, and I thought yeah why not seems like a reasonable thing to try :)

Hi OP here. To answer why PID, basically somebody I follow asked this question on twitter, and I thought yeah why not seems like a reasonable thing to try :) Actually I do conclude that PID is not quite the right thing for this problem. For me the learnings from making a PID sort of work for this problem were: 1. must use the right error function. like frequency not time. 2. must use shrinkage on the error to handle discrete number of server. 3. have to run controller at a multiple of server delay to avoid perturbations. Also I discuss the basic assumptions that a PID controller makes that are suboptimal in the video.