This is a pretty classic PID here. You're attempting to keep latency at 35ms and based on the latency on each packet as observed by the server, and previously r

This is a pretty classic PID here. You're attempting to keep latency at 35ms and based on the latency on each packet as observed by the server, and previously recorded latencies, you can add corrections. With some simulations and tuning I'll bet you can get a pretty good algorithm going. Simulating this stuff is pretty simple. Model each incoming stream as an MM1 queue and play around with your PID algorithm until it gives you the results you need.