Even RDP isn't great with latency over about 30ms from my experience. The mouse cursor never quite follows my hand movements and everything feels slightly slug

Even RDP isn't great with latency over about 30ms from my experience. The mouse cursor never quite follows my hand movements and everything feels slightly sluggish. Without tech whereby the client can somehow anticipate user actions and predict the next likely update I'm not sure how you solve that.