Do keep in mind that just because you might be using X11 doesn't mean you get any network transparency at all. The network transparency only "works" if the GUI

Do keep in mind that just because you might be using X11 doesn't mean you get any network transparency at all. The network transparency only "works" if the GUI applications you're running actually use X's 2D rendering system. Which many things don't do because it's old and terrible and slow. QT5's scene graph backend, for example, doesn't have a X renderer anymore. So those GUI apps already lost network transparency (and did anyone even notice?) On the other hand it turns out streaming video is super easy, entirely agnostic of the actual rendering stack in use, and not really that expensive in terms of bandwidth anymore.