5 ms·

I was a little disappointed to see that there wasn't a discussion of how Netflix provides (or doesn't!) a high-fidelity visual experience. I imagine certain art

I was a little disappointed to see that there wasn't a discussion of how Netflix provides (or doesn't!) a high-fidelity visual experience. I imagine certain artists require fluid-motion and previews without visual artifacting, so I was hoping there was some discussion about how they provide that. I've used a handful of remote desktop systems, and I haven't found anything that's able to provide even a passable video watching experience. The issue I've run into is that the technology designed for low-latency video heavily utilizes compression, which falls apart when looking at text. I really wish Microsoft would just throw a ton of money at making RDP an absolutely delightful experience. Last time I checked, RDP wasn't able to use your GPU at all unless you had a very specific, no-longer-maintained version of Windows 10 Pro (the Workstation variant).

geodel 6y ago
> I really wish Microsoft would just throw a ton of money at making RDP an absolutely delightful experience. I suspect they will make VSCode RDP plugin and call it next generation remote desktop.
waymon 6y ago
Teradici clients for realtime. For smooth playback they export to a mp4. For real-time smooth playback, they do SDI to a streaming box that provides low latency.
mikepurvis 6y ago
I think the gold standard here is what Chromecast does with YouTube— tell the cast device directly what video to show instead of re-encoding and passing the whole thing over the network. But of course, doing that requires application level support, or at least low-level hooks into the desktop rendering system to extract video streams and treat them differently from everything else. And if the video is being generated on the fly or is an uncompressed local preview, then you may be looking at an intermediate compression step regardless, but at least it could be compression suited for video instead of for text.
- zamadatix 6y ago
  Well, that works because there is a pre-rendered video that one can just point the Chromecast to most of the time. If you cast a tab or whole desktop to display anything that's not pre-rendered (which is what workstations would fall into) then Chromecast runs into the same problems as anything else.
- dstaley 6y ago
  > But of course, doing that requires application level support, or at least low-level hooks into the desktop rendering system to extract video streams and treat them differently from everything else. That's exactly what RDP does on Windows! I don't know the extent to which controls it supports rendering on the client, but the concept is that RDP is aware of things like text, buttons, title bars, etc., and is able to tell the client things like "draw a button at (20, 140)" instead of sending the raw pixel data. With enough engineering effort (and some standardization on the application UI side) I totally think Microsoft could make RDP the protocol of choice for virtual desktops.
  - mikepurvis 6y ago
    It makes total sense; OTOH I'm kind of cracking up here because isn't this what a web browser has become? A portal for viewing a remote application state, that can be serialized into a blob of semi-structured XML and JSON? It's hard to imagine that retrofitting such a serialization onto the total wild west that is desktop applications could be any more successful or consistent than what the web has already got.
  - banana_giraffe 6y ago
    > but the concept is that RDP is aware of things like text, buttons, title bars, etc The fun bit, at least as I understand it, with the move to more and more Electron based apps, this is somewhat moot since those apps draw their own thing. There are still performance improvements RDP brings over a simple "blit everything" protocol, but they're smaller and smaller. My biggest complaint with RDP in general is how many Windows apps react poorly to changing resolutions and DPI. I'm practically an expert now in rescuing windows that have decided "I was over at here on the right side of your ultra-wide monitor, so I'm still over here in this no man's land on this tiny RDP resolution."
zamadatix 6y ago
> Last time I checked, RDP wasn't able to use your GPU at all unless you had a very specific, no-longer-maintained version of Windows 10 Pro Not sure where you got that idea. RDP has been able to use the GPU for both encoding and the apps since Windows 7 era and both certainly still work on the current version of Windows 10. The main limitation I've run into is it's still limited to 30 FPS though.
- dstaley 6y ago
  It's my understanding that RemoteFX is only available in Windows Server and Windows 10 Pro for Workstations.
  - zamadatix 6y ago
    During the Windows 7 era it required Enterprise or Server, during the Windows 10 era it requires Pro, Enterprise, or Server (or that Pro for Workstations version you mention). Remember you have to actually enable it though, it's not configured by default. Perhaps you're thinking of the deprecation of the RemoteFX guest vGPU for the Hyper-V role?
nonane 6y ago
I work on Jump Desktop (https://jumpdesktop.com https://jumpdesktop.com) that's being used for remote video editing @ 60fps on up to 3 monitors. It supports very low latency remote desktop and even works in environments that have high latency. For example here's Jump connected from the east coast to a physical server in Paris with 150ms ping times between the two: https://twitter.com/jumpdesktop/status/1356679263806763013?s=20 https://twitter.com/jumpdesktop/status/1356679263806763013?s... It works on Macs and Windows. It works really well with text as well as full motion video.
- dan-robertson 6y ago
  What’s the actual latency you get between pressing a key and seeing the result on the screen? And how is it if the connection is good and the server is close?
  - nonane 6y ago
    This is a good question. I guess what you're actually interested in is the latency between the physical display updating vs the local display updating, right? I just did a quick test under good circumstances using this video (machines on the same lan): https://www.youtube.com/watch?v=OLxY0HDakRk https://www.youtube.com/watch?v=OLxY0HDakRk with the physical and remote display side by side. I noticed an approximately 3 frame lag between the physical display and the local / client display update.
    - dan-robertson 6y ago
      Well what I really meant was the time between pressing a key and seeing a character on the screen. It’s easy to measure (except for deciding when the key is pressed. I go for hitting the key fast and timing from when it stops going down) by using some high speed camera (or medium speed, see https://apps.apple.com/gb/app/is-it-snappy/id1219667593 https://apps.apple.com/gb/app/is-it-snappy/id1219667593 ) and it’s the latency that actually affects how usable something is for typing. The problem is that it’s only really good for comparing things as lots of devices in the middle add latency, eg keyboards can have 15-50ms of latency[1], typical deferred rendering can give you something like a frame of latency, monitors may add latency or just have slower switching times (it takes longer to switch a pixel to an accurate colour), and some pixels at the bottom of the screen will make it out of the computer 16ms (or whatever a frame is) later than those at the top. For comparison with my (not particularly optimized) set up going over a fast internet connection, I get something like 150ms between pressing a key and the character starting to show up in emacs (it takes a few ms for the pixels to finish switching). 10 frames feels like a lot to me. My best guess is that without changing anything drastic or eg reducing resolution, I might be able to get that down to 100ms which is still looks like a pretty big number (eg a round trip between London and New York is something like 70ms). Anyway, thank you for investigating this. A few frames from the network seems pretty good, especially with a reasonably large delta between frames in that video) [1] https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/
boulos 6y ago
You should probably check out Teradici. They even finally published Linux clients.
piotr_chomczyk 6y ago
Hey, we launched Renderro - cloud computers for filmmakers, graphic designers and animators in October 2020, and since then we are helping creatives all around the globe. Check us out here: https://renderro.com/ https://renderro.com/ We are supporting up to 4 monitors, each up to 4k res, and the quality we are offering (60 Hz and low latency) allows you to do real-time editing, eg. when using Premiere Pro or Avid MC running on our Cloud Computers. We are also used a lot for 3D design and rendering with Blender. Independent review: https://www.youtube.com/watch?v=LxM4mC5hwpo https://www.youtube.com/watch?v=LxM4mC5hwpo Tutorial showing how to use Renderro: https://www.youtube.com/watch?v=Uw44El8kMxM https://www.youtube.com/watch?v=Uw44El8kMxM If you have any questions, don't hesitate to contact us at info@renderro.com Please note, that with us you don't need any additional tools to connect to our Cloud Computers (no need for Teradici or Parsec) - we've got VDI solutions built in Renderro. We are focusing on delivering Renderro as super easy to use, out-of-the-box solution with clear pricing and superb quality, so I hope that you will enjoy it ;) Cheers!
xnyan 6y ago
This has been a quest of mine for a while now, “native” (or close enough that it does not matter) display and input over a data network. You can get very close today, but you need both very low network latency (<8ms or half a 16ms frame at a minimum) and a hardware + software stack that can quickly and efficiently dump it’s display framebuffer to the network. Getting all that to work at the same time is a real trick. Getting it to all work consistently has so far not proven worth the effort vs some other solution like just using another device.

geodel 6y ago

> I really wish Microsoft would just throw a ton of money at making RDP an absolutely delightful experience. I suspect they will make VSCode RDP plugin and call it next generation remote desktop.

waymon 6y ago

Teradici clients for realtime. For smooth playback they export to a mp4. For real-time smooth playback, they do SDI to a streaming box that provides low latency.

mikepurvis 6y ago

I think the gold standard here is what Chromecast does with YouTube— tell the cast device directly what video to show instead of re-encoding and passing the whole thing over the network. But of course, doing that requires application level support, or at least low-level hooks into the desktop rendering system to extract video streams and treat them differently from everything else. And if the video is being generated on the fly or is an uncompressed local preview, then you may be looking at an intermediate compression step regardless, but at least it could be compression suited for video instead of for text.

zamadatix 6y ago
Well, that works because there is a pre-rendered video that one can just point the Chromecast to most of the time. If you cast a tab or whole desktop to display anything that's not pre-rendered (which is what workstations would fall into) then Chromecast runs into the same problems as anything else.
dstaley 6y ago
> But of course, doing that requires application level support, or at least low-level hooks into the desktop rendering system to extract video streams and treat them differently from everything else. That's exactly what RDP does on Windows! I don't know the extent to which controls it supports rendering on the client, but the concept is that RDP is aware of things like text, buttons, title bars, etc., and is able to tell the client things like "draw a button at (20, 140)" instead of sending the raw pixel data. With enough engineering effort (and some standardization on the application UI side) I totally think Microsoft could make RDP the protocol of choice for virtual desktops.
- mikepurvis 6y ago
  It makes total sense; OTOH I'm kind of cracking up here because isn't this what a web browser has become? A portal for viewing a remote application state, that can be serialized into a blob of semi-structured XML and JSON? It's hard to imagine that retrofitting such a serialization onto the total wild west that is desktop applications could be any more successful or consistent than what the web has already got.
- banana_giraffe 6y ago
  > but the concept is that RDP is aware of things like text, buttons, title bars, etc The fun bit, at least as I understand it, with the move to more and more Electron based apps, this is somewhat moot since those apps draw their own thing. There are still performance improvements RDP brings over a simple "blit everything" protocol, but they're smaller and smaller. My biggest complaint with RDP in general is how many Windows apps react poorly to changing resolutions and DPI. I'm practically an expert now in rescuing windows that have decided "I was over at here on the right side of your ultra-wide monitor, so I'm still over here in this no man's land on this tiny RDP resolution."

zamadatix 6y ago

> Last time I checked, RDP wasn't able to use your GPU at all unless you had a very specific, no-longer-maintained version of Windows 10 Pro Not sure where you got that idea. RDP has been able to use the GPU for both encoding and the apps since Windows 7 era and both certainly still work on the current version of Windows 10. The main limitation I've run into is it's still limited to 30 FPS though.

dstaley 6y ago
It's my understanding that RemoteFX is only available in Windows Server and Windows 10 Pro for Workstations.
- zamadatix 6y ago
  During the Windows 7 era it required Enterprise or Server, during the Windows 10 era it requires Pro, Enterprise, or Server (or that Pro for Workstations version you mention). Remember you have to actually enable it though, it's not configured by default. Perhaps you're thinking of the deprecation of the RemoteFX guest vGPU for the Hyper-V role?

nonane 6y ago

I work on Jump Desktop (https://jumpdesktop.com https://jumpdesktop.com) that's being used for remote video editing @ 60fps on up to 3 monitors. It supports very low latency remote desktop and even works in environments that have high latency. For example here's Jump connected from the east coast to a physical server in Paris with 150ms ping times between the two: https://twitter.com/jumpdesktop/status/1356679263806763013?s=20 https://twitter.com/jumpdesktop/status/1356679263806763013?s... It works on Macs and Windows. It works really well with text as well as full motion video.

dan-robertson 6y ago
What’s the actual latency you get between pressing a key and seeing the result on the screen? And how is it if the connection is good and the server is close?
- nonane 6y ago
  This is a good question. I guess what you're actually interested in is the latency between the physical display updating vs the local display updating, right? I just did a quick test under good circumstances using this video (machines on the same lan): https://www.youtube.com/watch?v=OLxY0HDakRk https://www.youtube.com/watch?v=OLxY0HDakRk with the physical and remote display side by side. I noticed an approximately 3 frame lag between the physical display and the local / client display update.
  - dan-robertson 6y ago
    Well what I really meant was the time between pressing a key and seeing a character on the screen. It’s easy to measure (except for deciding when the key is pressed. I go for hitting the key fast and timing from when it stops going down) by using some high speed camera (or medium speed, see https://apps.apple.com/gb/app/is-it-snappy/id1219667593 https://apps.apple.com/gb/app/is-it-snappy/id1219667593 ) and it’s the latency that actually affects how usable something is for typing. The problem is that it’s only really good for comparing things as lots of devices in the middle add latency, eg keyboards can have 15-50ms of latency[1], typical deferred rendering can give you something like a frame of latency, monitors may add latency or just have slower switching times (it takes longer to switch a pixel to an accurate colour), and some pixels at the bottom of the screen will make it out of the computer 16ms (or whatever a frame is) later than those at the top. For comparison with my (not particularly optimized) set up going over a fast internet connection, I get something like 150ms between pressing a key and the character starting to show up in emacs (it takes a few ms for the pixels to finish switching). 10 frames feels like a lot to me. My best guess is that without changing anything drastic or eg reducing resolution, I might be able to get that down to 100ms which is still looks like a pretty big number (eg a round trip between London and New York is something like 70ms). Anyway, thank you for investigating this. A few frames from the network seems pretty good, especially with a reasonably large delta between frames in that video) [1] https://danluu.com/keyboard-latency/ https://danluu.com/keyboard-latency/

boulos 6y ago

You should probably check out Teradici. They even finally published Linux clients.

piotr_chomczyk 6y ago

Hey, we launched Renderro - cloud computers for filmmakers, graphic designers and animators in October 2020, and since then we are helping creatives all around the globe. Check us out here: https://renderro.com/ https://renderro.com/ We are supporting up to 4 monitors, each up to 4k res, and the quality we are offering (60 Hz and low latency) allows you to do real-time editing, eg. when using Premiere Pro or Avid MC running on our Cloud Computers. We are also used a lot for 3D design and rendering with Blender. Independent review: https://www.youtube.com/watch?v=LxM4mC5hwpo https://www.youtube.com/watch?v=LxM4mC5hwpo Tutorial showing how to use Renderro: https://www.youtube.com/watch?v=Uw44El8kMxM https://www.youtube.com/watch?v=Uw44El8kMxM If you have any questions, don't hesitate to contact us at info@renderro.com Please note, that with us you don't need any additional tools to connect to our Cloud Computers (no need for Teradici or Parsec) - we've got VDI solutions built in Renderro. We are focusing on delivering Renderro as super easy to use, out-of-the-box solution with clear pricing and superb quality, so I hope that you will enjoy it ;) Cheers!

xnyan 6y ago

This has been a quest of mine for a while now, “native” (or close enough that it does not matter) display and input over a data network. You can get very close today, but you need both very low network latency (<8ms or half a 16ms frame at a minimum) and a hardware + software stack that can quickly and efficiently dump it’s display framebuffer to the network. Getting all that to work at the same time is a real trick. Getting it to all work consistently has so far not proven worth the effort vs some other solution like just using another device.