3 ms·
You’re forgetting about foveated rendering. You don’t need the 4k60 stream if you’re not looking at it, and you can only look at one screen at once. I think it’
by elliottkember 2y ago
You’re forgetting about foveated rendering. You don’t need the 4k60 stream if you’re not looking at it, and you can only look at one screen at once. I think it’s doable.
- audunw 2y agoMaybe he didn’t forget but left this thought out for brevity: foveated rendering and other tricks is certainly possible. But that requires a lot more software development. Apple is launching an MVP. Probably primarily to get developers to start making apps for it. They haven’t had time to develop all the features that would make the AVP great. I suspect next generation of this solution will stream each window individually, and let you place Mac windows freely around your environment. Then you don’t need to stream anything from static windows, you mainly just need to stream data from the window you’re interacting with at the moment.
- wtallis 2y agoApplying foveated rendering to compressing video streams would require the Mac transmitting video to have realtime access to the Vision Pro's eye tracking data—which even apps running on the Vision Pro don't get access to for privacy reasons. And with the latency of WiFi plus encoding, the foveated rendering/compression would have to be fairly conservative about resolution changes to avoid distractions from detail popping in your peripheral vision. Dropping the frame rate of video streams the user isn't looking at would probably work very well for streams that are mostly static windows. Overall, it might be workable, but it definitely isn't trivial from a technical or UX perspective.