4 ms·
Applying foveated rendering to compressing video streams would require the Mac transmitting video to have realtime access to the Vision Pro's eye tracking data—
by wtallis 3y ago
Applying foveated rendering to compressing video streams would require the Mac transmitting video to have realtime access to the Vision Pro's eye tracking data—which even apps running on the Vision Pro don't get access to for privacy reasons. And with the latency of WiFi plus encoding, the foveated rendering/compression would have to be fairly conservative about resolution changes to avoid distractions from detail popping in your peripheral vision. Dropping the frame rate of video streams the user isn't looking at would probably work very well for streams that are mostly static windows. Overall, it might be workable, but it definitely isn't trivial from a technical or UX perspective.