3 ms·

Regardless of limitations on the Mac side, there's simply no way to stream data into the Vision Pro fast enough. WiFi can only come close with very heavy compre

Regardless of limitations on the Mac side, there's simply no way to stream data into the Vision Pro fast enough. WiFi can only come close with very heavy compression and a low bar for reliability. One 1080p60 uncompressed video stream is 3.2Gb/s, already well beyond what WiFi can be relied upon to deliver. The existing single 4k desktop streaming is pushing the limits of low-latency, high-quality compression.

nradov 3y ago
Right so the Vision Pro should have had a wired connection.
- tsimionescu 3y ago
  Strapping on a helmet with 4 HDMI cables running out of it seems like an even worse proposition than what they currently offer. Multi-screen Mac use is already an extremely niche market - if the Vision line up had to ride or die on this, it would never have been born.
  - nradov 3y ago
    No one suggested HDMI. USB can run 80Gb/s now which should be sufficient.
  - FireBeyond 3y ago
    > Strapping on a helmet with 4 HDMI cables running out of it Like 4 HDMI streams can't go into 'one' cable? My Samsung TV has a box where all the inputs connect, and then there's one cable the diameter of a spaghetti noodle that carries the video signal AND power to the TV. > Multi-screen Mac use is already an extremely niche market Because much as I am all-in on Mac, it's a niche market because multi-monitor support on macOS is largely garbage.
elliottkember 3y ago
You’re forgetting about foveated rendering. You don’t need the 4k60 stream if you’re not looking at it, and you can only look at one screen at once. I think it’s doable.
- audunw 3y ago
  Maybe he didn’t forget but left this thought out for brevity: foveated rendering and other tricks is certainly possible. But that requires a lot more software development. Apple is launching an MVP. Probably primarily to get developers to start making apps for it. They haven’t had time to develop all the features that would make the AVP great. I suspect next generation of this solution will stream each window individually, and let you place Mac windows freely around your environment. Then you don’t need to stream anything from static windows, you mainly just need to stream data from the window you’re interacting with at the moment.
- wtallis 3y ago
  Applying foveated rendering to compressing video streams would require the Mac transmitting video to have realtime access to the Vision Pro's eye tracking data—which even apps running on the Vision Pro don't get access to for privacy reasons. And with the latency of WiFi plus encoding, the foveated rendering/compression would have to be fairly conservative about resolution changes to avoid distractions from detail popping in your peripheral vision. Dropping the frame rate of video streams the user isn't looking at would probably work very well for streams that are mostly static windows. Overall, it might be workable, but it definitely isn't trivial from a technical or UX perspective.
akvadrako 3y ago
Why WiFi? We have 60Ghz line of sight WiGig for this kind of thing. It's 40gbps which with some compression could do 8k screens. Besides you could use eye tracking to only transmit what's being seen in high quality. https://en.m.wikipedia.org/wiki/IEEE_802.11ay https://en.m.wikipedia.org/wiki/IEEE_802.11ay
nox101 3y ago
And yet my Quest will do 6 screens just fine. You don't need every the content of each screen to update at 60-120fps. You only need it's representation to move smoothly. If I had a video or game in every screen then yes, I might need the bandwidth. But most screens are mostly static with only the cursor and my input making any changes.

nradov 3y ago

Right so the Vision Pro should have had a wired connection.

tsimionescu 3y ago
Strapping on a helmet with 4 HDMI cables running out of it seems like an even worse proposition than what they currently offer. Multi-screen Mac use is already an extremely niche market - if the Vision line up had to ride or die on this, it would never have been born.
- nradov 3y ago
  No one suggested HDMI. USB can run 80Gb/s now which should be sufficient.
- FireBeyond 3y ago
  > Strapping on a helmet with 4 HDMI cables running out of it Like 4 HDMI streams can't go into 'one' cable? My Samsung TV has a box where all the inputs connect, and then there's one cable the diameter of a spaghetti noodle that carries the video signal AND power to the TV. > Multi-screen Mac use is already an extremely niche market Because much as I am all-in on Mac, it's a niche market because multi-monitor support on macOS is largely garbage.

elliottkember 3y ago

You’re forgetting about foveated rendering. You don’t need the 4k60 stream if you’re not looking at it, and you can only look at one screen at once. I think it’s doable.

audunw 3y ago
Maybe he didn’t forget but left this thought out for brevity: foveated rendering and other tricks is certainly possible. But that requires a lot more software development. Apple is launching an MVP. Probably primarily to get developers to start making apps for it. They haven’t had time to develop all the features that would make the AVP great. I suspect next generation of this solution will stream each window individually, and let you place Mac windows freely around your environment. Then you don’t need to stream anything from static windows, you mainly just need to stream data from the window you’re interacting with at the moment.
wtallis 3y ago
Applying foveated rendering to compressing video streams would require the Mac transmitting video to have realtime access to the Vision Pro's eye tracking data—which even apps running on the Vision Pro don't get access to for privacy reasons. And with the latency of WiFi plus encoding, the foveated rendering/compression would have to be fairly conservative about resolution changes to avoid distractions from detail popping in your peripheral vision. Dropping the frame rate of video streams the user isn't looking at would probably work very well for streams that are mostly static windows. Overall, it might be workable, but it definitely isn't trivial from a technical or UX perspective.

akvadrako 3y ago

Why WiFi? We have 60Ghz line of sight WiGig for this kind of thing. It's 40gbps which with some compression could do 8k screens. Besides you could use eye tracking to only transmit what's being seen in high quality. https://en.m.wikipedia.org/wiki/IEEE_802.11ay https://en.m.wikipedia.org/wiki/IEEE_802.11ay

nox101 3y ago

And yet my Quest will do 6 screens just fine. You don't need every the content of each screen to update at 60-120fps. You only need it's representation to move smoothly. If I had a video or game in every screen then yes, I might need the bandwidth. But most screens are mostly static with only the cursor and my input making any changes.