4 ms·

I imagine it's mostly a CPU/battery/bandwidth concern at this point — having one wireless 4.5k/60 (or is it 90?) stream from your Mac is difficult enough; havin

by klausa 3y ago

I imagine it's mostly a CPU/battery/bandwidth concern at this point — having one wireless 4.5k/60 (or is it 90?) stream from your Mac is difficult enough; having a potentially unbound number of them (for arbitrary number of Mac windows open) is a different problem altogether. But there are 3rd party solutions that will let you do just that though: https://github.com/saagarjha/Ensemble https://github.com/saagarjha/Ensemble

huytersd 3y ago
This is it exactly. I’m surprised the HN crowd isn’t immediately picking up the bandwidth issues with streaming unlimited 4k screens from your MacBook.
- deutschepost 3y ago
  Noted that this would be a non issue if you could just connect a cable between your Mac and the Vision. You already have the huge battery pack dangling off the side. This is an apple issue, not a technical one.
  - saagarjha 3y ago
    I think Apple sees a device that doesn't have any cables in the near future; they just weren't able to pull it off this time.
    - wahnfrieden 3y ago
      It’s the product vision. But progress is slow reportedly because of physical and current scientific tech limits. Their product vision could take another 10-30 years.
  - klausa 3y ago
    Not really? Do the math of how much bandwidth like three windows/screens (because with this model each window is basically its own screen) at 4K/100hz/10bit color each would take. You're at limits of TB4 _very quickly_. You can compress the image, try to do something smart with foveated rendering (only stream the windows that users are looking at; but that breaks if you want to keep a window with logs in your peripheral vision), use chroma subsampling, etc; but those all are varying trade-offs with relation to image quality.
    - byteware 3y ago
      why would foveated rendering break down? it does not stop rendering where you are not looking just lowers the resolution
      - klausa 3y ago
        It'd break if you wanted to do the dumbest thing and just not stream the windows users aren't looking at; but lowering the stream resolution on the fly could work but now involves more complexity (on both sides to communicate when to adjust the resolution) and because it's not handled entirely on-device breaks the illusion of it being invisible. I can also imagine it having some weird privacy implications; like Mac apps somehow monitoring this and tracking whether you're actually looking at them, etc.
        saagarjha 3y ago
        If you're OK with rendering everything at high resolution (and then choosing what quality to send over) then you shouldn't have any privacy issues, assuming that this part is done by the OS.
      - numpad0 3y ago
        Foveated rendering don't work but not the way described in GP; human eyes are too fast against current latency and frame intervals for it to work.
    - deutschepost 3y ago
      Foveated rendering doesn’t stop streaming if the user isn’t looking at the window. Just streams in lower quality. If you would just trivially stream all windows to the vision of course it will at some point be at the limit of current technology. But I would assume a company like apple has the means to push the state of transmission media (rather than just using a now 4 year old standard) and be able to think of something smarter than „just stream all windows in maximum quality even if the user doesn’t look at them“
      - 3y ago
  - huytersd 3y ago
    No, it’s a processor and graphics card issue as well. You have to actually render a potentially unlimited number of 4K screens (at least the ones in your direct view) on the Mac Pro. It was never built to do that.
- hexomancer 3y ago
  AVP knows exactly which monitor you are looking at. The mac doesn't need to render all monitors at full resolution, just the one being looked at.
  - huytersd 3y ago
    That sort of seamless render switching doesn’t sound technologically doable if you include that the render is happening on a different machine. I could be wrong.
- matsemann 3y ago
  Because I don't care about the technical limitations. I care about if it's a useful gadget for me or not. It doesn't matter why a feature isn't there, that just becomes a "you're holding it wrong" thing.
- frompom 3y ago
  AVP already has foveated rendering so if MacOS had some awareness of what is being rendered it could potentially render unlimited windows since it would only need to render and stream enough to handle what is being looked at. The primary problem I would guess here would be latency though so maybe not feasible. The other possibility is if the actual rendering happens on device instead of streaming pixels. It would dramatically decrease bandwidth required. I think there's a solution somewhere for this.

by klausa 3y ago

huytersd 3y ago

This is it exactly. I’m surprised the HN crowd isn’t immediately picking up the bandwidth issues with streaming unlimited 4k screens from your MacBook.

deutschepost 3y ago
Noted that this would be a non issue if you could just connect a cable between your Mac and the Vision. You already have the huge battery pack dangling off the side. This is an apple issue, not a technical one.
- saagarjha 3y ago
  I think Apple sees a device that doesn't have any cables in the near future; they just weren't able to pull it off this time.
  - wahnfrieden 3y ago
    It’s the product vision. But progress is slow reportedly because of physical and current scientific tech limits. Their product vision could take another 10-30 years.
- klausa 3y ago
  Not really? Do the math of how much bandwidth like three windows/screens (because with this model each window is basically its own screen) at 4K/100hz/10bit color each would take. You're at limits of TB4 _very quickly_. You can compress the image, try to do something smart with foveated rendering (only stream the windows that users are looking at; but that breaks if you want to keep a window with logs in your peripheral vision), use chroma subsampling, etc; but those all are varying trade-offs with relation to image quality.
  - byteware 3y ago
    why would foveated rendering break down? it does not stop rendering where you are not looking just lowers the resolution
    - klausa 3y ago
      It'd break if you wanted to do the dumbest thing and just not stream the windows users aren't looking at; but lowering the stream resolution on the fly could work but now involves more complexity (on both sides to communicate when to adjust the resolution) and because it's not handled entirely on-device breaks the illusion of it being invisible. I can also imagine it having some weird privacy implications; like Mac apps somehow monitoring this and tracking whether you're actually looking at them, etc.
      - saagarjha 3y ago
        If you're OK with rendering everything at high resolution (and then choosing what quality to send over) then you shouldn't have any privacy issues, assuming that this part is done by the OS.
    - numpad0 3y ago
      Foveated rendering don't work but not the way described in GP; human eyes are too fast against current latency and frame intervals for it to work.
  - deutschepost 3y ago
    Foveated rendering doesn’t stop streaming if the user isn’t looking at the window. Just streams in lower quality. If you would just trivially stream all windows to the vision of course it will at some point be at the limit of current technology. But I would assume a company like apple has the means to push the state of transmission media (rather than just using a now 4 year old standard) and be able to think of something smarter than „just stream all windows in maximum quality even if the user doesn’t look at them“
    - 3y ago
- huytersd 3y ago
  No, it’s a processor and graphics card issue as well. You have to actually render a potentially unlimited number of 4K screens (at least the ones in your direct view) on the Mac Pro. It was never built to do that.
hexomancer 3y ago
AVP knows exactly which monitor you are looking at. The mac doesn't need to render all monitors at full resolution, just the one being looked at.
- huytersd 3y ago
  That sort of seamless render switching doesn’t sound technologically doable if you include that the render is happening on a different machine. I could be wrong.
matsemann 3y ago
Because I don't care about the technical limitations. I care about if it's a useful gadget for me or not. It doesn't matter why a feature isn't there, that just becomes a "you're holding it wrong" thing.
frompom 3y ago
AVP already has foveated rendering so if MacOS had some awareness of what is being rendered it could potentially render unlimited windows since it would only need to render and stream enough to handle what is being looked at. The primary problem I would guess here would be latency though so maybe not feasible. The other possibility is if the actual rendering happens on device instead of streaming pixels. It would dramatically decrease bandwidth required. I think there's a solution somewhere for this.