11 ms·

One of the aspects of the device that has been under-realized is that when mirroring your desktop/laptop display to the AVP, you can't break out its application

• orenlindsey 3y ago
  You can put VisionOS apps next to the Mac desktop, so it isn't as much of a problem as it seems.
• jwells89 3y ago
  I wouldn’t be surprised if this came in a visionOS update. On a shorter timescale it could also come in the form of third party apps, because there’s no technical limitations preventing a server app from cutting out windows on a desktop OS and sending them over a wire to a visionOS client.
  • spogbiper 3y ago
    There are third party apps that do this already on the Quest. I believe they can replicate Mac screens, they definitely can replicate Windows PC screens into the VR space. If Apple doesn't provide a 1st party solution I suspect someone else will soon.
  • orangecat 3y ago
    On a shorter timescale it could also come in the form of third party apps If Apple approves it, of course. This is one of my major concerns; there's a lot of potentially useful functionality that could be implemented, but you have to jump through the app store hoops and hope that Apple doesn't decide that it conflicts with their idea of what you should be allowed to do.
    • jwells89 3y ago
      Functionally speaking these apps would be scarcely distinguishable from the plethora of screen streaming apps that exist on the App Store already, like Screens and Moonlight. Of course Apple could reject these apps anyway but it seems unlikely.
• leetharris 3y ago
  I agree that this should be considered long term, however... you are able to snap VisionOS / iPadOS apps anywhere around your Macbook view AND you are able to control those very apps with your Macbook trackpad. So even though you have a sequestered Mac output alongside Vision apps, you can use the same controls for all them simultaneously. This should help in the interim.
  • mjamesaustin 3y ago
    Yeah when I found this out, it resolved my concerns. Most of my apps will have a native Vision release (email, web browser, slack, etc.) and my actual monitor screen will only need more professional software (e.g. Photoshop, Illustrator, InDesign).
• cududa 3y ago
  It seems very reasonable this will be a future feature. I've long suspected iPad OS' stage manager feature shipping so half baked was really more of getting the platform ready to support multiple apps and easier manipulation (from a developer perspective) of the double buffered "window" textures - given Vision Pro is based on iPad OS. With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation. Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option
  • refulgentis 3y ago
    They shipped iPad Stage Manager half-baked, to get iPad developers ready for double-buffered windows, so they could eventually ship the visionOS macOS integration half-baked? Doesn't sound right at all to my ears, even though I'm stoked for my order! EDIT: -5* doesn't make sense, this is the most polite way you can point out that getting macOS apps windowed on visionOS has ~0 to do with double-buffered windows on iPad OS. n.b. I didn't use half-baked, OP did.
    • cududa 3y ago
      No, I'm saying they shipped iPad stage manager half baked for their own uses/ to refine for AVP. I'm positing that a major reason for macOS stage manager's existence is as a transport layer/ "texture formatter"
    • Terretta 3y ago
      I don't think it's half-baked. I think it's lightly toasted. :-) I use iPad Pro as a kind of sidecar daily driver, in the magnetic dock magic keyboard w/ trackpad. As I type this, the screen shows a traditional MacOS style dock across bottom, four Stage Manager window clusters I can tap with a thumb on the left, and Safari plus Messages taking 2/3 and 1/2 of screen respectively. There's more app and pixel real estate than most Windows laptops, and bringing screen sets to the foreground or swapping them back to the side is so natural I almost feel like giving up that space on my Mac as well. The big thing I saw happen from apps over the past two version of iOS is app devs realizing their windows will not always be full screen or 1/2 screen size, but arbitrary size. By now, most iPad apps of any serious nature are effectively window size independent, making them play well with others in stage manager. It's easy to see how that would make them play well with the headset one day.
• losvedir 3y ago
  Yeah, I don't think "mirroring" is quite the right term. It's effectively a 4K monitor for the laptop, with the laptop screen going black. Most (all?) Mac laptops don't have a 4K screen, so you have more screen real estate than "mirroring" would make you think. But this is sufficient for many use cases (or at least, mine). I pre-ordered one with the idea that my main work will be on the 4K monitor, with most of my superfluous apps floating around as native visionOS apps. That's mail, a web browser, and zoom, which all have apps now, and Slack, which I could just use Safari for but may have a native app in the future.
  • grumbel 3y ago
    > 4K monitor It's more like 1080p monitor. The virtual monitor only covers a small part of the VisionPro's display. You can compensate a bit for a lack of resolution by making the virtual screen bigger or by leaning in, but none of that gives you a 4k display. To really take proper advantage of the VR environment you really need the ability to pull out apps into their own windows, as than you can move lesser used apps into your peripheral vision and leave only the important stuff right in front of you. You also miss out on the verticality that VR offers when you are stuck with a virtual 16:9 screen.
    • brookst 3y ago
      4k is important because of perspective, rotation, and aliasing. Just sending 1080p would look terrible.
    • threeseed 3y ago
      It is a 1440p display. Which is the resolution that the majority of PC users are likely using.
  • xattt 3y ago
    Is it mirrored as some HEVC video stream from the laptop, or are UI elements actually rendered on headset itself?
    • saagarjha 3y ago
• hyperthesis 3y ago
  Discontent over this implementation detail shows users are fully sold on the basic idea. Like, if the main complaint about the first Fords was the colour range.
  • nimblegorilla 3y ago
    I think it's a more important feature than just a cosmetic color. Imagine if you bought a truck to haul cargo, but were then told it can only transport one type of cargo at a time. That would suck.
    • hyperthesis 3y ago
      I meant it's a superficial rather than fundamental design flaw, easily rectified.
  • idontpost 3y ago
    [dead]
• nickrubin 3y ago
  It looks like someone is working on a Mac app that does exactly this, and they seem to have a functional prototype: https://x.com/TheOriginaliTE/status/1751251567641346340?s=20 https://x.com/TheOriginaliTE/status/1751251567641346340?s=20
  • wahnfrieden 3y ago
    Unclear if Apple will allow this in the store edit: Yes I know you can build apps before they're in the store
    • LeoPanthera 3y ago
      As long as it's open source, you can sideload onto any iOS device by building it yourself.
      • deleted 3y ago
        [deleted]
      • dns_snek 3y ago
        Iff you pay an annual $100 dev fee. Yes I know you can technically do this without a paid dev account, but it's practically useless because it has to be re-done every 7 days.
        • saagarjha 3y ago
          Surely considering the fact that you are streaming your Mac to Apple Vision Pro you should be able to easily resign it on that very device? :)
          • dns_snek 3y ago
            I don't know about you, but there's nothing easy about having to boot up an IDE every 7 days to sign and re-install an app you depend on. It's just one more thing to worry about and mentally keep on top of, the opposite to Apple's convenience and smooth UX ethos.
• zitterbewegung 3y ago
  The largest portable MacBook Pro 16.2 inch has a 3456-by-2234 native resolution at 254 pixels per inch which by default is halved . So I don’t know what she means exactly about 4k but there are enough pixels to do a portable 4k display.
• tiltowait 3y ago
  The inability to break out Mac windows curbed a lot of my enthusiasm for the AVP. I hope Apple will eventually add it, but I'm not going to spend $3500 on that hope.
  • pmarreck 3y ago
    The fact that pretty much everyone who owns both a Vision Pro and a Mac would want that feature means it's probably going to happen.
  • sleepybrett 3y ago
    It will be less of an issue for me if we start seeing native builds of popular IDEs like xcode, intelli-j/goland, etc for vision pro (and other apps for other people say photoshop). I think of the 'projected screen' feature more like a compatibility layer like Rosetta 2. You use it until you get a native build then it stops being a thing you bother with.
    • callalex 3y ago
      That will never happen. Nobody will make real software for iOS because nobody wants to pay a 30% fee for the privilege of randomly having their whole business shut down when an app reviewer is in a bad mood.
      • huytersd 3y ago
        What world do you live in? The iOS App Store is probably the most full fledged and populated app ecosystem there is.
        • callalex 3y ago
          And yet none of the software GP mentioned are available on it.
          • huytersd 3y ago
• dirtyv 3y ago
  Appears possible in theory: https://github.com/saagarjha/Ensemble https://github.com/saagarjha/Ensemble
• wtallis 3y ago
  Streaming an arbitrary collection of windows instead of a single finished, composited framebuffer increases the bandwidth requirements by at least an order of magnitude. That's never going to work well over WiFi.
  • ehsankia 3y ago
    As long as the total number of pixels is less, I don't see what that has to be true, at least bandwidth wise? Compute wise, the vision might have to do slightly more to separate the buffer and composite them into the AR view in different places, but the bandwidth should be direction proportional to the number/size of each window. If I can fit all the windows on a 4K screen, then I don't see why the software can't split that and lay it out separate in my view instead of in a single rectangle.
    • wtallis 3y ago
      It seems pretty obvious that nobody wants to be limited to the total number of pixels being merely equivalent to tiling across a 4k screen.
    • saagarjha 3y ago
      Some of the windows will be obscured by others. If you stream a 100 windows to visionOS, it's possible to lay them out so that none of them cover each other and you have to render them all. On a flat screen there is a limit to how many pixels you need to paint.
  • jayd16 3y ago
    Just compress the stream. Total pixels increase the vram on the device but popping out a static window shouldn't take any more than a trivial amount of streaming bandwidth.
    • wtallis 3y ago
      Everything is already compressed. Uncompressed 1080p is 3Gbit/s, which is already well beyond what's actually achievable with WiFi. Allowing an unbounded number of window surfaces to be streamed invites people to use it in a way that current technology simply cannot provide the kind of slick experience that Apple needs from this product.
• mtillman 3y ago
  the iPhone debuted without copy/paste. They'll get to it but maybe not immediately.
• timenova 3y ago
  Further, I wish they added support to make multiple virtual monitors from macOS Workspaces, like what happens today when you attach another monitor. Switching workspaces can be bound to keys in the Keyboard Settings. Moving windows to other workspaces is easy to do with third-party apps like Amethyst. It feels like the Vision Pro would definitely be a great replacement for people who (want to) buy multiple expensive monitors, but it doesn't fully reach that potential today, and mostly because of software? Although rendering 3 or 4 virtual workspaces through ad-hoc Wifi at 4K 60fps+ low-latency would certainly be a huge challenge.
  • reactordev 3y ago
    I do this sometimes on my meta quest. Go into desktop VR and pull up a couple desktop views so I can see things happen in real time on different “screens”.
• huytersd 3y ago
  I guess there is something to the macpro being able to handle the output for one screen at a time. If it has to render 4k outputs for 10 different screens simultaneously, performance is going to suffer.
• joshspankit 3y ago
  On Windows of all places (95ish to MEish) there was a remote tool called radmin and it had something that I wish companies had embraced: it hooked in to (maybe even before?) the window-rendering functions and sent the changes over the network. It’s hard to explain exactly what I mean because everyone is so used to streaming pictures of the screen over the network (if they even use remote access at all), but you could have less than 20ms latency while controlling over the internet while using tiny amounts of data (50kbps? 100? not sure but somewhere around there). OSX had the opportunity to follow that path before settling on the “render windows, capture the screen, compress the image, send it over the network to be decompressed” VNC-style remote access that’s bog-standard today, and if they had Vision Pro would be set up to be an absolute mind-blowing macOS experience.
  • jackvalentine 3y ago
    X windows system basically does that iirc, and I remember the magic you speak of.
    • int_19h 3y ago
      And yet in practice most X traffic these days is bitmaps.
  • lbussell 3y ago
    Yeah, you can do this with x forwarding on Linux. Not sure if there’s a modern Wayland equivalent.
    • aidenn0 3y ago
      Wayland clients don't draw things the way old-school X clients do (neither do modern X clients), so it doesn't make sense at the Wayland level. KDE or GTK could potentially implement something like this though.
  • sterlind 3y ago
    isn't that how Xorg remoting used to work as well? the display server and client are separate, so whether the pipe was local or remote didn't matter. In principle, Wayland could do it too, I think, if there were a way to synchronize texture handles (the Wayland protocol is also message-based, but IPCs GPU handles around instead of copying bitmaps.) I guess one downfall is that that your pipe has to be lossless, and there's no way to recover from a broken pipe (unless you keep a shadow copy of the window state, and have a protocol for resynchronizing from that, and a way to ensure you don't get out of sync.)
• freeone3000 3y ago
  That’s really weird, because even the hololens has this feature. Multiple windows, multiple desktops is how we want to work.