3 ms·

It’s pretty boring because it boils down to “I can totally imagine myself doing X with a headset” and “I can’t imagine myself doing X with a headset” — the oper

by pavlov 3y ago

It’s pretty boring because it boils down to “I can totally imagine myself doing X with a headset” and “I can’t imagine myself doing X with a headset” — the operative word being “imagine.” I’d love to read a comparison between visionOS SDK and OpenXR from someone who has actually used the latter. I gather that OpenXR is lower level, but what exactly does that mean for an app developer? How much more work would it be to build a “spatial experience” cross-platform with OpenXR? What would the compromises be for both approaches? Would it be feasible to build an engine that runs on both platforms?

Lorkki 3y ago
> I gather that OpenXR is lower level, but what exactly does that mean for an app developer? Mostly that they won't be using OpenXR directly. It's a standardised layer that lets many kinds of tools and libraries make use of the same hardware.
- pavlov 3y ago
  Well, ok, but since the tools and libraries don’t exist yet… Instead of waiting for somebody else to build middleware and hope it will be good, what would it look like today to design and implement an app that works on both visionOS and OpenXR? That’s a question that I find interesting. Even if I don’t end up building such an app, it would illuminate the differences between Apple’s SDK and the rest of the industry.
jimmySixDOF 3y ago
A lot is still just becoming more clear now the SDK is here today but as usual there are definitely fine print gotchas. There are waay more restrictions in the MR Shared space mode so whatever (Unity/webXR) exports only to RealityKit for rendering with tight Apple conditions. In Full VR mode without Passthrough you have full control so custom shaders but no gaze and a 1.5m motion range limit. The MR mode itself has two flavors single or multi app modes with some tradeoffs like no hand joints with multiapp. Hand Joints are non OpenXR standard so need translation to be crossplatform anyway. And so on. All that said it's great so many experiences can port and there is a lot more interest in WebXR now hopefully WebGPU will fall into place too and it will be interesting to see what we can do with the Internet in 3D.
- gpm 3y ago
  > but no gaze In the WWDC unity-metal demonstration they definitely seemed to have gaze tracking. Have you checked looked through the sdk and confirmed that there is no public way to do this?
  - jimmySixDOF 3y ago
    Yah, its very easy to get confused by them chopping all the sessions up for the conference. It was a little frothy but my understanding now is Gaze is provided in both MR modes known as Bound and Unbound volumes. So look and click works fine. In bound volume you don't get head or hand pose or room geometry. In the Fully immersed VR mode you don't have gaze data for understandable privacy reasons although room geometry could be handy but if it really matters then you build to MR targeting the more restrictive OS requirements like Unity Polyspatial and only Shader Graph. Even the Unity team had to correct a few edge cases around WWDC as the sessions dropped: https://twitter.com/pvncher/status/1666476686576332800?t=9Br3pzXth9JG5kRF_Rj34A&s=19 https://twitter.com/pvncher/status/1666476686576332800?t=9Br...
    - gpm 3y ago
      I don't think that tweet thread addresses gaze in particular? Is there a particular tweet I should be looking at or is it just asserting the (absolutely correct) assertion that it's confusing? --- I'm specifically talking about the "Bring your Unity VR app to a fully immersive space" WWDDC talk, which claimed to be built on Metal and ArKit, and had a very very brief demo in it talking about gaze, screenshot here: https://media.hachyderm.io/media_attachments/files/110/520/911/531/453/396/original/ea8773b2451b6d2f.png https://media.hachyderm.io/media_attachments/files/110/520/9... Note the demo wasn't actually on-device, and gaze was just following where the screen was pointing, so it's definitely possible I'm reading too much into this.
      - jimmySixDOF 3y ago
        No I am sure they don't give you access to the Passthrough data or gaze details to the app in VR mode unless there are quite a few people wrong on this too. And since in VR you are completely outside the rendering subsystem I don't even know how you would interface whats in your app and the gaze hover trigger etc. My guess is the slide is referring to the MR unbound mode (full space they can call it to muddy the water). I would definitely be interested to know I am wrong here. See this : https://twitter.com/Tojiro/status/1667028702734209025?t=1SnM9c3IaBSrDeH76CH_pA&s=19 https://twitter.com/Tojiro/status/1667028702734209025?t=1SnM... EDIT: Checked with our team and so VR mode gaze has event triggers on any Apple UI object without exposing the details so that works unless your UI is not easily portable. And that's the same across all MR and VR modes.
srslack 3y ago
I have a Valve Index and I’ve watched movies in a virtual cinema many times, most of those with friends. It’s immersive but becomes a bit of a sweat box after a few hours. The FOV and resolution are lacking and I can’t use it for virtual monitors. I can see this device fixing most of those problems if the specs are true. But I don’t see myself taking it out in public. That’s laughable. And I’d only drop that price if I was sure that the ecosystem is relatively open.

by pavlov 3y ago

Lorkki 3y ago

> I gather that OpenXR is lower level, but what exactly does that mean for an app developer? Mostly that they won't be using OpenXR directly. It's a standardised layer that lets many kinds of tools and libraries make use of the same hardware.

pavlov 3y ago
Well, ok, but since the tools and libraries don’t exist yet… Instead of waiting for somebody else to build middleware and hope it will be good, what would it look like today to design and implement an app that works on both visionOS and OpenXR? That’s a question that I find interesting. Even if I don’t end up building such an app, it would illuminate the differences between Apple’s SDK and the rest of the industry.

jimmySixDOF 3y ago

A lot is still just becoming more clear now the SDK is here today but as usual there are definitely fine print gotchas. There are waay more restrictions in the MR Shared space mode so whatever (Unity/webXR) exports only to RealityKit for rendering with tight Apple conditions. In Full VR mode without Passthrough you have full control so custom shaders but no gaze and a 1.5m motion range limit. The MR mode itself has two flavors single or multi app modes with some tradeoffs like no hand joints with multiapp. Hand Joints are non OpenXR standard so need translation to be crossplatform anyway. And so on. All that said it's great so many experiences can port and there is a lot more interest in WebXR now hopefully WebGPU will fall into place too and it will be interesting to see what we can do with the Internet in 3D.

gpm 3y ago
> but no gaze In the WWDC unity-metal demonstration they definitely seemed to have gaze tracking. Have you checked looked through the sdk and confirmed that there is no public way to do this?
- jimmySixDOF 3y ago
  Yah, its very easy to get confused by them chopping all the sessions up for the conference. It was a little frothy but my understanding now is Gaze is provided in both MR modes known as Bound and Unbound volumes. So look and click works fine. In bound volume you don't get head or hand pose or room geometry. In the Fully immersed VR mode you don't have gaze data for understandable privacy reasons although room geometry could be handy but if it really matters then you build to MR targeting the more restrictive OS requirements like Unity Polyspatial and only Shader Graph. Even the Unity team had to correct a few edge cases around WWDC as the sessions dropped: https://twitter.com/pvncher/status/1666476686576332800?t=9Br3pzXth9JG5kRF_Rj34A&s=19 https://twitter.com/pvncher/status/1666476686576332800?t=9Br...
  - gpm 3y ago
    I don't think that tweet thread addresses gaze in particular? Is there a particular tweet I should be looking at or is it just asserting the (absolutely correct) assertion that it's confusing? --- I'm specifically talking about the "Bring your Unity VR app to a fully immersive space" WWDDC talk, which claimed to be built on Metal and ArKit, and had a very very brief demo in it talking about gaze, screenshot here: https://media.hachyderm.io/media_attachments/files/110/520/911/531/453/396/original/ea8773b2451b6d2f.png https://media.hachyderm.io/media_attachments/files/110/520/9... Note the demo wasn't actually on-device, and gaze was just following where the screen was pointing, so it's definitely possible I'm reading too much into this.
    - jimmySixDOF 3y ago
      No I am sure they don't give you access to the Passthrough data or gaze details to the app in VR mode unless there are quite a few people wrong on this too. And since in VR you are completely outside the rendering subsystem I don't even know how you would interface whats in your app and the gaze hover trigger etc. My guess is the slide is referring to the MR unbound mode (full space they can call it to muddy the water). I would definitely be interested to know I am wrong here. See this : https://twitter.com/Tojiro/status/1667028702734209025?t=1SnM9c3IaBSrDeH76CH_pA&s=19 https://twitter.com/Tojiro/status/1667028702734209025?t=1SnM... EDIT: Checked with our team and so VR mode gaze has event triggers on any Apple UI object without exposing the details so that works unless your UI is not easily portable. And that's the same across all MR and VR modes.

srslack 3y ago

I have a Valve Index and I’ve watched movies in a virtual cinema many times, most of those with friends. It’s immersive but becomes a bit of a sweat box after a few hours. The FOV and resolution are lacking and I can’t use it for virtual monitors. I can see this device fixing most of those problems if the specs are true. But I don’t see myself taking it out in public. That’s laughable. And I’d only drop that price if I was sure that the ecosystem is relatively open.