3 ms·

Their approach of having a desktop metaphor within the VR environment is one that I have argued for a long time is the "right" approach to eventually grow VR as

Their approach of having a desktop metaphor within the VR environment is one that I have argued for a long time is the "right" approach to eventually grow VR as a general purpose compute platform. The only two other examples of this going right now are Microsoft's Windows Mixed Reality platform with Windows Holographic and the Magic Leap One's interface. Windows MR works really well, Magic Leap is extremely half-baked and has had very little effort put into it (well, at least that was the case a year ago. I ended up selling my headset for lack of use). Being that I've been a Windows developer for 20 years, it's extremely frustrating to me that Microsoft won't make a standalone VR headset. They have all the technical capability to do it. The HoloLens 2 is a sufficiently powerful compute platform and runs the full Windows MR experience. The PC-tethered Windows MR headsets are very high resolution and some of them are even quite comfortable (I regularly use a Samsung Oddyssey+ for work). But they seem hell-bent on pushing difractive waveguide displays for any mobile devices, which I think by now have pretty handily proven to be straight up garbage. How a Linux headset could succeed here where others are really stagnating is to develop: A) Hotswappable batteries. You're not going to get all-day usage out of a single battery pack anytime soon (my Quest 2 with the Elite Strap doubles the battery life to a whole TWO HOURS OMG!), and having to connect a wire to a device that is supposed to be standalone just for power is kind of like playing with one of those line-tethered model airplanes: what you really want is an RC model airplane, so you're just stuck being disappointed all the time. B) A really good spatialized audio system. Spatialized audio is an often overlooked component of making a believable VR scene. All of the ambisonic audio drivers are platform-specific right now, and they each have their pros and cons. I really like Microsoft's HFRT as it seems more realistic than Oculus'. It'd be especially nice to have work put into Chromium and Firefox to upgrade the WebAudio system to use the system's spatializer rather than implement their own (Google's Resonance is particularly bad). C) Really big emphasis on accelerated 2D rendering. 3D rendering is great for games, but work is all text and text is all 2D rendering. Some of the most costly components of my VR projects are the text rendering. You can go a long way with low-fi 3D as an aesthetic, but 2D absolutely needs to be crisp and tight. And it's not just about the resolution of the display. Your 2D buffers aren't going to ever map one-to-one to the display's pixels, because your head is moving around, you'll be looking at things at slight angles and in motion, etc., and most mipmapping algorithms are designed for gradients, not sharp edges. So making clear text is really hard. Oculus has a hack where 2D quads can be rendered in a separate pass from the rest of the 3D environment, but there are issues with it regarding scene compositing and hit testing. D) A native scene graph, something akin to extending the desktop metaphor into 3-dimensions, not just for compositing 2D windows but for allowing 3D applications to mix and match objects. Windows MR and SteamVR are the only systems that really even attempt to do this, with neither really seeing enough emphasis. Every VR app right now runs in a completely exclusive, retained mode. That's fine for games, but it's completely unnecessary for things like teleconferencing apps. Why shouldn't you be able to "spaceshare" like we do screen-sharing? Then you'd be able to have your teleconferencing app separate from your whiteboard app.

georgewsinger 5y ago
Great points. - (A) is tractable, and will be placed on our queue. - RE (B): Simula has developed a special text filter to help improve its text quality. See https://github.com/SimulaVR/Simula#text-quality https://github.com/SimulaVR/Simula#text-quality for a demonstration. As you point out though, there are even deeper things that can be done. We have considered making a vector-based based terminal before (in which text is rendered on a vector basis). - (D) is a point brought up by Forrest Reiling in his master's thesis on window managers (which influenced Simula's early founding). See [Toward General Purpose 3D User Interfaces: Extending Windowing Systems to Three Dimensions](https://github.com/evil0sheep/MastersThesis/blob/master/thesis.pdf?raw=true https://github.com/evil0sheep/MastersThesis/blob/master/thes...).
- moron4hire 5y ago
  Jaron Lanier did a lot of writing on what a VR system interface might look like. He was (well, probably still is, though I take it he doesn't really program anymore) interested in cutting away from traditional process and window models entirely. Indeed, he argued that even something like interprocess communication should take place solely through the VR environment, as bot avatars interfacing with the same virtual control interfaces that a human avatar would have to do. I think his main concern was mostly dog-fooding the VR environment, not building back-door interfaces that the bots could use that the humans could not. I don't know if it really has to go that far, but there is a lot that needs to be done to define common interfaces between objects. Whenever I do my own thinking on how such interfaces would be built, I always end up with something akin to Bluetooth GATT profiles, which is... less than ideal. There's certainly a lot to like about GATT, in that it has a lot of different functionality pre-defined. The dream of device and software interoperability is certainly there. But at the same time, vendors in the wild seem to just shove everything into the public use space and vertically integrate with their own stand alone apps, so maybe Lanier was right. It probably needs to be something more akin to how AR systems attempt to understand their surrounding world. AR apps also run exclusively, but they do have to consider the huge design challenge of not owning the entire environment. Perhaps a model of AR that can't differentiate between the real world and the virtual world that includes other AR apps is the way to go.

georgewsinger 5y ago

Great points. - (A) is tractable, and will be placed on our queue. - RE (B): Simula has developed a special text filter to help improve its text quality. See https://github.com/SimulaVR/Simula#text-quality https://github.com/SimulaVR/Simula#text-quality for a demonstration. As you point out though, there are even deeper things that can be done. We have considered making a vector-based based terminal before (in which text is rendered on a vector basis). - (D) is a point brought up by Forrest Reiling in his master's thesis on window managers (which influenced Simula's early founding). See [Toward General Purpose 3D User Interfaces: Extending Windowing Systems to Three Dimensions](https://github.com/evil0sheep/MastersThesis/blob/master/thesis.pdf?raw=true https://github.com/evil0sheep/MastersThesis/blob/master/thes...).

moron4hire 5y ago
Jaron Lanier did a lot of writing on what a VR system interface might look like. He was (well, probably still is, though I take it he doesn't really program anymore) interested in cutting away from traditional process and window models entirely. Indeed, he argued that even something like interprocess communication should take place solely through the VR environment, as bot avatars interfacing with the same virtual control interfaces that a human avatar would have to do. I think his main concern was mostly dog-fooding the VR environment, not building back-door interfaces that the bots could use that the humans could not. I don't know if it really has to go that far, but there is a lot that needs to be done to define common interfaces between objects. Whenever I do my own thinking on how such interfaces would be built, I always end up with something akin to Bluetooth GATT profiles, which is... less than ideal. There's certainly a lot to like about GATT, in that it has a lot of different functionality pre-defined. The dream of device and software interoperability is certainly there. But at the same time, vendors in the wild seem to just shove everything into the public use space and vertically integrate with their own stand alone apps, so maybe Lanier was right. It probably needs to be something more akin to how AR systems attempt to understand their surrounding world. AR apps also run exclusively, but they do have to consider the huge design challenge of not owning the entire environment. Perhaps a model of AR that can't differentiate between the real world and the virtual world that includes other AR apps is the way to go.