19 ms·

But you don't really. (Well, I don't). It seems obvious to me that there's a progression. Back in Apple II, C64, DOS days computers generally ran a single app

But you don't really. (Well, I don't). It seems obvious to me that there's a progression. Back in Apple II, C64, DOS days computers generally ran a single app at a time. Windowing systems were invented allowing multiple apps to share the same screen. This is what we've had for 30-40 years. For AR though we need multiple apps to share the same 3D space all running at the same time just like I have 14 apps running on my laptop right now. I need my mapping app drawing the path to my destination. I need my transportation app showing me the next bus/train coming. I need my virtual pet app adding my virtual pet around my feet. I need my social networking app to show me my friends info next to them in me view. I need my dating app highlighting people that are potential matches. All of this has to happen simultaneously and at 90fps. That will require an entirely new OS and possibly new hardware (dual GPUs, ... or pre-emptable GPUs) one for compositing the 3D world at 90fps and one for apps so if app takes too long to generate it's portion of the scene the entire OS doesn't start running low. An old OS that only runs a single 3D app at a time and who's apps don't share the same 3D space will be a huge hindrance to AR being truly revolutionary. I'm not saying Magic Leap's OS is that OS but that OS is not MacOS/iOS/Windows/Linux/Android

Klathmon 8y ago
Why can't that OS be one of the standard ones? Sure, it might require special hardware and software (even your screen has it's own software and processing that it is doing on the signals coming from your desktop), and at the end of the day the OS would need to change and evolve to handle this new paradigm, but it's not impossible to imagine a system where the "PC" does what it does now, and just tells the "VR" sub-processor/sub-machine where to put the rendered "window". Already there are some VR apps that basically wrap your normal "windows" desktop in almost a cylinder around you and you can move screens around on it as much as you want with current VR headsets.
- Stanleyc23 8y ago
  Imo it's not that the traditional OSs can't be the OS of the future. It's that the future OS needs to be developed from scratch without carrying over the assumptions and obligations to older design principles. Historically, this has been very difficult. E. G. Cocoatouch (ios) has significant differences to its cocoa (osx). Bottom line is you have to go all to do it right, but its risky for those big companies that have a financial interest in squeezing out as much value as possible from their cashcow OSs
  - kthejoker2 8y ago
    One of Andromeda's target platforms is AR (and not just HL2.) And Linux of course gets it. Android, too. Plenty of experimental builds out there, I'm not worried about pivoting.
- azinman2 8y ago
  And those are terrible experiences.
teddyh 8y ago
> I'm not saying Magic Leap's OS is that OS but that OS is not MacOS/iOS/Windows/Linux/Android One of those is not like the others: “Linux” is not a graphical environment. If you had said “X11” or “Wayland” (or even “Gnome” or “KDE”, etc.), I would have agreed with you, but “Linux” could perfectly well be the base of an entirely new 3D-based application environment.
bograt 8y ago
> For AR though we need multiple apps to share the same 3D space all running at the same time This is something that I've given some practical thought to. The approach that I considered was to treat each application process as a client to a '3D space' service through which it could add and modify defined geometry. Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience. I imagined that the service interface itself would mandate the availability of simplified geometries together with meta information. This would allow the service to appropriately degrade the 3D rendered environment to maintain high framerates. I think this approach could be effective and would not require a new OS.
- XorNot 8y ago
  SteamVR basically does this - it is a very evolved implementation of a VR OS. Apps which crash or otherwise stop rendering trigger a fade out to the white-room VR space, whereas frame-skips are handled by hard drops (so the effect is like a teleport to avoid motion sickness). Like, Valve have really done an incredible job on the user application interface there.
- photojosh 8y ago
  I had a similar thought a few weeks ago, but for smartphone camera and mapping apps (and the intersection of the two when it comes to world-based AR). Rather than every new app out there having to implement their own camera and map view, have them provide an extension that can be called by the native camera/map apps (providing the view bounds/coordinates/etc), that returns a rendered layer that gets overlaid on the view. Examples: In my state we have a govt funded web/native app that provides the fuel prices at all the service stations around you. If there was a mapping extension you could turn on, it would show this directly within the mapping app, and also an indicator "cheapest fuel within X km is at Y location" when you're in navigation mode. In camera view with an extension turned on, provide object detection similar to how QR codes are recognised.
- dingo_bat 8y ago
  > Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience. This is already true with even the oldest and most rudimentary AR/VR systems, eg, Gear VR. The interface and head tracking is 90fps all the time. If the app is slow, the viewport will not freeze. Just it will be stuck on one frame.

Klathmon 8y ago

Why can't that OS be one of the standard ones? Sure, it might require special hardware and software (even your screen has it's own software and processing that it is doing on the signals coming from your desktop), and at the end of the day the OS would need to change and evolve to handle this new paradigm, but it's not impossible to imagine a system where the "PC" does what it does now, and just tells the "VR" sub-processor/sub-machine where to put the rendered "window". Already there are some VR apps that basically wrap your normal "windows" desktop in almost a cylinder around you and you can move screens around on it as much as you want with current VR headsets.

Stanleyc23 8y ago
Imo it's not that the traditional OSs can't be the OS of the future. It's that the future OS needs to be developed from scratch without carrying over the assumptions and obligations to older design principles. Historically, this has been very difficult. E. G. Cocoatouch (ios) has significant differences to its cocoa (osx). Bottom line is you have to go all to do it right, but its risky for those big companies that have a financial interest in squeezing out as much value as possible from their cashcow OSs
- kthejoker2 8y ago
  One of Andromeda's target platforms is AR (and not just HL2.) And Linux of course gets it. Android, too. Plenty of experimental builds out there, I'm not worried about pivoting.
azinman2 8y ago
And those are terrible experiences.

teddyh 8y ago

> I'm not saying Magic Leap's OS is that OS but that OS is not MacOS/iOS/Windows/Linux/Android One of those is not like the others: “Linux” is not a graphical environment. If you had said “X11” or “Wayland” (or even “Gnome” or “KDE”, etc.), I would have agreed with you, but “Linux” could perfectly well be the base of an entirely new 3D-based application environment.

bograt 8y ago

> For AR though we need multiple apps to share the same 3D space all running at the same time This is something that I've given some practical thought to. The approach that I considered was to treat each application process as a client to a '3D space' service through which it could add and modify defined geometry. Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience. I imagined that the service interface itself would mandate the availability of simplified geometries together with meta information. This would allow the service to appropriately degrade the 3D rendered environment to maintain high framerates. I think this approach could be effective and would not require a new OS.

XorNot 8y ago
SteamVR basically does this - it is a very evolved implementation of a VR OS. Apps which crash or otherwise stop rendering trigger a fade out to the white-room VR space, whereas frame-skips are handled by hard drops (so the effect is like a teleport to avoid motion sickness). Like, Valve have really done an incredible job on the user application interface there.
photojosh 8y ago
I had a similar thought a few weeks ago, but for smartphone camera and mapping apps (and the intersection of the two when it comes to world-based AR). Rather than every new app out there having to implement their own camera and map view, have them provide an extension that can be called by the native camera/map apps (providing the view bounds/coordinates/etc), that returns a rendered layer that gets overlaid on the view. Examples: In my state we have a govt funded web/native app that provides the fuel prices at all the service stations around you. If there was a mapping extension you could turn on, it would show this directly within the mapping app, and also an indicator "cheapest fuel within X km is at Y location" when you're in navigation mode. In camera view with an extension turned on, provide object detection similar to how QR codes are recognised.
dingo_bat 8y ago
> Failure/slowness of any given application would leave the existing application geometry in-situ within the 3D environment and avoid the most jarring extremes of user experience. This is already true with even the oldest and most rudimentary AR/VR systems, eg, Gear VR. The interface and head tracking is 90fps all the time. If the app is slow, the viewport will not freeze. Just it will be stuck on one frame.