> every app likely won't be responsible for drawing its UX onto the user's vision - it will probably submit some graphic or model to the OS, which will then anc

> every app likely won't be responsible for drawing its UX onto the user's vision - it will probably submit some graphic or model to the OS, which will then anchor that model "onto" reality and handle the user moving their head around it. Much like how every Windows app is not responsible for resizing or moving its window, that's the window manager's job to do. In AR we would probably have a reality manager or something. I don't think this is true. The problem with an app just giving 3D data to the OS is then no apps can compete on quality. There's a reason Cyberpunk 2077 (or whatever the top graphics game is) looks different than Wii Bowling. Even with 2D windows each app does its own rendering using the GPU and provides the results as a rectangle of pixels that the app computed. Some apps look amazing because they used 100% of the GPU power. Some look so-so or maybe they have a great aesthetic (Animal Crossing?) but they aren't spending power on rendering. Now, add that across apps. Your virtual pet app wants to render photo realistic fur where the fur appears to bend in relation to things in the real world, your virtual fashion app wants to show you're friends outrageous auto-reactive clothing, but your navigation app just wants to draw some solid more mostly solid colored lines. The virtual pet app will hog all of the GPU just to draw its fur bogging the entire OS down. It won't be a simple as it was for desktop PCs because you can't let any app slow down the system (which is what happens on PCs) since you need the system to always run at 60 or 90 or 120fps. At best you could maybe make a preemptable GPU (not sure any current GPUs are preemptable), or use 2 GPUs, one for the OS, and one for apps. I think Apple or someone will ship an AR device without solving these issues. I think it will fail just like the Apple Newton failed. Too early. Running AR glasses with an iOS like OS that runs one app at a time is just not a useful paradigm for AR and solving the problem of letting 5 to 20 apps all run at the same time, each using as much GPU as possible for their specific idea of quality, AND have that all run fast enough to keep up with latency issues to actually have the virtual stuff not swim over the real stuff just has a long ways to go. Maybe an M17 but not a M1