4 ms·

> The reality-shaping power of Apple's demo maestros is truly admirable. It really isn't just the demos that wow us, and it isn't just the high-end device. The

> The reality-shaping power of Apple's demo maestros is truly admirable. It really isn't just the demos that wow us, and it isn't just the high-end device. The device is packed with tech, for sure. But 80% of the story here is Apple's strong design choices: "Here's how you will interact with the world. Here's how other people fit in. This is how UI is going to display. We rewrote our entire UI stack on top of a physically based renderer so developers can comfortably recompile their millions of existing apps. Here's how we process-isolated the gaze tracking so no third-party app will be able to data-mine it for personalised Ads" They didn't just build a headset and call it a day. They didn't show useless, aspirational demos (though during the keynote, Disney certainly showed a whole lot of rubbish). They developed the UI paradigms, human interface guidelines, put huge thought towards accessibility. Those of us impressed with this are not wowed by just hardware, we're amazed by the amount and strength of design thought put into every aspect of your interaction with this computer. There's so much _opinion_ about what this computer should be, and how we should use it, that's the exciting thing Their opinions may not pan out in the future. But I'm so glad _someone_ is having strong opinions about human computer interaction in the AR/VR space, because Meta and the rest certainly don't

taeric 3y ago
Pretty sure if I have my headset on and I'm in the middle of the room with my kids, I'm not a "hip father" doing cool shit with my kids. Consider if this was someone on a phone in the kitchen discussing something. We wouldn't think, "how progressive this person is to have work conversations while mostly ignoring his kids." Granted, as a parent, the scene of the airline person upset that there is a kid on the flight is... off putting. To the extreme. Especially when the response is "ugh, thank goodness I can turn off all visual senses around me." SMH And I am struggling to see any UI design/consideration that was new in this clip. Do you have a rundown on exactly what was amazing?
- interpol_p 3y ago
  Oh I totally agree. The scenes with parents/kids were so off, and felt emotionally jarring > And I am struggling to see any UI design/consideration that was new in this clip. Do you have a rundown on exactly what was amazing? The State of the Union video has more detail. All of SwiftUI and UIKit now sits on top of their material-based renderer. You can now render a Metal Shader on a SwiftUI view in one line of code. They have consistent and detailed paradigms for how menus, toolbars, productivity apps etc should work on this platform (likely from porting all of their own). 3D models fit seamlessly into the UI view hierarchy. They have detailed design and interaction constraints for all new types of spatial windows, as well as legacy apps. All of this even goes back to their Objective-C APIs, which are updated appropriately Then they've implemented things like process isolation for eye-gaze tracking. Simply so apps can't read your gaze and abuse that data. No one else makes decisions like this in a v1 product. It probably required a whole team of people to get right, considering that gaze is the primary interaction method a user will be using with your app It's hard to explain just how much depth of thought has been put into this platform. They have addressed a lot of hard questions with some very good design, and above all, they are opinionated about their platform. In every corner and detail there is a strong design choice about the right way to design and implement that interaction. That opinionated design is why people get so excited, and it can be so jarring when they get it wrong
  - taeric 3y ago
    The "material-based renderer" sounds... fluff? Most any system should have it so that rendering a menu is a single line of code. Same for toolbars and other. Those are typically simple registrations of ("name" "description" handler). Anything more that that, and you are doing tailored menu/toolbar and are straying on purpose. That leads, then, to the spatial windows and APIs that go with them. And... quite frankly, I have zero faith that anybody can pull off a V1 for that API. Expect churn and capability growth. For the process isolation on eye-gaze tracking... assuming you can get "hover" events for things you are gazing at, than I fail to see how they can keep you from reading the gaze. I expect they will try and lock down abuses of that, but as soon as you enable coding against what a user is looking at, developers will find a way to leak that. And if they expect that the primary way I will interact with an application is by looking at it.... that feels very very sad, all told. I don't want to just look at things, I want to manipulate them. And for that, I am almost certainly going to want some haptic/tactile interaction. Such that this won't live on its own. Happy to be proven wrong, of course. And, for what it is, this does look impressive. I just don't buy the marketing spin on it, at the moment. Way way way too "dreams work, bro!"
    - interpol_p 3y ago
      The gaze tracking is process isolated. Only when you tap is that location ever communicated to the application process — so they had to build a UI API that supports configurable hover behaviour where that hover behaviour is never readable by developers They've thought through the accessibility features, how VoiceOver works with the headset, dynamic type sizes, etc. Designing this sort of consistent API across a whole platform, where it's not only easy to use, but really delightful, is the bit that's exciting to a lot of people I have collected a bunch of A/VR headsets and developed for them. None of them think deeply about these things, so they just haven't been that interesting. They are more "we supply the hardware, you supply the opinion about how it should be used." I really want the other side of that story The Disney part of the keynote was all "dreams work." It was complete nonsense with no clear direction. Apple's stuff, while it might be wrong, is at least opinionated and clear There is a lot of marketing sheen, but there's a clear design opinion about human computer interaction that comes through. That's the exciting bit that I think people react to, sometimes without realising

taeric 3y ago

Pretty sure if I have my headset on and I'm in the middle of the room with my kids, I'm not a "hip father" doing cool shit with my kids. Consider if this was someone on a phone in the kitchen discussing something. We wouldn't think, "how progressive this person is to have work conversations while mostly ignoring his kids." Granted, as a parent, the scene of the airline person upset that there is a kid on the flight is... off putting. To the extreme. Especially when the response is "ugh, thank goodness I can turn off all visual senses around me." SMH And I am struggling to see any UI design/consideration that was new in this clip. Do you have a rundown on exactly what was amazing?

interpol_p 3y ago
Oh I totally agree. The scenes with parents/kids were so off, and felt emotionally jarring > And I am struggling to see any UI design/consideration that was new in this clip. Do you have a rundown on exactly what was amazing? The State of the Union video has more detail. All of SwiftUI and UIKit now sits on top of their material-based renderer. You can now render a Metal Shader on a SwiftUI view in one line of code. They have consistent and detailed paradigms for how menus, toolbars, productivity apps etc should work on this platform (likely from porting all of their own). 3D models fit seamlessly into the UI view hierarchy. They have detailed design and interaction constraints for all new types of spatial windows, as well as legacy apps. All of this even goes back to their Objective-C APIs, which are updated appropriately Then they've implemented things like process isolation for eye-gaze tracking. Simply so apps can't read your gaze and abuse that data. No one else makes decisions like this in a v1 product. It probably required a whole team of people to get right, considering that gaze is the primary interaction method a user will be using with your app It's hard to explain just how much depth of thought has been put into this platform. They have addressed a lot of hard questions with some very good design, and above all, they are opinionated about their platform. In every corner and detail there is a strong design choice about the right way to design and implement that interaction. That opinionated design is why people get so excited, and it can be so jarring when they get it wrong
- taeric 3y ago
  The "material-based renderer" sounds... fluff? Most any system should have it so that rendering a menu is a single line of code. Same for toolbars and other. Those are typically simple registrations of ("name" "description" handler). Anything more that that, and you are doing tailored menu/toolbar and are straying on purpose. That leads, then, to the spatial windows and APIs that go with them. And... quite frankly, I have zero faith that anybody can pull off a V1 for that API. Expect churn and capability growth. For the process isolation on eye-gaze tracking... assuming you can get "hover" events for things you are gazing at, than I fail to see how they can keep you from reading the gaze. I expect they will try and lock down abuses of that, but as soon as you enable coding against what a user is looking at, developers will find a way to leak that. And if they expect that the primary way I will interact with an application is by looking at it.... that feels very very sad, all told. I don't want to just look at things, I want to manipulate them. And for that, I am almost certainly going to want some haptic/tactile interaction. Such that this won't live on its own. Happy to be proven wrong, of course. And, for what it is, this does look impressive. I just don't buy the marketing spin on it, at the moment. Way way way too "dreams work, bro!"
  - interpol_p 3y ago
    The gaze tracking is process isolated. Only when you tap is that location ever communicated to the application process — so they had to build a UI API that supports configurable hover behaviour where that hover behaviour is never readable by developers They've thought through the accessibility features, how VoiceOver works with the headset, dynamic type sizes, etc. Designing this sort of consistent API across a whole platform, where it's not only easy to use, but really delightful, is the bit that's exciting to a lot of people I have collected a bunch of A/VR headsets and developed for them. None of them think deeply about these things, so they just haven't been that interesting. They are more "we supply the hardware, you supply the opinion about how it should be used." I really want the other side of that story The Disney part of the keynote was all "dreams work." It was complete nonsense with no clear direction. Apple's stuff, while it might be wrong, is at least opinionated and clear There is a lot of marketing sheen, but there's a clear design opinion about human computer interaction that comes through. That's the exciting bit that I think people react to, sometimes without realising