4 ms·

Oh I totally agree. The scenes with parents/kids were so off, and felt emotionally jarring > And I am struggling to see any UI design/consideration that was ne

Oh I totally agree. The scenes with parents/kids were so off, and felt emotionally jarring > And I am struggling to see any UI design/consideration that was new in this clip. Do you have a rundown on exactly what was amazing? The State of the Union video has more detail. All of SwiftUI and UIKit now sits on top of their material-based renderer. You can now render a Metal Shader on a SwiftUI view in one line of code. They have consistent and detailed paradigms for how menus, toolbars, productivity apps etc should work on this platform (likely from porting all of their own). 3D models fit seamlessly into the UI view hierarchy. They have detailed design and interaction constraints for all new types of spatial windows, as well as legacy apps. All of this even goes back to their Objective-C APIs, which are updated appropriately Then they've implemented things like process isolation for eye-gaze tracking. Simply so apps can't read your gaze and abuse that data. No one else makes decisions like this in a v1 product. It probably required a whole team of people to get right, considering that gaze is the primary interaction method a user will be using with your app It's hard to explain just how much depth of thought has been put into this platform. They have addressed a lot of hard questions with some very good design, and above all, they are opinionated about their platform. In every corner and detail there is a strong design choice about the right way to design and implement that interaction. That opinionated design is why people get so excited, and it can be so jarring when they get it wrong

taeric 3y ago
The "material-based renderer" sounds... fluff? Most any system should have it so that rendering a menu is a single line of code. Same for toolbars and other. Those are typically simple registrations of ("name" "description" handler). Anything more that that, and you are doing tailored menu/toolbar and are straying on purpose. That leads, then, to the spatial windows and APIs that go with them. And... quite frankly, I have zero faith that anybody can pull off a V1 for that API. Expect churn and capability growth. For the process isolation on eye-gaze tracking... assuming you can get "hover" events for things you are gazing at, than I fail to see how they can keep you from reading the gaze. I expect they will try and lock down abuses of that, but as soon as you enable coding against what a user is looking at, developers will find a way to leak that. And if they expect that the primary way I will interact with an application is by looking at it.... that feels very very sad, all told. I don't want to just look at things, I want to manipulate them. And for that, I am almost certainly going to want some haptic/tactile interaction. Such that this won't live on its own. Happy to be proven wrong, of course. And, for what it is, this does look impressive. I just don't buy the marketing spin on it, at the moment. Way way way too "dreams work, bro!"
- interpol_p 3y ago
  The gaze tracking is process isolated. Only when you tap is that location ever communicated to the application process — so they had to build a UI API that supports configurable hover behaviour where that hover behaviour is never readable by developers They've thought through the accessibility features, how VoiceOver works with the headset, dynamic type sizes, etc. Designing this sort of consistent API across a whole platform, where it's not only easy to use, but really delightful, is the bit that's exciting to a lot of people I have collected a bunch of A/VR headsets and developed for them. None of them think deeply about these things, so they just haven't been that interesting. They are more "we supply the hardware, you supply the opinion about how it should be used." I really want the other side of that story The Disney part of the keynote was all "dreams work." It was complete nonsense with no clear direction. Apple's stuff, while it might be wrong, is at least opinionated and clear There is a lot of marketing sheen, but there's a clear design opinion about human computer interaction that comes through. That's the exciting bit that I think people react to, sometimes without realising
  - taeric 3y ago
    I didn't actually watch the Disney part. I saw the dad "working" in what seemed like a kitchen? Can't remember. The shallow interaction with the kids, though, gave me cringes. Then there is the juxtaposition of, "this let's you present as if there" with the, "of course you have to fly places to be with people, so when you do, zone out like a champ" Of course, zoning out better not need you to interact much, as hearing on a plane is tough, and pinching your neighbor is not smiled upon. I will mostly have to take your word for it that hover actions are not observable by the application. Will make games... More amusing than usual, with no memory of being looked at for characters. And not showing any extra peripherals is a big part of "dreams work." Physical feedback is huge, and a big part of why controllers are needed. Even just the vibration of the standard PS5 controllers go a long way. For driving games, a haptic wheel is more immersive than the vr.
    - interpol_p 3y ago
      > I will mostly have to take your word for it that hover actions are not observable by the application. Will make games... More amusing than usual, with no memory of being looked at for characters. It's described in the Keynote. This is the exact quote: "For example, where you look is very personal. It can give away something about what you're thinking. In Apple Vision Pro, where you look stays private. Eye input is isolated to a separate background process, so apps and websites can't see where you are looking. Only when you tap your fingers do results get communicated, just like a mouse click or tap on other Apple devices." I can't speak to whether extra controllers are a good idea for Apple. From the impressions I have watched and read from VR enthusiasts (e.g, Norm from Tested) they have very high praise for the input and UI. I agree with you, however. I don't think typical VR games will be big on this device

taeric 3y ago

The "material-based renderer" sounds... fluff? Most any system should have it so that rendering a menu is a single line of code. Same for toolbars and other. Those are typically simple registrations of ("name" "description" handler). Anything more that that, and you are doing tailored menu/toolbar and are straying on purpose. That leads, then, to the spatial windows and APIs that go with them. And... quite frankly, I have zero faith that anybody can pull off a V1 for that API. Expect churn and capability growth. For the process isolation on eye-gaze tracking... assuming you can get "hover" events for things you are gazing at, than I fail to see how they can keep you from reading the gaze. I expect they will try and lock down abuses of that, but as soon as you enable coding against what a user is looking at, developers will find a way to leak that. And if they expect that the primary way I will interact with an application is by looking at it.... that feels very very sad, all told. I don't want to just look at things, I want to manipulate them. And for that, I am almost certainly going to want some haptic/tactile interaction. Such that this won't live on its own. Happy to be proven wrong, of course. And, for what it is, this does look impressive. I just don't buy the marketing spin on it, at the moment. Way way way too "dreams work, bro!"

interpol_p 3y ago
The gaze tracking is process isolated. Only when you tap is that location ever communicated to the application process — so they had to build a UI API that supports configurable hover behaviour where that hover behaviour is never readable by developers They've thought through the accessibility features, how VoiceOver works with the headset, dynamic type sizes, etc. Designing this sort of consistent API across a whole platform, where it's not only easy to use, but really delightful, is the bit that's exciting to a lot of people I have collected a bunch of A/VR headsets and developed for them. None of them think deeply about these things, so they just haven't been that interesting. They are more "we supply the hardware, you supply the opinion about how it should be used." I really want the other side of that story The Disney part of the keynote was all "dreams work." It was complete nonsense with no clear direction. Apple's stuff, while it might be wrong, is at least opinionated and clear There is a lot of marketing sheen, but there's a clear design opinion about human computer interaction that comes through. That's the exciting bit that I think people react to, sometimes without realising
- taeric 3y ago
  I didn't actually watch the Disney part. I saw the dad "working" in what seemed like a kitchen? Can't remember. The shallow interaction with the kids, though, gave me cringes. Then there is the juxtaposition of, "this let's you present as if there" with the, "of course you have to fly places to be with people, so when you do, zone out like a champ" Of course, zoning out better not need you to interact much, as hearing on a plane is tough, and pinching your neighbor is not smiled upon. I will mostly have to take your word for it that hover actions are not observable by the application. Will make games... More amusing than usual, with no memory of being looked at for characters. And not showing any extra peripherals is a big part of "dreams work." Physical feedback is huge, and a big part of why controllers are needed. Even just the vibration of the standard PS5 controllers go a long way. For driving games, a haptic wheel is more immersive than the vr.
  - interpol_p 3y ago
    > I will mostly have to take your word for it that hover actions are not observable by the application. Will make games... More amusing than usual, with no memory of being looked at for characters. It's described in the Keynote. This is the exact quote: "For example, where you look is very personal. It can give away something about what you're thinking. In Apple Vision Pro, where you look stays private. Eye input is isolated to a separate background process, so apps and websites can't see where you are looking. Only when you tap your fingers do results get communicated, just like a mouse click or tap on other Apple devices." I can't speak to whether extra controllers are a good idea for Apple. From the impressions I have watched and read from VR enthusiasts (e.g, Norm from Tested) they have very high praise for the input and UI. I agree with you, however. I don't think typical VR games will be big on this device