43 ms·

Notes on Vision Pro

• xivusr 3y ago
  Great notes and I really liked his idea of large persistent info spaces and sharing those with others.
• gnicholas 3y ago
  > Given how ambitious the hardware package is, the software paradigm is surprisingly conservative. I actually see this as a bit less surprising. After all, if you change the hardware in a big way, and you change the software in a big way, users will have a harder time adjusting to the new platform. Instead, they're making a big leap on the hardware side, keeping legacy apps and concepts, and then will presumably iterate to more 'native' experiences that were previously impossible/unimaginable/unintuitive.
  • stephc_int13 3y ago
    Text is still a large part of the interfaces we use. We are all highly trained to read text, it seems basic but it is in practice quite abstract. Text is still best read on a flat surface. The great innovation I can see with this new Apple device is eye tracking, they have not invented it, but they might have perfected it enough to be useable. Eyes could be better than a mouse.
    • roblh 3y ago
      Yeah, if their eye tracking plus foveated rendering works as advertised, it could be a huge step forward. I'm really curious how responsive the gesture controls will be too, it was really cool seeing the finger pinches(?) being used as an input method. I wonder if it's specifically designed just for that thing or if it's all built out to track any arbitrary hand gesture accurately. And I wonder what the language/api for describing hand gestures would even look like.
      • MBCook 3y ago
        I don’t doubt them. The PSVR2 has both and it supposedly works very well. That lacks other features of course and must be tethered to a PlayStation 5. But eye tracking + foveated rendering seems like it’s going to become table stakes in the next few years.
    • Gigachad 3y ago
      Eye tracking is almost certainly more accurate and faster. To the point where we make it an entire game to see who can who can move the mouse to what their eyes are already looking at fastest, in the form of shooter games.
      • gnicholas 3y ago
        Interesting point — if you no longer need to use a joystick to aim your weapon, how will controllers evolve? Will the second joystick be used for some other function, or will it be replaced by a different type of input method? It would be funny if controllers evolved to be more like the single-joystick models that we had decades ago, with the joystick on the left and rows of buttons on the right. History doesn't repeat itself, but perhaps it'll rhyme?
        • Fire-Dragon-DoL 3y ago
          One joystick has to exist since there are many videogames where the character is not in focus (barely), thinking of hack and slash with top down camera (the recent diablo 4),the majority of the focus is on where the attacks are going. In top down bullet hell shooters the player had to aim and shoot in one direction as it walks in another (potentially opposite), so one stick is still needed
          • gnicholas 3y ago
            Agreed — I'm thinking we'd have just one joystick, to control movement. The second joystick could evolve into some new interface, or devolve into a row of buttons.
            • cubefox 3y ago
              PSVR2 already has eye tracking. And unlike the Apple Vision it also has controllers with a stick. Perhaps there are already such games which let you aim with your eyes (assuming you are shooting at something), while you move around with one stick, and the second stick isn't used at all.
  • gutnor 3y ago
    That was repeated a bunch of time during the keynote: "familiar"
  • balaji1 3y ago
    In the demo, I was hoping to see some app/screen/display anchored to certain walls or being fixed in a 3D spot. That way I can walk to a place to see that "screen".
    • endisneigh 3y ago
      Isn’t that an intractable problem without LiDAR? I suppose the naive solution would be to fix the screens in spots as you suggest as opposed to pinning onto real life locations. edit: ah, right the vision pro does have lidar. I look forward to seeing this pinning if it's implemented. I'm still skeptical it would work well - I've used ARKit and it's not really that accurate when you're moving around.
      • fauntle 3y ago
        It should be relatively simple with a QR code or similar physical marker in the real world.
        • endisneigh 3y ago
          With yeah I agree. Without though it seems difficult
          • balaji1 3y ago
            why do you guys say this? Amazon's "View in your room" feature does a decent job of placing a furniture in AR. We can even walk around/towards it. And that tech is just Apple ARKit right?
            • endisneigh 3y ago
              I don’t think it’s that good. “Decent” is accurate
              • balaji1 3y ago
• gnicholas 3y ago
  PSA: this page only scrolls if your mouse is on the left side.
  • acherion 3y ago
    Good thing my browser settings are to show scrollbars all the time!
    • gnicholas 3y ago
      Interestingly 'meta' given that Andy talks about the disappearance of interactivity cues that happened about a decade back...
  • recursive 3y ago
    Firefox reader mode works well here.
    • gnicholas 3y ago
      Brave's reader mode wasn't shown as an option, though my BeeLine Reader extension was able to parse the text when I invoked its Clean Mode. I was a little surprised, since the BeeLine extension didn't detect/color the text inline, like it usually does.
  • bmacho 3y ago
    Windows works the same (for me at least), scrolling follows the cursor (but input does not).
• russellbeattie 3y ago
  > But it does put an enormous amount of pressure on the eye tracking. As far as I can tell so far, the role of precise 2D control has been shifted to the eyes. I've been researching eye tracking for my own project for the past year. I have a Tobii eye tracker which is probably the best eye tracking device for consumers currently (or the only one really). It's much more accurate than trying to repurpose a webcam. The problem with eye tracking in general is what's called the "midas touch" problem. Everything you look at is potentially a target. If you were to simply connect your mouse pointer to your gaze, for example, any sort of hover effect on a web page would be activated simply by glancing at it. [1] Additionally, our eyes are constantly making small movements call saccades [2]. If you track eye movement perfectly, the target will wobble all over the screen like mad. The ways to alleviate this are by expanding the target visually so that the small movements are contained within a "bubble" or by delaying the targeting slightly so the movements can be smoothed out. But this naturally causes inaccuracy and latency. [3] Even then, you can easily get a headache from the effort of trying to fixate your eyes on a small target (trust me). Though Apple is making an effort to predict eye movements to give the user the impression of lower latency and improve accuracy, it's an imperfect solution. Simply put, gazing as an interface will always suffer from latency and unnatural physical effort. Until computers can read our minds, that isn't going to change. Apple decided to incorporate desktop and mobile apps into the device, so it seems this was really their only choice, as they need the equivalent of a pointer or finger to activate on-screen elements. They could do this with hand tracking, but then there's the issue of accuracy as well as clicking, tapping, dragging or swiping - plus the effort of holding your arms up for extended periods. I think it's odd that they decided that voice should not be part of the UI. My preference would be hand tracking a virtual mouse/trackpad (smaller and more familiar movements) plus a simple, "tap" or "swipe" spoken aloud, with the current system for "quiet" operation. But Apple is Apple, and they insist on one way to do things. They have a video of using Safari and it doesn't look particularly efficient/practical to me [4]. But who knows - I haven't tried it yet, maybe Apple's engineers nailed it. I have my doubts. 1. https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-phenomenon-in-ux-design-36827204edd https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-... 2. https://en.m.wikipedia.org/wiki/Saccade https://en.m.wikipedia.org/wiki/Saccade 3. https://help.tobii.com/hc/en-us/articles/210245345-How-to-set-up-Tobii-Ghost https://help.tobii.com/hc/en-us/articles/210245345-How-to-se... 4. https://developer.apple.com/videos/play/wwdc2023/10279/ https://developer.apple.com/videos/play/wwdc2023/10279/
• zmmmmm 3y ago
  Interesting notes. I'm disappointed even though it's entirely predictable that VisionOS is built on the iOS/iPadOS foundation rather than OSX. I guess we'll see how "walled in" it is but it's hard to see any reason Apple isn't going to be just as constraining and controlling about what happens on this OS as they are on iOS, if not more so. Which ultimately means I'll be very reluctant to ever adopt it in any meaningful way as my primary computing device.
  • kfarr 3y ago
    Definitely pros and cons. I remember reading that because of the tight integration with iOS this allowed them to achieve best in class latency for iPad + pencil that couldn't be achieved on any other platform. Having followed Oculus / Quest development that low latency is not optional in this context and every millisecond counts so I can see why they would go this route. On the other hand, the closed ecosystem is definitely cause for concern. Fingers crossed that WebXR support comes out from behind a feature flag to allow for progressive (spatial) web apps.
    • atchoo 3y ago
      > best in class latency for iPad + pencil Hmm. Marketing horse-puckey. A 2018 Dell XPS that has a Wacom digitiser and OneNote gives me lower latency and better inking experience than an iPad and pencil 2. I used them side by side for long-form writing for a while and the iPad was left in the dust.
      • kfarr 3y ago
        You might be right it's more marketing than substance, but FWIW this is the article I was thinking of praising low latency of iPad + stylus: https://danluu.com/term-latency/ https://danluu.com/term-latency/
  • wahnfrieden 3y ago
    Be more specific what you mean by building on "OSX". AppKit is very old and crusty. Why would they build on AppKit? You want that?
• JohnBooty 3y ago
  But it does put an enormous amount of pressure on the eye tracking. As far as I can tell so far, the role of precise 2D control has been shifted to the eyes. I've got one good eye and one bad eye. The bad eye is legally blind, has an off-center iris and is kind of lazy w.r.t. tracking. I'm extremely curious to know how Vision Pro deals with this. One certainly hopes there's some kind of "single eye" mode; certainly seems possible with relatively small effort and the % of the population who'd benefit seems fairly significant. Eye tracking most certainly sounds like the way to go, relative to hand-waving. The Minority Report movie probably set the industry back by a decade or two. Waving your hands around to control stuff seems logical but is quickly exhausting.
  • miketery 3y ago
    Track record wise, apple is one of the best in terms of serving accessibility. So I’d bet greater than 50% odds that they’re thinking about lazy eye or one eye or derivatives there of.
    • ahahahahah 3y ago
      [flagged]
      • deleted 3y ago
        [deleted]
      • avidphantasm 3y ago
        Well, they have pretty consistently invested in and showcased accessibility, famously ROI-be-damned (to paraphrase Tim Cook). So, it’s not a stretch to expect (but not assume) they will have the perspicacity to do the same here.
      • spacephysics 3y ago
        I agree not to drink the apple copium, but they do excel in accessibility by a large degree. Honestly, I think a large part of the industry taking into account accessibility in a more serious ways comes from the competitive advantage apple has created. Doesn’t mean accessibility hasn’t been done in the past. By no means. But it’s a market segment they have on lockdown
• stavros 3y ago
  Does anyone know how these notes are generated? Is it a custom site, or something off the shelf?
  • Karunamon 3y ago
    He says it's a custom site: https://notes.andymatuschak.org/About_these_notes https://notes.andymatuschak.org/About_these_notes That said, you can have something identical to this with Obsidian with a few plug-ins and their hosting service. https://obsidian.md/ https://obsidian.md/ https://github.com/deathau/sliding-panes-obsidian https://github.com/deathau/sliding-panes-obsidian
    • stavros 3y ago
      Thanks, that looks very useful! I wrote a script to export Markdown from Joplin notes (https://gitlab.com/stavros/notes/ https://gitlab.com/stavros/notes/), but the UI in this site is better.
  • andymatuschak 3y ago
    It’s custom.
    • stavros 3y ago
      Ah, thank you, they look great.
    • ttepasse 3y ago
      If you got time and energy in the near future, please, consider slightly enhancing it according to the average web surfers expectations. Particular irritants were the broken scrolling (bound to a container), the broken footnote links and the forbidden content behind links. Additionally the classic escape hatch, Safari Reader, didn’t work.
      • andymatuschak 3y ago
        Sorry. I built it in a weekend several years ago and haven't touched it since. Someday I'll give it some more attention…
• dvt 3y ago
  This is a bit of a rant, but I bought a Quest Pro last year, and I feel it was a total waste of money. It's actually embarrassing how much of an unpolished piece of garbage it feels like. Sure, it's fun as a PCVR headset, but as a productivity tool or as a multimedia consuming device, or as a social tool, it completely misses the mark. And I'd say that 80% of its failure can be attributed to bad software. It's buggy, I constantly have to re-calibrate my work/play area, the hand tracking is janky, and even though it supports (incredible) eye tracking, almost nothing takes actual advantage of the hardware. It's insane that a billion-dollar company like Meta actually felt proud to release such a steaming pile of trash. Do they seriously expect VR enthusiasts to build an entire operating system for them? Much like the reports we're getting about the Vision Pro, the eye tracking in the Quest Pro feels like magic, but nothing uses it—basically it's irrelevant to navigating the operating system and barely any games use foveated rendering. It's infuriating. If I was Zuck, I'd fire all my product managers. I say good for Apple. I'll probably be selling my Quest Pro and buying the Vision Pro.
  • balaji1 3y ago
    Fitness and gaming wasn't demoed much yesterday, but are going to be insane on the Vision Pro - so I guess it was just too obvious to demo. Or is it? Also Meta could catch up if it was just software. But how big of a deal is the M2+R1? Anyway the metaverse was already being retracted, else this Vision Pro could have hammered Meta. Clearly Occulus has to evolve to match VisionPro. Controller has to be optional. It has to have better AR. Meta has to be the more "open" alternative to Apple's ecosystem barriers.
    • dvt 3y ago
      > Also Meta could catch up if it was just software. This was Microsoft's warcry for years: who cares if Apple has better software? We have better hardware. Well guess what, software matters and it matters more than hardware. If you have both, you have literally a generationally-defining product (e.g. the iPhone), but even if you just have the software, you'll still eat your competitor's lunch. Why does everyone have MacBook Pros these days? Prior to Apple Silicon, it certainly wasn't the hardware. Meta's about to get a rude awakening because "we can fix the software later." As if technical debt doesn't exist and as if you can so easily get engineers that eagerly want to clean up someone else's mess.
• BoppreH 3y ago
  I echo the awe at the hardware and the disappointment at the paradigm. I wish the keynote had three extra demos: - Something whimsical to inspire developers. Give me files represented as physical blocks that I can pile, or a task manager that shows processes as plants around me. Close apps by scrunching their screen like paper. A globe with my geotagged pictures. - A game, any game. Beat Saber was the killer app for me to get my Vive, and Valve already spent tens of millions of dollars to create a triple-A VR game. Neither seem compatible with the Vision Pro input methods. Apple could at least play their strengths, like an AR hide-and-seek, or a horror game with eye tracking. - Content creation or professional work. The last thing we need is another passive device to watch 3D TikTok. Show someone customizing their "home screen" environment to look like a fantasy potion shop; or a mechanic looking at a 20x magnified broken part; or an SRE watching their kubernetes cluster as a floating 3D graph; or a VFX artist in the scene scrubbing forward and backward to adjust an effect. It feels like inventing the first smartphone, camera apps and all, but it doesn't make calls and the only input is tapping on icons.
  • TX81Z 3y ago
    Nobody may need TikTok on their face, but that doesn’t mean many wouldn’t want it. TikTok on your face likely has better market potential than the dreadful Horizon Worlds.
  • krackers 3y ago
    > or a task manager that shows processes as plants around me. We finally have the technology to bring https://www.cs.unm.edu/~dlchao/flake/doom/chi/chi.html https://www.cs.unm.edu/~dlchao/flake/doom/chi/chi.html to life
  • mk_stjames 3y ago
    > Give me files represented as physical blocks that I can pile Man, people still want a file manager like fsn on IRIX systems, 30 years later...
    • BoppreH 3y ago
      I used Eagle Mode[1][2] for a number of years, and it was non-ironically the best file manager I've ever used. I only stopped because it required dozens of very out-of-date packages for its preview features. But the point of whimsical demos is inspiration, like a concept car, not complete realism. [1] https://eaglemode.sourceforge.net/ https://eaglemode.sourceforge.net/ [2] https://www.youtube.com/watch?v=G6yPQKt3mBA https://www.youtube.com/watch?v=G6yPQKt3mBA
• david_van_loon 3y ago
  His comments on anchoring virtual tools in physical space reminds me of the work Microsoft was doing with Hololens and Windows Mixed Reality. If I recall correctly, there were APIs for anchoring things in space.
  • cobertos 3y ago
    I recall stumbling upon those too in the Win 10 mixed reality API docs. There were a lot of interesting things in there that never really saw the light of day/any significant use EDIT: Found the API docs. There's some really interesting classes in there! I remember handling coordinates to be confusing/a pain though haha. https://learn.microsoft.com/en-us/uwp/api/windows.perception.spatial.spatialanchorexporter?view=winrt-22621 https://learn.microsoft.com/en-us/uwp/api/windows.perception...
• TheRealPomax 3y ago
  How about we wait until someone actually has one to review?
  • suzzer99 3y ago
    This is pretty good: https://www.youtube.com/watch?v=OFvXuyITwBI https://www.youtube.com/watch?v=OFvXuyITwBI
    • TheRealPomax 3y ago
      While I do enjoy me some MKBHD, I'd like the opinion of someone who's not fully bought into the Apple ecosystem already. This is a good perspective to have, but it's an incomplete perspective. And I'd like to know how realistic this thing is for "not apple users". It's a VR headset, can I use it with Windows to play MS Flight Simulator or Star Citizen? Can I use it on Linux to get wall-to-wall emavs? More real world testing is required.
      • musictubes 3y ago
        It’s not a VR headset, it is a computer with its own OS and M2 processor. I don’t think that VR or AR was mentioned a single time in the Apple event. They say this is the first device for spatial computing and that this is equivalent to the first Mac as far as changing OS UI goes. This will be another part of the Apple system along with Macs, iPhones, iPads, etc. You will never be able to use this as just a monitor equivalent for any other OS.
        • TheRealPomax 3y ago
          Right, so: it's just another VR headset, just beefier. Have you not been paying attention to Meta's Quest and the like? Modern VR headsets are computers, they run their own OS, with applications/games loaded onto the device itself, and they can be used as a VR display client for other operating systems. This is an M2 computer strapped to your head: it can bloody well do VR, let's see what folks do with this device that Apple never asked them to, but they do anyway because Apple doesn't get to tell them how to use their overpriced hardware (especially if the moment you buy it, the buyback is $50 if you're lucky)
• armchairhacker 3y ago
  I think the big usecase of the Vision Pro is just a larger screen and immersion. That's what it looks like Apple highlights in the demos. Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper. Imagine playing a video game but the environment wraps around you. Imagine watching movies in a virtual theater with a simulated 100-inch TV (one of Apple's demos). Even reading and browsing the web can be improved with an ambient environment and extra space for more stuff. Is it worth $3500? For end-users probably not, but if it genuinely makes professionals and hobbyists substantially more productive, it will be worth $3500 or even way more. How much money would you spend to write, code, create faster? Of course, this assumes the VR actually performs, and whether a bigger screen and immersion actually makes people work faster. As of now it seems VR is still a gimmick which impresses people at first, but doesn't provide much outside of niche experiences; which, if this holds for the Vision Pro, makes it very much not worth the price.
  • ttcbj 3y ago
    If you want that immersion, I think $3500 is totally reasonable. I paid more than that for my two additional monitors. The problem for me is that I like monitors better. I like the feeling of real space around me. I cannot conceive of wearing googles on my head all day. I just don’t want to. And I definitely don’t want to be separated from other humans by googles. It just not attractive to me.
  • throwaway4aday 3y ago
    > Imagine developing with the ultimate 10x setup: several files, documentation, and debuggers open at once, with a zen nature background. Or making music, or painting, or even writing a research paper. I used to believe this was the ideal setup, hell I even implemented it with monitors, but since I've integrated LLMs into my development process I find all I need are two side by side windows and now with CopilotX those are both in VSCode. I occasionally venture out for documentation but even that will make its way into the editor at some point. Using Edge and the Bing sidebar I can even save myself the trouble of reading through all of the documentation on a page by just asking Bing to retrieve the info I want from the page I'm on. Doing all that in VR would be cool but I don't think it's going to increase my productivity nearly as much as LLMs have. Disclaimer: I'm bullish on VR and I love gaming and socializing in VR
• deleted 3y ago
  [deleted]
• layer8 3y ago
  Regarding the last part, I was wondering if an AR UI would maybe start a trend back to a more 3D/skeuomorphic design (in the sense of NextStep/Windows 95), which then would also have a comeback on the desktop. That's mostly just wishful thinking on my part, though. ;)
• sebkomianos 3y ago
  Slightly offtopic: What a cool personal website!
• lukevp 3y ago
  From years of observing first-hand how hard it is to develop software, I’m shocked that Apple continually makes products that fit their aesthetic (eg. The crown and headband evoking the Apple Watch) and have coherent and mostly-baked software on launch like this. Is there a book about how they’re able to operate at scale and deliver such consistent things? Do they just have an insanely good product team? How do they document requirements? It’s fascinating to me, because I think it’s their real competitive advantage - they get to the finish line with product that would make sense to a single, invested, super smart person, instead of a pile of inconsistencies and incoherencies that contradict each other as you move through the application. The only other product I feel is delivered at Apple’s level is CloudFlare, and I think that’s why they’ve grown so fast.
  • notnotjake 3y ago
    Creative Selection by Ken Kocienda is the closest, modern look inside Apple's product development that I've found. Focus is on the development of the software for the original iPhone and is a great book.
    • zerealshadowban 3y ago
      Yes! this book is a first-person account of product development at Apple, showing how both the design and the features of a system tool keep progressing under the selective pressure of iterative review sessions, with creative and specific challenges provided by managers at multiple levels, all the way to the top (Steve Jobs at the time of the book's tales). It's a great book, very well written, the only one I've read that provides insight into (some of) Apple's creative process. When they say that Steve's DNA has profoundly shaped the company, I imagine this is one aspect of what they mean.
  • andymatuschak 3y ago
    You might enjoy Ken Kocienda's "Creative Selection", which gives his first-person account of Apple's creative process.
  • anonred 3y ago
    Hypothesis: Apple is a hardware company that also does software, and that ethos allows them to ship more complex products over much longer time scales than their competitors.
• smugma 3y ago
  Reading through his notes, I couldn’t help but think “I don’t know this guy, but it seems like he has spent years thinking deeply about interfaces. Also, he sounds ex-Apple.” It made me smile when I got towards the end and my suspicions were confirmed. Andy’s thoughts were infinitely deeper than anything else I’ve read about the device+interface so far. In particular, I liked his observations that so far it’s mostly a 2D plane, with exceptions like the Breathe app and a few demos, which might just be good for demos. Seeing the heart in 3D reminded a lot of what we saw 10 years ago around what book publishers were trying to do with the iPad. Cool demos but limited real world use. Here was one particularly thoughtful section: “unlike the physical version, a virtual infospace could contend with much more material than could actually fit in my field of view, because the computational medium affords dynamic filtering, searching, and navigation interactions (see Softspace for one attempt). And you could swap between persistent room-scale infospaces for different projects. I suspect that visionOS’s windowing system is not at all up to this task.“
  • andymatuschak 3y ago
    That's very kind; thank you.
  • thewebcount 3y ago
    Reading through it, I was struck by the contrast of how broken the web page was with what the author was saying about UX. The author seems experienced in the field, but the site felt completely broken. I don’t know if it’s a device or browser issue, but reading on my iPad, if I tried to use my right thumb to scroll, the page acted as if I had reached the end of the article, even though it was clear there was more because the last visible line was only half-visible. (And this was after having to manually zoom in because all the text was shoved to one side and tiny.) Scrolling with my left thumb worked about 50% of the time. The rest of the time it worked as poorly as scrolling with my right thumb. I gave up after about 5 paragraphs.
    • nsfmc 3y ago
      that site is not really suitable for mass consumption, iirc it used to even have a caveat that said it really only existed to make certain leaf nodes sharable. it's basically a personal zettelkasten, but this update was also sent to his patreons as well, where the writing just ended up being whatever your email client made of it.
• skydhash 3y ago
  > What if programs live in places, live in physical objects in your space? For instance, you might place all kinds of computational objects in your kitchen: timers above your stove; knife work reference overlays above your cutting board; a representation of your fridge’s contents; a catalog of recipes organized by season; etc. I'd prefer an OS that spans multiple devices. Something how computers behave in “The Expanse” where someone can flick stuff from their mobile device to another system for a better workspace. If I'm typing this on my workstation, but want to switch environment and bring something light like a tablet or just my phone, I could send the tab and it would retain its state. The closest to something like this is a server-client architecture like Logitech Media Server. A paradigm like this would be more useful than immersion. This is also why I like Vim. Buffer is a separate concept from File, adding an extra flexibility – in terms of layout – to how I edit to complete my current task. I want a similar flexibility with applications, at least lightweight – state wise – ones. CloudKit and Handoff are close, but very brittle. Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless. Another good example is handoff between the HomePod and the iPhone.
  • feifan 3y ago
    > Apps like Bear, Reeder, Anybox, Things 3, make switching devices seamless I think Apple's whole point is that they provide the APIs so that apps can do this themselves. Which, yeah, is a bit disappointing; I'd love to have a 100% fidelity, flick-to-another-device type of Continuity built into the OS, but the APIs are at least a starting point.
  • throwuwu 3y ago
    Simple protocols would enable this for existing hardware and software
  • ElFitz 3y ago
    Reminds me of some descriptions of Bell’s Plan 9.
• AlexanderTheGr8 3y ago
  Sorry for hijacking this thread for a question about personal curiosity but does anyone know of an affordable way of measuring pupil distance? I want to measure my pupil distance while browsing social media. The reasoning is my reading in "Thinking Fast and Slow" that pupils dilate when we see something we find interesting / something we like / when we are thinking and vice-versa. I want to put it to the test and it seems like finally the consumer tech is close to making it possible. Does quest support pupil tracking? I did some cursory research but couldn't find any reference for it. Industry pupil tracking headsets are way too expensive; My last hope is that someone will jailbreak vision pro...
  • CamperBob2 3y ago
    Hold up a ruler to your face and look in the mirror. Or take a photo so you can measure distances in an image-editing app.
  • throwuwu 3y ago
    A high res webcam and OpenCV?
• robwwilliams 3y ago
  Vision Pro opens opportunities for a new web interface of the type that Gary Flake and team were building with Silverlight and Pivot at Live Labs. Gary has a great TED talk (Mar 2010) on this radical but intuitive spatial sorting interface. Foveation and hand movements are perfect user links. https://www.youtube.com/watch?v=LT_x9s67yWA https://www.youtube.com/watch?v=LT_x9s67yWA Combined with a Vision Pro—OMG.
• andsoitis 3y ago
  Punch Cards --> Text Terminals --> GUIs/WIMP --> ? ? =! Vision Pro
• bestcitynews 3y ago
  [flagged]
• bestcitynews 3y ago
  [flagged]
• for_i_in_range 3y ago
  [flagged]
• notnotjake 3y ago
  I was hoping to see more paradigm shifting user interface ideas. I think those exist inside Apple Park, but are waiting for their time. It's telling that this was announced at the developers conference. The focus now is on the platform, and I think mind-blowing user experiences will come later. (At least I hope)
• brandall10 3y ago
  Anyone surprised they didn't incorporate an EEG sensor and use that in conjunction with eye tracking? They could acquire someone like Neurosky. I thought at this stage, it would be fairly foolproof to use that at the very least to simulate something like a click, no?
  • tsunamifury 3y ago
    They want to ease us into the dystopian nightmare…
• cobertos 3y ago
  I find it interesting that Microsoft had _so much of this_ 9 years go. I developed with the HoloLens 1 around 2016 and I recall: * Fingers together gesture clicking. * Voice activated menu navigation. It was glitchy/I never used it though. In 2023 these sorts of systems are much better though. * No controllers. It was all gesture based. Opening the start menu required your hand upturned, fingers together then outstretch. Kind of like an "open" gesture. * "Pointing" based on head looking vector, which was annoying. * Spatial anchors and being able to remember past spaces and how you used them. There was a whole set of SDK APIs for the spatial stuff built into Win 10 The Vision Pro iterates on some of these. Eye tracking for pointing and more cameras for tracking hand pose for clicking, to reduce the annoyance/strain. IMO all mandatory for long term usage (I frequently got motion sick with the HoloLens and the 3 months of debugging/developing on it were a challenge). Maybe if Microsoft had switched to more commonplace display technology and continued to iterate on their product instead of let it languish, they could have had a solid competitor to this, if not maybe be first to the market.
  • askvictor 3y ago
    This is classic Apple. They're never the first mover; they let others fail, learn from those mistakes, and make a pretty polished product that everyone thinks they created the genre.
    • rchaud 3y ago
      Classic is right. That period ended with the iPad reveal almost 15 years ago.
      • originalvichy 3y ago
        Are we gonna pretend the Apple Watch didn't exist? AirPods, if we want to stretch? Pebbles and other smart watches filled the criteria, but never delivered a cohesive package. I do admit that the first 3 or so generations of Apple Watches are pretty rubbish, but 4th onward with the possibility of all-day battery life, the watch became a clear winner compared to all others by manufacturers. Garmins and other fitness-first watches hold the crown when it comes to the more slimmed down and focused feature set, but for the generic smartwatch the Apple Watch is untouchable.
• mcintyre1994 3y ago
  Related to his comment about the windows, I’m really curious how window management works in this. They’ve shown you can have multiple windows kinda in the space at the same time. But how do you rearrange them/move them around in space? How do you close them? Is there some way to put 2 apps next to each other in the same window, and keep them together when you move stuff around? I’m mostly curious because interacting with this stuff is still a mess on iPad, and the hand gestures they demoed for Vision Pro are a lot more limited than what we’ve seen with multitouch.
  • pocketarc 3y ago
    The way it was presented seems to indicate that you can just set them wherever - if they don't move with you, then they'd have a physical position, right? And for me, that sounds super exciting. Imagine going into a room and you have apps on the wall. You go into your home office, you have apps ready to go (picture a big status dashboard covering a wall, your music app to the left of your desk, stuff like that). If it actually does work that way, that really is a whole new world.
    • mcintyre1994 3y ago
      That's definitely really cool, but there must be a way to detach from that paradigm too for things like flights and hotel rooms, where you want the work/whatever to be wherever you are. And you'd still want to be able to rearrange the things in your home office room.
• sensanaty 3y ago
  [flagged]
• nullandvoid 3y ago
  The primary input is eye tracking to me is really exciting, feels like the future. I wonder what happens in games though. Will this not essentially be aimbot, looking at the target and then shoot?
• jillesvangurp 3y ago
  I think Apple is doing smart things here with their software: - leverage existing content means they won't have an empty room problem or a big "now what?!" moment for users after the novelty wears off. Without Steam, most existing VR platforms would be completely pointless. So far VR is for games and most of those are published on Steam. That's because most VR hardware vendors suck at software and end up outsourcing that to game studios. Meta included. Despite their ambitions, their goggles are mainly devices for running games not developed by Meta. - a focus on the living room experience with a high end movie theater experience running, again, off existing content. Genius move because people already spend lots of money on home theaters. Some people buy 10K$ plasma screens even. This market is very real. Also, involving Disney with their huge back catalog of completely unmonetized 3D movies ... so obvious. - Extend the highly successful IOS ecosystem to the new experience. They have tens of millions of apps already. And they'll work fine in AR. Why not do that? The strategy is about content. It's the right strategy. The first generation hardware is of course amazing too and it will bootstrap a new generation of application developers that will be using Apple SDKs and tools to target all this with new content. But to bootstrap the ecosystem they need an audience. Here too the strategy is genius: SDKs are based on things developers already use. They just co-announced a push into gaming for Mac along with some convenient porting kits for developers. That was just a footnote buried deep into the mac related announcements. But of course this means more content coming to AR as well. All that lovely content currently available via Steam. Top to bottom the strategy is about compelling content. I think it's going to work. This is not the final answer to AR but the opening salvo in a decades long push to completely own this space. Step 0 is to get millions of these things in the market with enough content to get people hooked and keep them consuming content. The rest will come later. It's appropriately radical, pragmatic, and conservative at the same time.
  • kristianc 3y ago
    If the strategy is fully about content (which would make this device about selling services on top of the Vision Pro), why is the Vision Pro specced as it is, and why is the price point so high? The specification of the device seems to be complete overkill for the kind of strategy you are describing, which frankly does not need extreme hardware power, and makes it cost prohibitive to many of the people who would otherwise buy the content. All I can think of is that there are lower specced and lower priced devices coming, but I am sort of at a loss as to why they didn’t start with those - as if this fails, they aren’t going to get off the ground.
• jb1991 3y ago
  I use quite a few Apple products every day of the last 10 years, and I’m very impressed with certainly all the technology and reviews and explanations of how this device works and what the user experience will be. However, this is the first Apple device that really makes me pause. All of the marketing material, the WWDC videos, all of it feels very uncomfortable to see all these people isolating themselves in a room with big goggles on. I find it really hard to comprehend this is the direction that technology is taking our lives. Imagine walking into a house, and a lot of the people are just sitting around by themselves in corners with goggles on. The whole thing just feels very strange and post apocalyptic to me. And these 3D spacial moment recordings, imagine children growing up in a house where when something nice happens, the parent rushes to put on goggles and stare at them through them, their little virtual eyes displayed on the outside, it’s frankly creepy to me.
  • theonlybutlet 3y ago
    Honestly you could replace everything you said there with an iPhone and it'd still apply.
  • gofreddygo 3y ago
    There are 3 things that standout about apple's vision product line. First there's technology. This is clearly better than anything else in the market. Enough has been said on this. Then there's usability and market. There's no real trojan horse feature that will get this product mainstream right now. Apple's hope is that the dev community will either build one or users will point them in the right direction with usage stats and telemetry. Maybe they will, or they won't nobody knows. Finally there's the future. I see people comparing this to previous apple products and their past experiences with them etc. There appears to be two factions here, one that says "This is the future I want and I love it", and the other saying "This is the future I don't want and hate it". Both groups however are saying one common thing ... "this is the future". And that's pretty much Apple's bet.
  • dwayne_dibley 3y ago
    I don't disagree, however when was the last time you walked into a room where at least one person wasn't solely focused on their phone. The 3D photo thing is super weird, they could release a stand alone camera in future, BUT, I think this is a v1 product, imagine where it'll be in 5 years time, or an SE model which isn't wrap around. I already wear glasses so having AR glasses is no big leap.
• pflenker 3y ago
  The vision pro showcases one use case which is compelling enough to make me consider buying it (in a couple of years, when it’s more settled): As a replacement for a monitor for work. Seriously, having a huge monitor everywhere is a game changer for me.
  • smelendez 3y ago
    I agree this is the compelling use case. The benefit of a big monitor without having to devote space to it, and on the go. Also, a private monitor where nobody can look over your shoulder, which is a game changer for doing any kind of sensitive work or personal business in a public or shared space.
• gampleman 3y ago
  It's a strange time to be releasing this product. Not from a technological perspective (clearly the hardware Apple managed to put together is extremely impressive and they seem to believe that it finally got good enough to have real appeal albeit at a rather hefty price), but from a political perspective. It seems like the last few years have generally had a technological backlash with worries about both invasive spying and surveillance capitalism on the one hand and mental health issues (especially in teens) sprouting on the other. So it seems strange to react to all that by offering a product that glues a computer straight on your eyeballs. Now of course Meta has already been going down that road, but let's just say that company doesn't have much to loose in the reputation department. To be fair, Apple seems to have made a few interesting design choices. They seem to have gone out of their way to create an illusion of two way transparency to their headset making casual human communication at least theoretically possible (although how much conversation you can have with someone actually wearing these due to distraction is an open question - if it's like trying to talk to someone on their phone that it will be at best half their attention).
• ftxbro 3y ago
  > "Something in the Apple omertà makes me uncomfortable naming my collaborators as I normally would, even as I discuss the project itself." I don't understand this. It's things they made them sign? I saw another article saying they make them sign a promise to not show they are wearing the headset, allegedly because they might look like huge dorks and cause brand risk.
• elif 3y ago
  I'm about 90% sold but I'd love to have someone's experience using it as an emacs terminal for coding. As long as it doesn't get sweaty or nauseating after 30 minutes, I would still be content with it as a glorified monitor, not needing new UI paradigms or gesture controls. Distraction-free huge monitor computing anywhere in the world is a huge advantage over laptops imo.
• tromp 3y ago
  > the amazing power of the iPhone map is that I can fly to Tokyo with no plans and have a great time, no stress. Figuring out how you're going to get mobile data access in a foreign country without being charged through the nose does feel somewhat stressful...
  • sebzim4500 3y ago
    Even the most scammy carrier isn't going to charge you an amount that is significant compared to the cost of the holiday. If you want to plan ahead to figure out how to get it cheap than go ahead, but it certainly isn't worth ruining your holiday over once you are there.
  • nickpp 3y ago
    Airalo app allows me to buy an eSIM for Japan in the comfort of my home. Starting at $5 for 1GB and 1 week.
• ganesh7 3y ago
  Every day another submission about these goggles nobody seems to like. I'm just annoyed by all that unwarranted attention. Just because it's Apple.
  • lewisflude 3y ago
    Anecdotally I like them, and not just because it's Apple, seems like great hardware/software/industrial design etc. But valid to be annoyed at it, Apple do tend to be quite polarising for people.
• sw104 3y ago
  Has anything been said about how the eye-display proximity might affect vision? Maybe this is a better time than ever to start a new opticians.
• dboreham 3y ago
  VR has been "just about to become mainstream" my entire career. I'm quite old now. Perhaps it'll break through before I retire. That said, the same thing was true about video phones, to the point that everyone assumed humans do not need to see each other on a phone call. Then we had a pandemic and things changed.
• whywhywhywhy 3y ago
  Hit me this morning they never showed anyone working with files. Can’t believe after the failed attempts to kill files on iPhone and iPad they’ve learned nothing and still think we can pretend a professional work tool can exist without a proper file system. Not saying it should have desktop icons strewn across your living room, just that the “files siloed in apps transferring via share buttons” iPad model is objectively productivity hell and puts mental load menu fumbling where MacOS is instinctive drag and drop.
  • hoytschermerhrn 3y ago
    You’ve posted a 3-paragraph rant on a baseless assumption for a product which hasn’t been released yet. Let’s all withhold judgement until we have more information about how this truly works.
  • MikaEfrat 3y ago
    One of the WWDC developer videos shows the Files app icon around 10 minutes in: https://developer.apple.com/videos/play/wwdc2023/10203/ https://developer.apple.com/videos/play/wwdc2023/10203/
    • whywhywhywhy 3y ago
      Files app is a crutch and a band aid to solve the problem of not building a file centric interface to begin with. If it's anything like iOS/iPadOS it just doesn't work for getting work done or keeping projects organized, buildable upon and sane.
• wouldbecouldbe 3y ago
  Yeah projecting they eyes is such a stretch, seems to suggest they think we really will hangout with eachother while having the goggles on. Maybe they are right in the end, but weird. Same with the cameras displaying the world instead of using the eyes. Interesting choice, but also a big bet.
• uxp100 3y ago
  Interesting to read that the iOS 7 depth/parallax effect was tried all over the UI. It makes sense to indicate interactivity in a better way, but probably the right call to ditch that.
• amelius 3y ago
  > The hardware seems faintly unbelievable—a computer as powerful as Apple’s current mid-tier laptops (M2) So I take it that you can basically leave your laptop home and take your Vision visor with you and work from a virtual environment?
• dav_Oz 3y ago
  I dig his vision of huge, persistent infospaces after all the "method of loci" [0] is a very effective and well described mnemonic device at least since antiquity. The basic observation: It takes very little effort and exposure to remember places (spatial information) in contrast to taking in linear heavy processed information from writing and numbers (e.g. "facts"). Combine these two and you have a powerful way navigating through the information space itself. If you look at it, we are confined to rectangular boxes, fancy interactive books enhanced by moving pictures, audio and some very limited tactile feedback but nevertheless bound by the superstructure of a static tiny detail in our overall spatial awareness. Of course we are now so used to the endless iterations of rectangular boxes that we are extraordinarily proficient at extracting a lot of richness/information from that: in scrolling, clicking, typing, imagining ... I'm reminded by the scene of "The Matrix" where Cipher let Neo glimpse the Matrix code on his screens: "You get used to it, I don't even see the code. All I see is blond, brunette, redhead."[1] IIRC once a 50s western movie was shown to a rural "test" public from the Eurasian steppe some of who saw a moving picture for the very first time. The reactions were mostly enthusiastic during slow "panorama shots" but the audience reacted very agitated in "close-ups" or "medium shots" where e.g. in the heat of the action the horse's legs were "cut off". We take it as given that we learn the language of movies and find them at times 'realistic' and 'immersive' when in fact contrasted to 'reality' itself qbit by qbit they are basically - with some rounding error - as highly processed as books and thereby rely heavily on our imagination filling the gaps. While we are on the subject of of naïve visions, another area which isn't appreciated enough imho is the immense dexterity of our hands as with "vision processing" the devices themselves are glued on our dexterity of "exact" and "repetitive" moves, there is too little wiggle room for organic experimenting, refining and expression like one would naturally do for a musical instrument. [0]https://en.m.wikipedia.org/wiki/Method_of_loci https://en.m.wikipedia.org/wiki/Method_of_loci [1]https://m.youtube.com/watch?v=MvEXkd3O2ow https://m.youtube.com/watch?v=MvEXkd3O2ow
  • generalizations 3y ago
    > I dig his vision of huge, persistent infospaces This is immediately where I went as well. Been hoping for something like this since the first few iterations of VR tech. I can imagine that will become one of the 'killer apps' of this platform. Unfortunately, he also said it might take some iterations on their windowing system.
• 8jef 3y ago
  To me, Apple Vision vision's, so-called spacial computing, is exactly what I was always looking for in a portable computing device: beefy processor, pixeless experience, app and file navigation using natural eyes and hands movement. This has the potential of replacing all desktop, laptop and tablet I may own. Now I just have to wait for a Linux compatible version. Because there's no way I'm gonna use any Apple software as if it was twenty five years ago, before iTunes insanities and whatever happened since. NFW.
  • generalizations 3y ago
    > Now I just have to wait for a Linux compatible version. That'll probably be about 15-20 years. Apple will have to prove that their interfaces work, and the rest of the industry will have to follow, and hardware will have to be hacked, and new paradigms built, and eventually -- maybe -- it'll be possible to do a non-janky version of this on linux. But the UX is paramount here, and linux is notoriously bad at that.
    • zaptrem 3y ago
      I think VisionOS apps can spawn multiple windows, so you could just spin up a fancy Remote Desktop app that serves Wayland windows and snaps the mouse to whichever window you're looking at.
• coryfklein 3y ago
  Side note unrelated to Apple Vision: Why would anyone want text fully aligned to the left of the browser window?! At least it's distinctive, I don't think I've ever seen that on any other website in the last 15 years.
• fio_ini 3y ago
  I cant stop thinking about how cool it will be to finally remove all these monitors and wires from my desk. Operating inside of a virtual space has an amazing appeal to me to de clutter desk and office space area. Scale screens bigger and smaller and not be confined to: tight windows, bringing in more physical screens, setting them all up, switching between work spaces with hotkeys. Even the virtual workspace on a single physical screen uses extra mental RAM capacity to always be remembering which space the tabs are and where. This evolution frees up my neck, my posture, minus the bit of weight on my head, I can be sitting or standing. This next development from this company I'm connected to seems to go to such a more natural way of thinking about and interacting with our computing space. Besides all the great human side of it's value added, this headset is tangible for me because it's connected to an ecosystem that I'm part of. If I were to get a Quest for example, now what? I'm connected to Facebook? Great... The price for Apple Vision is pretty high but damn it's a pretty amazing piece of technology, and blows most other major headsets out of the water, there's over 20 million pixels for each eye for God's sake. The amount of virtual real estate we're talking about is like walls of 4k screens all wired together throughout your house a hundred different HDMI/DVI/DisplayPort Adapters to GPUs, what a freakin nightmare that would be. Easily all costing well over $3.5k.
• nerdbert 3y ago
  I'd like to connect this thing to a drone and go flying.
  • xenolith234 3y ago
    I too was wondering if it could act as DJI's FPV helmet.
• cudgy 3y ago
  “… you won’t be confined to “a tiny black rectangle”. You’ll use all the apps you already use. You don’t have to wait for developers to adapt them. This is not a someday-maybe tech demo of a future paradigm; it’s (mostly) today’s paradigm, transliterated to new display and input technology.“ This is why I think this device will be different than its predecessors. There are practical reasons to buy it … assuming you can use it without getting sick or strained.