12 ms·

Apple unveils new accessibility features

726 points by interpol_p 5mo ago

runeks 5mo ago
> The total amount due on the bill is $83.89. Please verify this amount with your utility provider or by using Text Detection before making a payment. 1. Use AI to determine how much a bill is for 2. Call up the people who billed you and ask them how much they billed you 3. Pay billed amount
- deleted 5mo ago
  [deleted]
- tramc 5mo ago
  It’s still useful to get the information instantly and verify it later. Arguably asking someone you trust to read the number for you might be a better idea than calling the company. Not everybody has that option though.
  - Someone 5mo ago
    And not everybody wants to use that option all the time. Asking a human makes you feel dependent more than using a tool does.
    - kotaKat 5mo ago
      Aaaand the logistics of making that call to the company to confirm the amount on the bill can get awkward. IVR and hold-time hell just to get a human to have to explain your predicament as to why you're asking for such a mundane piece of information that was in fifty other touchpoints that you couldn't access as quickly or easily. (I'm also picturing the poor CSR at the other end of the phone wading through hundreds and hundreds of call logs over the years for simple requests and managers up above screaming 'why is this guy calling us all the damn time costing us money'...)
- stellamariesays 5mo ago
  [flagged]
- dewey 5mo ago
  Once you paid the same bill for a few months you'll know how much your phone bill will roughly be and you'll not have to do. They obviously have to put that line in there, just like ChatGPT saying "Please verify everything we tell you" in the footer.
- martinflack 5mo ago
  It might be useful if it remembered a bill for, say, 60 days, and could also comment on percent difference since the last one. "The total amount due on the bill is $83.89 which is 4% higher than last month's bill from the same company."
- kube-system 5mo ago
  I presume that calling customer support is at least as frustrating for people with disabilities as it is for anyone.
nechuchelo 5mo ago
This looks like a genuinely useful application of LLMs. I wish more companies focused on how they can help humans instead of replacing us or squeezing us as hard as possible in the name of productivity.
- koolala 5mo ago
  Its with their servers right? Do they trust a iPhone with their life? Or they are trusting their data center?
  - nechuchelo 5mo ago
    Looks like some of the features might use on-device models. They mention subtitle generation works on-device.
- bsanders343 5mo ago
  I agree. There seems to be a lot of potential in this space (from my outsider view). I really hope that this issue from an earlier article (https://news.ycombinator.com/item?id=48178378 https://news.ycombinator.com/item?id=48178378) doesn't become common enough to make useful functionality like this a danger. Seems unlikely in the short term but as use cases grow, so might the bad actors.
- c0wb0yc0d3r 5mo ago
  I think we should reserve judgment until this lands in the hands of the people it helps. My experience is limited to my elderly parents who have trouble seeing. With the text size Apple allows them to set it to, their phones are unreadable. Text runs off the screen in every app, 1st and 3rd party. In their bill example, the user is told to confirm with the provider. Why not offer to call the number on the bill? Instead of telling them to use text detection, do it for them? Presumably Apple Intelligence would already have that capability. I’m afraid this will be a gimmick at best. EDIT: Forgot to mention, the grip is good to see. Hopefully they don’t charge the apple tax on it.
  - kps 5mo ago
mohsen1 5mo ago
Fun fact: This video was made accessible to sighted people because no blind person would ever listen to voice at that speed. Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds.
- Sweepi 5mo ago
  dont you worry, as a sighted person I am also infuriated by apples slooow reading speed, e.g. for "Announce Notifications".
  - hightrix 5mo ago
    Also as a sighted person, this is why I hate the modern trend of using the video format to show 3-4 bullet points. Just give me the text.
- embedding-shape 5mo ago
  > Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds. Even better, fire up Orca (or whatever screenreader application your OS comes with) yourself and try to use your computer while shutting your eyes, kind of eye-opening (no pun intended) what kind of experience these sort of users typically get. And also, you quickly start to understand why they set the speech rate for their voice synthesizer to be so fast, it's almost unbearable navigating applications (and particularly lists) otherwise.
  - seviu 5mo ago
    That time my Mac display broke and I had to log in taught me much about how important learning accessibility is even for non blind people.
  - jchw 5mo ago
    When I was at Google, I'd periodically test our (internal-only) app with Chromevox with the display off. It's not that it sounded like it would be easy, but it really is a challenge, and I can only imagine the muscle memory built up over time of trying to work around accessibility bugs and strange behaviors. Unfortunately it seems impossible to get all that much funding for accessibility work :/ I wonder what ever happened to the Newton accessibility bus intended to supplement Wayland...
jansan 5mo ago
Since Apple uses Gemini to power its AI, are those features actually powered by Google Gemini?
- k4rnaj1k 5mo ago
  [dead]
- jjice 5mo ago
  They don't get, but they will be using Gemini derived models with iOS 27. For now it's all their own models.
exitb 5mo ago
As Apple shifts towards services and fancy software features, I wonder how do they expect to stay competitive by only releasing them for a subset of languages.
- layer8 5mo ago
  They roughly know how many of their users use a particular language.
tekacs 5mo ago
I'm super glad that they're doing this, but once again unexcited for another decade of Apple self-privileging on this stuff so they're the only ones allowed to touch or improve any of this surface, or UX outside an app's tiny box. People talk a lot about how MacOS has gone downhill but I feel like it would have been a good start if developers could continue to patch over Apple's shortcomings like they used to be able to. I imagine that we would be a few years into a spectrum of tools like this if they didn't lock it down like they do. Totally aware that plenty of HN commenters are very glad that Apple keeps this locked down. I'm just the other opinion, that's all.
abhikul0 5mo ago
On-device video subtitles generation is exciting, should help with watching videos on mute. This seems like a low hanging fruit that should've already been grabbed by an app but I can't find any.
randusername 5mo ago
Accessibility features are such a great way to keep technology focused on real-world problems and real-world experiences. I think the trap in creating anything is doing it for a crowd. Art, software, anything... it turns out better when it is made with a specific, named individual in-mind. Accessibility features are almost always championed and field-tested with one specific loved one in mind and I think that's what keeps the technical solutions personable and grounded.
Almondsetat 5mo ago
I have difficulty trusting this. There are plenty of videos online of LLMs making up stuff like "I just ate a hot dog, is there mustard around my mouth?" "No, everything is clean" while there is a big yellow stain om the guy's face
- postalrat 5mo ago
  Like coding, creating images or text, maybe the alternative of doing it yourself is too easy or enjoyable for you. Don't expect that will be true for everyone.
  - Almondsetat 5mo ago
    Did you reply to the wrong person? What are you even trying to say here?
    - postalrat 5mo ago
      You say you don't trust it but whats your alternative assuming you lost your vision?
- WarmWash 5mo ago
  The problem is using a language model to assess images. Probably 80% of "LLM's are below expectation" complaints (from the general population) involves some form of image analyses. Image tokenization is hard because unlike language tokenization, where every token is extremely dense with meaning, image tokens tends to be meaningless or irrelevant but are processed all the same. Give an SOTA LLM a picture of toothpicks and ask it to move one to make a square, and it will probably struggle and fumble it. But give a mid-size LLM from 2 years ago the same problem in verbal form, and it will nail it almost every time. That takeaway is, do everything you can to avoid having the LLM need to rely on images for the answer.
  - gruez 5mo ago
    I thought all the recent models are "multimodal"? Is the image part just sticking an image recognizer in front of the text model?
Darwins_Toffees 5mo ago
"Vehicle Motion Cues come to visionOS, which can help reduce motion sickness for people who use Apple Vision Pro as a passenger in a moving vehicle. Vision Pro will also support face gestures for performing taps and system actions, plus a new way to select elements with one’s eyes while using Dwell Control." Maybe just don't wear them in a car?
- dmix 5mo ago
  Wearing a headset in the back of an Uber doesn't sound that crazy, I use those motion cues on my iPhone even though I don't struggle with motion sickness https://www.youtube.com/shorts/OxbjggMcKrk https://www.youtube.com/shorts/OxbjggMcKrk
  - nozzlegear 5mo ago
    I use them as well. I'm usually the driver so I don't typically look at my phone while the car is moving, but I recently rode along with a family member to an event. They handed me their iPhone to look at something and I felt totally disoriented trying to look at a moving screen in a moving car. I had to resist the urge to turn on the motion cues.
- brookst 5mo ago
  Trains are a thing.
- yreg 5mo ago
  >Maybe just don't wear them in a car? Why not?
  - throwaway132448 5mo ago
    Because the more we reject our shared reality and substitute it with each our own, the less humane we become.
    - jkman 5mo ago
baxuz 5mo ago
Now we know why the new AirPods will have cameras!
testfrequency 5mo ago
I don’t want to discredit more advancements in accessibility, but this feels like accessibility porn. I have fond memories of an old coworker 10 years ago who is blind. He would use his phone no problem, texting, going about his day, he was even on Tinder (credit to Tinder for making their app so accessible long ago). He would commute on his own, walk to the train station, even transfer to another train during peak rush hour. I’m not saying it was all easy for him, but nothing in this video really stood out to me more than what shirt was on the bed. I know other services/apps have long existed to be the “eyes” for people who need support, but this video feels….uneventful? I may be cynical about this though, as I often hate how Apple’s marketing makes these emotional bids about how life-critical they are to society - which is fair to a degree..but it just feels cheap to be glamorising “look! we saved this person from pending doom, cool right??”
- lwkl 5mo ago
  I mean even if it is marketing for them they still did the work and developed these features. I had some vision issues recently and was glad there were options to make text more legible to me. Additionally I don't believe this is just marketing. This is adaption to a changing market. Apple's customer base is aging and having these kinds of features will allow them to keep using Apple products for a longer.
  - testfrequency 5mo ago
    They have done the work, but I don’t see much work that’s beyond what’s been previously capable without Apple Intelligence. The marketing of Apple Intelligence is weak here, not the foundational abilities.
- RobMurray 5mo ago
  for every person like your coworker, there are probably several who have a much harder time with technology and who would benefit from a simpler interface. If this includes improvements to the screen recognition feature in Voice Over, it could provide accessibility for apps where the developer doesn't care about accessibility, which is extremely common. The vision capabilities could be useful if they are done well, but I suspect that will always be covered better by 3rd party apps.
devinprater 5mo ago
There's my dopamine hit for the year.
mistersquid 5mo ago
> A new power wheelchair control feature leverages the precision eye-tracking system on Apple Vision Pro to offer a responsive input method for compatible alternative drive systems. [0] The above caption for Apple Vision Pro is for a video that to me, as an Apple Vision Pro user, is discomforting. More questions are raised than are answered by the short video: Is the user able to fit the Apple Vision Pro by him/herself? What happens when dwelling on a directional control misregisters? Can the user recalibrate the "Eyes and Hands" setting? Dwelling on a control displaces focus and there may be impeding objects in the path of the power wheelchair. Is this really a good idea? To my sensibility, the video is unsettling (at best), especially given how cumbersome Apple Vision Pro is. [0] https://www.apple.com/newsroom/2026/05/apple-unveils-new-accessibility-features-and-updates-with-apple-intelligence/#:~:text=A%20new%20power%20wheelchair%20control%20feature%20leverages%20the%20precision%20eye%2Dtracking%20system%20on%20Apple%20Vision%20Pro%20to%20offer%20a%20responsive%20input%20method%20for%20compatible%20alternative%20drive%20systems. https://www.apple.com/newsroom/2026/05/apple-unveils-new-acc...
- jkman 5mo ago
  Your concerns are completely nonsensical. It's clearly being marketed as a healthcare tool for people with debilitating injuries that preclude the use of hand-powered wheelchair controls, severe situations where there's no neck-down control and users would be limited to controls like head-tilt or mouth actuated systems. These people obviously require daily care to simply get them out of bed and into the chair and back again every single day - their nurse could just put on their Vision Pro for them! This seems like an incredible leap forward for people in this situation, if they iterate on this and it gets better then this could be a very viable wheelchair control system in the future.
  - mistersquid 5mo ago
    > Your concerns are completely nonsensical. With all due respect, my concerns are not nonsensical but borne of my daily use with Apple Vision Pro and my awareness of the limitations of dwell control. Iterating on this idea with a device lighter than Apple Vision Pro and improvements to dwell control would likely be required before this could ship to larger populations of disabled users, but that is not what is depicted in the video. My sense is that the possibility of an accessibility affordance with people who are severely disabled is driving opinions in this case more than the reality of what’s available. To my mind, much of these AX announcements are reminiscent of the circumstance that led John Gruber to author “Something Is Rotten in the State of Cupertino”, which is that these are not shipping features but ones slated for “some time later this year”. I’m a huge AX fan and work directly in the domain space, but something about that video in particular coupled with my near-daily use of Apple Vision Pro doesn’t feel right.
seeeeebt 5mo ago
Surely a blind person relies a lot on audio input?
- isityettime 5mo ago
  Maybe on a smartphone, but usually not on a computer. Keyboards are pretty good. The other thing is that if you're around others, voice input means you have no privacy. Even if you're not doing anything particularly private, it's a bit awkward and potentially embarrassing. If you use touch input in conjunction with a screen reader, you can be more like a "normal" user in that what you're doing is just between you and your phone.
- asadotzler 5mo ago
  Audio input is far more commonly used by people with mobility difficulties. Imagine that your hands shake a lot or that you don't have limbs. That makes using a keyboard and mouse difficult or impossible and voice input can help. Blind users generally use keyboards for input, the typical ones you find on a PC but also sometimes the keys on their Braille display.
zersiax 5mo ago
Honestly as a blind person and blind developer myself, most of these features get a shrug at best. For one, there's already a bunch of third-party apps that do most if not all of this (Seeing AI, Envision AI, BeMyEyes, Aira, etc.). So at best, this does what all those apps are doing but faster and on-device, which may or may not mean it is also more inaccurate, we'll have to see. In the meantime, Mac OS's screen reader, VoiceOver, has been left to essentially exist in maintenance mode for years, where users have had to build, arguably impressive, third-party solutions to add features to the thing that comparable screen readers on Windows have had for a really long time. Through that lens, this all looks a bit performative to me, but again, maybe I'll be pleasantly surprised. The one thing I'm mildly excited to see is the improvement to Voice Control, as guessing what the programmatic name of a button is or having to constantly use a numbers grid to target elements doesn't sound fun. To respond to what I see in some of the comments: - On speech rate: It does take quite a bit of practice to crank up the speech rate and there's a degree of retraining you need to do when you switch voices. A lot of more "human" sounding voices are harder to follow at super high speeds which is why a lot of people prefer more robotic but consistent speech and generally aren't convinced by AI-powered TTS yet; they often fall apart if you raise the speech rate past a certain point. - Re: actually waiting for the target audience's verdict: This is so important. I see more and more companies, individuals etc. talk about accessibility, build accessibility solutions and evangelize AI for accessibility without EVER talking to the people they claim to help. This will almost certainly mean mistakes will be made, up to and including doing more harm than good. If you want to do accessibility right, that includes AI products of any kind, hire people with lived experience or you'll get the equivalent of machine-translated text, hackerproof security in one click or an AI-powered coffee bar that orders thousands of rubber gloves. Coincidental note: I have time for new projects right now :P
- Tox46 5mo ago
  it's so validating getting the same conclusion we got to from someone that i've never met. it seems that they create these products without ever speaking with someone with that problem. Funnily enough we're creating a competitor of these third party app that you mention, with the huge experience of my colleague that is son of blind parents. We have an mvp online but it's not much yet and i really don't want to be the "do you know i have an app?" guy.
nikhilpareek13 5mo ago
[flagged]
deleted 5mo ago
[deleted]
deleted 5mo ago
[deleted]
postalcoder 5mo ago
One thing Apple really needs to get right is speech to text transcription. They've nailed accessibility in so many ways and yet it feels like they're a decade behind on properly transcribing voices. At least half a decade. Input on the iPhone is so dreadful nowadays. Their palm rejection is definitely worse than before, so mistyping is more frequent. Their text-correction algorithm for typing is worse than before, and it frequently makes incorrect corrections to words that I don't notice, because they change words a few words back from where I typed. And STT hasn't improved. On top of that, my fingers are tired of the phone form factor. Please make the iphone not a chore to use, apple.
- terabytest 5mo ago
  Wispr Flow is a masterclass in STT. Apple's solution feels like it's from the last century in comparison. Same applies with Apple's TTS when you have ElevenLabs and OpenAI running laps around it. All I need is for my iPhone to do those things natively at the same quality level (because in Apple's walled garden that's the only way to get them usable everywhere).
  - adamcharnock 5mo ago
    FWIW - I also really like Wispr Flow, but I moved to running the 'Whisper Large' model locally using Handy (https://github.com/cjpais/Handy https://github.com/cjpais/Handy), which has been essentially as good, while also having lower latency.
    - dceddia 5mo ago
      Handy is great. It exposes a bunch of open models beyond Whipser too, and though I haven’t tried too many of them, I’ll throw in a rec for the Parakeet model which feels pretty much on par with Whisper for accuracy and is way way faster.
  - jjice 5mo ago
    But Apple's uses so few system resources and runs fully on device on newer iPhone models (16+ I believe). It's so efficient. I really enjoy using Handy with Parakeet as the model, but the system resource usage is a monster compared to Apple's (although very good). Looks like Wispr Flow uses a cloud model [0]: > Cloud based speech processing infrastructure for 1B users It gets to be a messy comparison because my iPhone can do STT with no latency pretty well fully on device, but Wispr Flow requires a cloud model, but to be fair, older Apple devices do as well. It's not an apples and oranges comparison, but I think those technical details make this a non direct comparison in a few ways. For on-device with low system resource usage, Apple's is pretty damn good. [0] https://wisprflow.ai/post/technical-challenges https://wisprflow.ai/post/technical-challenges
jrm-veris 5mo ago
this is such a great use case for the technology
brightbeige 5mo ago
A while ago I signed up as a sighted person on Be My Eyes. I didn't get as many calls as I had hoped, but I was glad to help out on the few that I could. One call was to read envelopes of incoming mail, another was to read pill bottles, and then there was the two funny guys on big cozy chairs with shopping bags of cereal boxes and wanted to know what was what. I remember one guy really didn't like one type. The app had a unique feature for the sighted person to turn on the camera of the vision impaired person. https://www.bemyeyes.com https://www.bemyeyes.com
- nonethewiser 5mo ago
  Who called you? Blind people?
  - arijun 5mo ago
    That’s the app. Or vision-impaired, at least.
    - deleted 5mo ago
      [deleted]
    - nonethewiser 5mo ago
      But in practice it could just be anyone whose outsourcing vision right? It's basically a free manual vision service.
      - Faaak 5mo ago
        In practice, it's also to help vision impaired people
      - foobiekr 5mo ago
        Are you implying that it might be exploited by unethical tech bros looking to build data sets?
        5mo ago
dgllghr 5mo ago
Putting aside the fact that no company should have direct access to anyone's brain, how cool would it be to be building toward VISOR (from TNG) instead of this. If we could translate sensor signals to the neural circuitry of the brain directly, we wouldn't even need an LLM in the mix. But to have it as an overlay, as supplementary data! With the ability to turn it off of course. (Would a person even be able to turn it off? In the same sense as whether someone can "turn off" social media?) If only we had meaningful human rights and institutions that really protected them... I still can't fully give up the techno-optimism that made me love tech in the first place (and TNG for that matter).
- dagmx 5mo ago
  Brain Control Interface support was already announced last year and afaik is part of iOS already. https://developer.apple.com/documentation/accessibility/brain-computer-interface-hid-reference-for-connecting-to-apple-platforms https://developer.apple.com/documentation/accessibility/brai...
gobdovan 5mo ago
I'm not blind but I sometimes I can't process where things are, even if in front of me. Would be cool to just point to a messy table and see where the keys are. If they offer this as some Vision/Core ML feature, I'd implement the messy table app as soon as these features land. Probably already possible, but simpler if they release this.
MagicMoonlight 5mo ago
And this is why androidlets will never win. They’re too busy selling your data to ever think of disabled people or usability. iOS is just painfully good. I can pause a video, put my finger on text inside the video, and copy it. Until they added it, I didn’t even know how much I needed that.
- f33d5173 5mo ago
  Until they added it, you didn't need it, then suddenly a phone was unusable without it.
- NicuCalcea 5mo ago
  I can do that on my Pixel 6.
  - LocalPCGuy 5mo ago
    Features don't exist until Apple "invents" them /sarcasm
- lern_too_spel 5mo ago
  Live captions has existed for more than 6 years on Android. https://ai.googleblog.com/2019/10/on-device-captioning-with-live-caption.html https://ai.googleblog.com/2019/10/on-device-captioning-with-... Selecting text from anything displayed on the screen for more than 7 years. https://www.thurrott.com/mobile/android/165834/android-9-pie-feature-focus-overview-selection https://www.thurrott.com/mobile/android/165834/android-9-pie... I guess if you knew people using Android, you would have known you needed it 7 years ago?
yreg 5mo ago
It's a shame Apple removed the screen reader announcements ("the Apple logo") from the youtube version of the commercial. https://www.youtube.com/watch?v=B3SmsSCvoss https://www.youtube.com/watch?v=B3SmsSCvoss Those made the ad stand out in my opinion.
- Washuu 5mo ago
  Change the audio language to "English descriptive".
happyPersonR 5mo ago
A lot of us forget it, but things like text to speech, subtitles etc are there for the differently abled Without that, there wouldn’t really be great vlm and conversational models. The AI companies might have paid for the dictation of some videos on their own but voice assistants etc wouldn’t have existed and our ability to have AI that eventually understands the world would be much much harder.
- nonethewiser 5mo ago
  So we're blaming disabled people now.
  - happyPersonR 5mo ago
    lol I’m saying working on accessibility features has helped more than those of us that are sighted. Often times for a lot of us, it’s a drag and comes lower on the priority list, but without it AI, llms etc wouldn’t have the ability to programmatically understand the world. You however…. Maybe need to switch to decaf?
- latexr 5mo ago
  > A lot of us forget it, but things like text to speech, subtitles etc are there for the differently abled They are there for everyone. You don’t need to have a permanent disability to benefit from accessibility features. A device designed to work one handed is useful to someone without an arm or a person with two arms who is holding a baby. Subtitles are useful to someone who can’t hear or someone lying to a sleeping spouse or in a noisy place. “Accessibility needs can be permanent, temporary or situational.” https://www.coursearc.com/accessibility-content-fundamentals/ https://www.coursearc.com/accessibility-content-fundamentals...
everforward 5mo ago
Seems like everyone skipped over this part, but optical controls for motorized wheelchairs is a cool idea (at least to me, maybe that's an old idea). Full VR hasn't done well, but it does continue to make me wonder if there's a market for a stripped and slimmed device. I'd maybe be interested in a device that does optical controls if it fit in regular-sized glasses. I'd be super interested if it had a HUD system (even a super basic one that can only show a handful of symbols). Better still if it had some basic audio, but maintaining the "regular glasses" form factor is more important to me than the HUD or audio.
- willwade 5mo ago
  It's been done for a while - follow the links to who they reference. ie https://www.tolt.tech https://www.tolt.tech but it's their integration they've done into the OS is interesting.
  - jaybeavers 5mo ago
    Hi Will, thanks for the representation. And an actual link!
- hannahstrawbrry 5mo ago
  Seems like a pretty strong indicator that AR glasses are still being worked on, this definitely feels like one of those features Apple ships to refine before the proper hardware is ready.
  - everforward 5mo ago
    I lack that confidence here, because it doesn't appear to really involve AR/VR. Eye tracking is the only feature this really uses afaict. The AR seems like a net negative here, it's just the only device Apple has that has a consistent and remotely convenient view of your eyes. The device is large and makes the user look weird and non-present, which are net negatives. The only benefit of the AR is showing the directional arrows, but they could get the same thing with much less weird looking non-prescription glasses with arrows sharpied on them. More realistically, anyone really using for mobility probably develops muscle memory for which direction to look to go where and then they don't even need that. At that point it's just a really expensive, really clunky camera.
dawnerd 5mo ago
Didn’t they already have subtitle generation for uncaptioned video? Edit: was thinking about this feature https://support.apple.com/guide/iphone/get-live-captions-of-spoken-audio-iphe0990f7bb/ios https://support.apple.com/guide/iphone/get-live-captions-of-...
abhinav-t 5mo ago
These are pretty helpful features for differently abled people. I think it would be really cool if Apple made AI glasses that could communicate with the iPhone thus eliminating the need to point your phone at everything (especially, if you are moving outdoors or in a crowd).
- asadotzler 5mo ago
  We use "disabled people" these days. Or, "people with disabilities." There's debate around person first or not, but I'll leave that to you all to read up one. Regardless of where you come down on DP vs PWD, "differently abled" is a thing of the past.
nrmitchi 5mo ago
These are great improvements, it's good to see Apple investing in improvements like this (especially with the Vision Pro) but I can't help but feel that they utility will remain very low until they make the Vision Pro look significantly less distopian than it does. The form-factor is a significant issue for real-world usage, and it's kind of unclear if there is a plan for a future product line given its (pretty abysmal) initial receiption.
- brokencode 5mo ago
  I don’t think abysmal is the right word. The hardware was widely praised except for being dorky looking and a few other complaints. The price and lack of content and developer interest have been the main problems. And ultimately, people just don’t seem that interested in this product category. Meta ran into the same issue, though at least they targeted gaming where there is a decent niche. VR/AR tech seems cool and futuristic, but hasn’t quite found its killer app yet.
  - bigyabai 5mo ago
    Meta did sell over 20 million headsets. The Quest is definitely lower-margin hardware than Vision Pro, but in terms of install base that's an order of magnitude larger audience. Apple really screwed themselves by only supporting WebXR for cross-platform VR experiences. Soon Valve will ship the Steam Frame, which will likely cost a fraction of the Vision Pro and support bog-standard PC games like H3VR, flight simulators and flatscreen PC titles. Meanwhile, AVP owners will have paid $3,500 for a more powerful chip/headset with a fraction of the content library and featureset that Valve and Meta offer. Vision Pro's lack of audience is entirely a self-imposed failure, it seems.
    - brokencode 5mo ago
      Yeah, the gaming market is a decent sized market. It’s not huge, though, and is not growing very fast. It was a strategic mistake for Apple to not focus on gaming. But realistically, the AVP was always going to be way too expensive for basically anything. Maybe if you could pick one up for like $800 and there was a lot of great 3D immersive content, it could take off. But even then, I feel like it’s just not a product category the average person is that excited about.
JeremyHerrman 5mo ago
Apple loves to stealth test new tech in full public view by sneaking it into relatively mundane places, so debuting agentic AI via accessibility is very on brand. A few other examples: - The Touch Bar was much more than an OLED strip, it was Apple’s first move in the transition to Apple Silicon on macs. The Apple T1 chip in the 2016 Touch Bar MacBooks was the first solely Apple-designed processor to appear in a Mac and took over several responsibilities away from intel chipsets like power management, fans, sleep/wake, access to the camera & mic, and the secure enclave powering touch ID. Then the T2 added encryption of the SSD, audio management, image processing for the camera, and prevented tampering with the boot process - The iPhone 3G shipped with a Liquidmetal SIM eject tool, which is made from a strong custom metal alloy which is "practically unbendable by hand unless you want to hurt or cut your fingers." Although Apple hasn’t released anything with the alloy since then, now nearly 20 years later Apple is rumored to be using liquid metal in their upcoming foldable iPhone. - RealityKit had 3D scanning and a lot of other cool AR capabilities for years which didn’t make sense until the Apple Vision Pro was released.
- paul7986 5mo ago
  And their upcoming smart glasses are the best UX for almost everything they showed the user holding up her phone for. I read their glasses when taking video or pics the lense will light up and or flash more prominently then Metas. Maybe that will help the whole privacy issue and also it's not Meta (do love my Meta or smart glasses as a whole will ditch Metas for Apple quickly as both pair of Metas broke & there's no store for support).
  - bookofjoe 5mo ago
    Same for me! Both pairs of Metas are now inoperative because I can't get them back to factory reset. Once you pull that little blue tab out of them when they're new, it's GAME OVER.
    - diseasedyak 5mo ago
      Dang, really? Why would that be? I mean, I believe you, I'm just shocked. I'm glad I read this, as I was thinking of getting some but not now.
deleted 5mo ago
[deleted]
- deleted 5mo ago
  [deleted]
tonymet 5mo ago
Kudos to apple for providing some of the best accessibility features across their devices. I’ve always appreciated the consistency of reduce transparency, increased contrast, reduced motion, reduced white point, touch areas, color blindness support. And they work well across third party apps. That demands a lot of effort on the API and UI framework to have broad support for something that is mostly a non-sellable feature.
percentcer 5mo ago
Unfortunate thumbnail on that embedded video
- fckgw 5mo ago
  It's a picture of a blind person. The type of person these features are for.
  - mold_aid 5mo ago
    How is it that you know they are blind?
brador 5mo ago
Her phone in the thumbnail has oval camera bumps. It is also extra long. Mine has round camera bumps. Is that a new iphone?
TZubiri 5mo ago
I feel that if we build the UI for blind users first, we would get much more powerful systems, rather than building UI for the seeing users and then slapping a CV to text model of what's shown on the screens. Did not test it yet, but blind users may be more prone to dominate Command Line Interfaces, which are becoming increasingly popular due to its easy integration with LLM
dzonga 5mo ago
actual useful & impactful A.I features not the snake oil being sold daily.
sscaryterry 5mo ago
Luckily the European Accessibility Act has pretty much made PDF/UA a requirement. This should really be the last resort.
- RobMurray 5mo ago
  Accessible PDFs are quite rare in reality. Especially if there are tables, graphics, maths, forms or anything more than plane text.
  - sscaryterry 5mo ago
    Agreed, it is a problem, but it is being legislated in as required for many businesses in the EU going forward.
halapro 5mo ago
Are these already available? I regularly read these announcements and years later I still don't know where to find them, or are not actually functional.
tempodox 5mo ago
Generated subtitles for video does sound useful. Sometimes actors mumble so horribly, I don’t understand a word they’re saying.
- dzhiurgis 5mo ago
  Wonder if it can translate it too. My biggest gripe with Netflix is that they only have like 3 languages and no auto translation. And even bigger gripe is that it's because of the union racket. They apparently need to pay hundreds of thousands for something computers do for free. Insanity.
  - klausa 5mo ago
    It's licensing and/or product decision, not union labor. I guarantee you for most (if not literally all) titles you see, they actually have more subtitles, they just don't / won't show them to _you_. Netflix has gotten pretty good at VPN detection, but you used to be able to see this _very_ clearly by VPNing to a different country in the EU, and the available audio and subtitle languages changed dramatically.
celsoazevedo 5mo ago
Nice. It would be good if Apple could find the time to improve the readability of the white text on green bubbles too. edit: it seems that asking Apple to follow their own accessibility guidelines isn't popular on HN :-(
- kps 5mo ago
  That wouldn't let them say “AI”.
- skiing_crawling 5mo ago
  They won't, its literally part of their sales funnel. They've specifically engineered a bad experience for anyone outside the ecosystem by making it all of their friend's problem too. Its very important for their stock price that text messages sent by non apple products are just slightly more difficult to read.
jmyeet 5mo ago
Can Apple “unveil” Touch ID as a “new” accessibility feature because Face ID is an accessibility nightmare?
OhMeadhbh 5mo ago
I miss Apple during it's hey-day. There was a time when Apple was the sine qua non for #a11y and #hci. Then Steve came back.
an_d_rew 5mo ago
As someone slowly and idiopathically losing their hearing, and as someone just... getting older and losing visual acuity... Thank you, Apple, for taking accessibility seriously and dedicating resources towards it. I very much appreciate it, and the work of the entire accessibility team.
- aucisson_masque 5mo ago
  Have you ever tried the accessibility feature on Android ? Would you recommend one or the other for your use case ? And what about windows (if you use it) ? I think that we should all be concerned by the accessibility feature, we never know what is going to happen in life.
  - an_d_rew 5mo ago
    I haven’t used Android or Windows in any meaningful way for years, so I cannot comment on them. I can tell you that the hearing accommodations on the AirPod Pro 2/3 headphones brought literal tears to my eyes because of how fabulous it makes music sound for me. This is a a LOT more work than just adding an equalizer because you have to do multiband real time compression and expansion, in relation to other frequency bands and respecting band-specific sound energy limits. I know I might sound like I’m gushing, and I kinda’ am. They didn’t have to put in the time or energy to do that or maintain it and they did ... and for that, like I said, I am extraordinarily grateful.
jonnyasmar 5mo ago
Different angle from the developer side: Apple's a11y API at the OS level is genuinely good. It's the WebKit-embedded-in-native gap that breaks. Shipped a Tauri app where Monaco editor lived inside WKWebView and found out the hard way that VoiceOver's `accessibilitySupport: auto` mode silently breaks backward text selections in Monaco — only setting it to "off" gave us correct selections. Which meant choosing between functional text selection or VoiceOver support, and the answer was selection. Rock-solid in AppKit/UIKit. Falls over at the embedded-WebView seam where most modern desktop apps actually live.
MYEUHD 5mo ago
This feels like cart before the horse. As of macOS 15 (and I don't think they fixed it in 26), you can only increase the font size of first-party apps on macOS. The global font size setting doesn't apply to third-party apps, even those built using Apple's frameworks.
diogenescynic 5mo ago
Cool, but why is Apple making the new iPhone 17e chips in Israel when Israel just coordinated a mass terrorist attack using pagers? I personally don't want my iPhone being used as a bomb if I say something to criticize Israel. Israel is the last place I want to have anything to do with my phone.
cybercatgurrl 5mo ago
this is exactly how you fight people’s notion that AI is bad. you change their lives with it. make it something indispensable for a subset of users so that being anti-AI is indefensible. for instance, “how dare you threaten people’s ability to navigate the world independently”
commandersaki 5mo ago
Apple accessibility is the #1 reason why I'm in the Apple ecosystem. It does have its shortcomings, but it seems miles ahead of everything else in tech after having tried other ecosystems. It is hard to place a price on these features, but I imagine it is worth significantly more than the base cost of hardware.
Brajeshwar 5mo ago
All of the brilliant video and voice over was expected, I love the final, “The Apple Logo," that is that taking care of the back of the fences. With AI-this-&-AI-that, the human intuition to think of the unnoticable subtle differentiation will be the thing that stands out of your cohort.
jcattle 5mo ago
And here I was, wondering why accessibility features are on top of HN. Of course it's AI.
quotz 5mo ago
I can't think of a worse place to be blind than the NYC subway, especially because even being non-blind is still quite horrible there.

726 points by interpol_p 5mo ago

> The total amount due on the bill is $83.89. Please verify this amount with your utility provider or by using Text Detection before making a payment. 1. Use AI to determine how much a bill is for 2. Call up the people who billed you and ask them how much they billed you 3. Pay billed amount

deleted 5mo ago
[deleted]
tramc 5mo ago
It’s still useful to get the information instantly and verify it later. Arguably asking someone you trust to read the number for you might be a better idea than calling the company. Not everybody has that option though.
- Someone 5mo ago
  And not everybody wants to use that option all the time. Asking a human makes you feel dependent more than using a tool does.
  - kotaKat 5mo ago
    Aaaand the logistics of making that call to the company to confirm the amount on the bill can get awkward. IVR and hold-time hell just to get a human to have to explain your predicament as to why you're asking for such a mundane piece of information that was in fifty other touchpoints that you couldn't access as quickly or easily. (I'm also picturing the poor CSR at the other end of the phone wading through hundreds and hundreds of call logs over the years for simple requests and managers up above screaming 'why is this guy calling us all the damn time costing us money'...)
stellamariesays 5mo ago
[flagged]
dewey 5mo ago
Once you paid the same bill for a few months you'll know how much your phone bill will roughly be and you'll not have to do. They obviously have to put that line in there, just like ChatGPT saying "Please verify everything we tell you" in the footer.
martinflack 5mo ago
It might be useful if it remembered a bill for, say, 60 days, and could also comment on percent difference since the last one. "The total amount due on the bill is $83.89 which is 4% higher than last month's bill from the same company."
kube-system 5mo ago
I presume that calling customer support is at least as frustrating for people with disabilities as it is for anyone.

nechuchelo 5mo ago

This looks like a genuinely useful application of LLMs. I wish more companies focused on how they can help humans instead of replacing us or squeezing us as hard as possible in the name of productivity.

koolala 5mo ago
Its with their servers right? Do they trust a iPhone with their life? Or they are trusting their data center?
- nechuchelo 5mo ago
  Looks like some of the features might use on-device models. They mention subtitle generation works on-device.
bsanders343 5mo ago
I agree. There seems to be a lot of potential in this space (from my outsider view). I really hope that this issue from an earlier article (https://news.ycombinator.com/item?id=48178378 https://news.ycombinator.com/item?id=48178378) doesn't become common enough to make useful functionality like this a danger. Seems unlikely in the short term but as use cases grow, so might the bad actors.
c0wb0yc0d3r 5mo ago
I think we should reserve judgment until this lands in the hands of the people it helps. My experience is limited to my elderly parents who have trouble seeing. With the text size Apple allows them to set it to, their phones are unreadable. Text runs off the screen in every app, 1st and 3rd party. In their bill example, the user is told to confirm with the provider. Why not offer to call the number on the bill? Instead of telling them to use text detection, do it for them? Presumably Apple Intelligence would already have that capability. I’m afraid this will be a gimmick at best. EDIT: Forgot to mention, the grip is good to see. Hopefully they don’t charge the apple tax on it.
- kps 5mo ago

mohsen1 5mo ago

Fun fact: This video was made accessible to sighted people because no blind person would ever listen to voice at that speed. Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds.

Sweepi 5mo ago
dont you worry, as a sighted person I am also infuriated by apples slooow reading speed, e.g. for "Announce Notifications".
- hightrix 5mo ago
  Also as a sighted person, this is why I hate the modern trend of using the video format to show 3-4 bullet points. Just give me the text.
embedding-shape 5mo ago
> Honestly if you ever observe a blind person using computers you'd impressed how they can listen to audio at unimaginable speeds. Even better, fire up Orca (or whatever screenreader application your OS comes with) yourself and try to use your computer while shutting your eyes, kind of eye-opening (no pun intended) what kind of experience these sort of users typically get. And also, you quickly start to understand why they set the speech rate for their voice synthesizer to be so fast, it's almost unbearable navigating applications (and particularly lists) otherwise.
- seviu 5mo ago
  That time my Mac display broke and I had to log in taught me much about how important learning accessibility is even for non blind people.
- jchw 5mo ago
  When I was at Google, I'd periodically test our (internal-only) app with Chromevox with the display off. It's not that it sounded like it would be easy, but it really is a challenge, and I can only imagine the muscle memory built up over time of trying to work around accessibility bugs and strange behaviors. Unfortunately it seems impossible to get all that much funding for accessibility work :/ I wonder what ever happened to the Newton accessibility bus intended to supplement Wayland...

jansan 5mo ago

Since Apple uses Gemini to power its AI, are those features actually powered by Google Gemini?

k4rnaj1k 5mo ago
[dead]
jjice 5mo ago
They don't get, but they will be using Gemini derived models with iOS 27. For now it's all their own models.

exitb 5mo ago

As Apple shifts towards services and fancy software features, I wonder how do they expect to stay competitive by only releasing them for a subset of languages.

layer8 5mo ago
They roughly know how many of their users use a particular language.

tekacs 5mo ago

I'm super glad that they're doing this, but once again unexcited for another decade of Apple self-privileging on this stuff so they're the only ones allowed to touch or improve any of this surface, or UX outside an app's tiny box. People talk a lot about how MacOS has gone downhill but I feel like it would have been a good start if developers could continue to patch over Apple's shortcomings like they used to be able to. I imagine that we would be a few years into a spectrum of tools like this if they didn't lock it down like they do. Totally aware that plenty of HN commenters are very glad that Apple keeps this locked down. I'm just the other opinion, that's all.

abhikul0 5mo ago

On-device video subtitles generation is exciting, should help with watching videos on mute. This seems like a low hanging fruit that should've already been grabbed by an app but I can't find any.

randusername 5mo ago

Accessibility features are such a great way to keep technology focused on real-world problems and real-world experiences. I think the trap in creating anything is doing it for a crowd. Art, software, anything... it turns out better when it is made with a specific, named individual in-mind. Accessibility features are almost always championed and field-tested with one specific loved one in mind and I think that's what keeps the technical solutions personable and grounded.

Almondsetat 5mo ago

I have difficulty trusting this. There are plenty of videos online of LLMs making up stuff like "I just ate a hot dog, is there mustard around my mouth?" "No, everything is clean" while there is a big yellow stain om the guy's face

postalrat 5mo ago
Like coding, creating images or text, maybe the alternative of doing it yourself is too easy or enjoyable for you. Don't expect that will be true for everyone.
- Almondsetat 5mo ago
  Did you reply to the wrong person? What are you even trying to say here?
  - postalrat 5mo ago
    You say you don't trust it but whats your alternative assuming you lost your vision?
WarmWash 5mo ago
The problem is using a language model to assess images. Probably 80% of "LLM's are below expectation" complaints (from the general population) involves some form of image analyses. Image tokenization is hard because unlike language tokenization, where every token is extremely dense with meaning, image tokens tends to be meaningless or irrelevant but are processed all the same. Give an SOTA LLM a picture of toothpicks and ask it to move one to make a square, and it will probably struggle and fumble it. But give a mid-size LLM from 2 years ago the same problem in verbal form, and it will nail it almost every time. That takeaway is, do everything you can to avoid having the LLM need to rely on images for the answer.
- gruez 5mo ago
  I thought all the recent models are "multimodal"? Is the image part just sticking an image recognizer in front of the text model?

Darwins_Toffees 5mo ago

"Vehicle Motion Cues come to visionOS, which can help reduce motion sickness for people who use Apple Vision Pro as a passenger in a moving vehicle. Vision Pro will also support face gestures for performing taps and system actions, plus a new way to select elements with one’s eyes while using Dwell Control." Maybe just don't wear them in a car?

dmix 5mo ago
Wearing a headset in the back of an Uber doesn't sound that crazy, I use those motion cues on my iPhone even though I don't struggle with motion sickness https://www.youtube.com/shorts/OxbjggMcKrk https://www.youtube.com/shorts/OxbjggMcKrk
- nozzlegear 5mo ago
  I use them as well. I'm usually the driver so I don't typically look at my phone while the car is moving, but I recently rode along with a family member to an event. They handed me their iPhone to look at something and I felt totally disoriented trying to look at a moving screen in a moving car. I had to resist the urge to turn on the motion cues.
brookst 5mo ago
Trains are a thing.
yreg 5mo ago
>Maybe just don't wear them in a car? Why not?
- throwaway132448 5mo ago
  Because the more we reject our shared reality and substitute it with each our own, the less humane we become.
  - jkman 5mo ago

baxuz 5mo ago

Now we know why the new AirPods will have cameras!

testfrequency 5mo ago

I don’t want to discredit more advancements in accessibility, but this feels like accessibility porn. I have fond memories of an old coworker 10 years ago who is blind. He would use his phone no problem, texting, going about his day, he was even on Tinder (credit to Tinder for making their app so accessible long ago). He would commute on his own, walk to the train station, even transfer to another train during peak rush hour. I’m not saying it was all easy for him, but nothing in this video really stood out to me more than what shirt was on the bed. I know other services/apps have long existed to be the “eyes” for people who need support, but this video feels….uneventful? I may be cynical about this though, as I often hate how Apple’s marketing makes these emotional bids about how life-critical they are to society - which is fair to a degree..but it just feels cheap to be glamorising “look! we saved this person from pending doom, cool right??”

lwkl 5mo ago
I mean even if it is marketing for them they still did the work and developed these features. I had some vision issues recently and was glad there were options to make text more legible to me. Additionally I don't believe this is just marketing. This is adaption to a changing market. Apple's customer base is aging and having these kinds of features will allow them to keep using Apple products for a longer.
- testfrequency 5mo ago
  They have done the work, but I don’t see much work that’s beyond what’s been previously capable without Apple Intelligence. The marketing of Apple Intelligence is weak here, not the foundational abilities.
RobMurray 5mo ago
for every person like your coworker, there are probably several who have a much harder time with technology and who would benefit from a simpler interface. If this includes improvements to the screen recognition feature in Voice Over, it could provide accessibility for apps where the developer doesn't care about accessibility, which is extremely common. The vision capabilities could be useful if they are done well, but I suspect that will always be covered better by 3rd party apps.

devinprater 5mo ago

There's my dopamine hit for the year.

mistersquid 5mo ago

> A new power wheelchair control feature leverages the precision eye-tracking system on Apple Vision Pro to offer a responsive input method for compatible alternative drive systems. [0] The above caption for Apple Vision Pro is for a video that to me, as an Apple Vision Pro user, is discomforting. More questions are raised than are answered by the short video: Is the user able to fit the Apple Vision Pro by him/herself? What happens when dwelling on a directional control misregisters? Can the user recalibrate the "Eyes and Hands" setting? Dwelling on a control displaces focus and there may be impeding objects in the path of the power wheelchair. Is this really a good idea? To my sensibility, the video is unsettling (at best), especially given how cumbersome Apple Vision Pro is. [0] https://www.apple.com/newsroom/2026/05/apple-unveils-new-accessibility-features-and-updates-with-apple-intelligence/#:~:text=A%20new%20power%20wheelchair%20control%20feature%20leverages%20the%20precision%20eye%2Dtracking%20system%20on%20Apple%20Vision%20Pro%20to%20offer%20a%20responsive%20input%20method%20for%20compatible%20alternative%20drive%20systems. https://www.apple.com/newsroom/2026/05/apple-unveils-new-acc...

jkman 5mo ago
Your concerns are completely nonsensical. It's clearly being marketed as a healthcare tool for people with debilitating injuries that preclude the use of hand-powered wheelchair controls, severe situations where there's no neck-down control and users would be limited to controls like head-tilt or mouth actuated systems. These people obviously require daily care to simply get them out of bed and into the chair and back again every single day - their nurse could just put on their Vision Pro for them! This seems like an incredible leap forward for people in this situation, if they iterate on this and it gets better then this could be a very viable wheelchair control system in the future.
- mistersquid 5mo ago
  > Your concerns are completely nonsensical. With all due respect, my concerns are not nonsensical but borne of my daily use with Apple Vision Pro and my awareness of the limitations of dwell control. Iterating on this idea with a device lighter than Apple Vision Pro and improvements to dwell control would likely be required before this could ship to larger populations of disabled users, but that is not what is depicted in the video. My sense is that the possibility of an accessibility affordance with people who are severely disabled is driving opinions in this case more than the reality of what’s available. To my mind, much of these AX announcements are reminiscent of the circumstance that led John Gruber to author “Something Is Rotten in the State of Cupertino”, which is that these are not shipping features but ones slated for “some time later this year”. I’m a huge AX fan and work directly in the domain space, but something about that video in particular coupled with my near-daily use of Apple Vision Pro doesn’t feel right.

seeeeebt 5mo ago

Surely a blind person relies a lot on audio input?

isityettime 5mo ago
Maybe on a smartphone, but usually not on a computer. Keyboards are pretty good. The other thing is that if you're around others, voice input means you have no privacy. Even if you're not doing anything particularly private, it's a bit awkward and potentially embarrassing. If you use touch input in conjunction with a screen reader, you can be more like a "normal" user in that what you're doing is just between you and your phone.
asadotzler 5mo ago
Audio input is far more commonly used by people with mobility difficulties. Imagine that your hands shake a lot or that you don't have limbs. That makes using a keyboard and mouse difficult or impossible and voice input can help. Blind users generally use keyboards for input, the typical ones you find on a PC but also sometimes the keys on their Braille display.

zersiax 5mo ago

Honestly as a blind person and blind developer myself, most of these features get a shrug at best. For one, there's already a bunch of third-party apps that do most if not all of this (Seeing AI, Envision AI, BeMyEyes, Aira, etc.). So at best, this does what all those apps are doing but faster and on-device, which may or may not mean it is also more inaccurate, we'll have to see. In the meantime, Mac OS's screen reader, VoiceOver, has been left to essentially exist in maintenance mode for years, where users have had to build, arguably impressive, third-party solutions to add features to the thing that comparable screen readers on Windows have had for a really long time. Through that lens, this all looks a bit performative to me, but again, maybe I'll be pleasantly surprised. The one thing I'm mildly excited to see is the improvement to Voice Control, as guessing what the programmatic name of a button is or having to constantly use a numbers grid to target elements doesn't sound fun. To respond to what I see in some of the comments: - On speech rate: It does take quite a bit of practice to crank up the speech rate and there's a degree of retraining you need to do when you switch voices. A lot of more "human" sounding voices are harder to follow at super high speeds which is why a lot of people prefer more robotic but consistent speech and generally aren't convinced by AI-powered TTS yet; they often fall apart if you raise the speech rate past a certain point. - Re: actually waiting for the target audience's verdict: This is so important. I see more and more companies, individuals etc. talk about accessibility, build accessibility solutions and evangelize AI for accessibility without EVER talking to the people they claim to help. This will almost certainly mean mistakes will be made, up to and including doing more harm than good. If you want to do accessibility right, that includes AI products of any kind, hire people with lived experience or you'll get the equivalent of machine-translated text, hackerproof security in one click or an AI-powered coffee bar that orders thousands of rubber gloves. Coincidental note: I have time for new projects right now :P

Tox46 5mo ago
it's so validating getting the same conclusion we got to from someone that i've never met. it seems that they create these products without ever speaking with someone with that problem. Funnily enough we're creating a competitor of these third party app that you mention, with the huge experience of my colleague that is son of blind parents. We have an mvp online but it's not much yet and i really don't want to be the "do you know i have an app?" guy.

nikhilpareek13 5mo ago

[flagged]

deleted 5mo ago

[deleted]

deleted 5mo ago

[deleted]

postalcoder 5mo ago

One thing Apple really needs to get right is speech to text transcription. They've nailed accessibility in so many ways and yet it feels like they're a decade behind on properly transcribing voices. At least half a decade. Input on the iPhone is so dreadful nowadays. Their palm rejection is definitely worse than before, so mistyping is more frequent. Their text-correction algorithm for typing is worse than before, and it frequently makes incorrect corrections to words that I don't notice, because they change words a few words back from where I typed. And STT hasn't improved. On top of that, my fingers are tired of the phone form factor. Please make the iphone not a chore to use, apple.

terabytest 5mo ago
Wispr Flow is a masterclass in STT. Apple's solution feels like it's from the last century in comparison. Same applies with Apple's TTS when you have ElevenLabs and OpenAI running laps around it. All I need is for my iPhone to do those things natively at the same quality level (because in Apple's walled garden that's the only way to get them usable everywhere).
- adamcharnock 5mo ago
  FWIW - I also really like Wispr Flow, but I moved to running the 'Whisper Large' model locally using Handy (https://github.com/cjpais/Handy https://github.com/cjpais/Handy), which has been essentially as good, while also having lower latency.
  - dceddia 5mo ago
    Handy is great. It exposes a bunch of open models beyond Whipser too, and though I haven’t tried too many of them, I’ll throw in a rec for the Parakeet model which feels pretty much on par with Whisper for accuracy and is way way faster.
- jjice 5mo ago
  But Apple's uses so few system resources and runs fully on device on newer iPhone models (16+ I believe). It's so efficient. I really enjoy using Handy with Parakeet as the model, but the system resource usage is a monster compared to Apple's (although very good). Looks like Wispr Flow uses a cloud model [0]: > Cloud based speech processing infrastructure for 1B users It gets to be a messy comparison because my iPhone can do STT with no latency pretty well fully on device, but Wispr Flow requires a cloud model, but to be fair, older Apple devices do as well. It's not an apples and oranges comparison, but I think those technical details make this a non direct comparison in a few ways. For on-device with low system resource usage, Apple's is pretty damn good. [0] https://wisprflow.ai/post/technical-challenges https://wisprflow.ai/post/technical-challenges

jrm-veris 5mo ago

this is such a great use case for the technology

brightbeige 5mo ago

A while ago I signed up as a sighted person on Be My Eyes. I didn't get as many calls as I had hoped, but I was glad to help out on the few that I could. One call was to read envelopes of incoming mail, another was to read pill bottles, and then there was the two funny guys on big cozy chairs with shopping bags of cereal boxes and wanted to know what was what. I remember one guy really didn't like one type. The app had a unique feature for the sighted person to turn on the camera of the vision impaired person. https://www.bemyeyes.com https://www.bemyeyes.com

nonethewiser 5mo ago
Who called you? Blind people?
- arijun 5mo ago
  That’s the app. Or vision-impaired, at least.
  - deleted 5mo ago
    [deleted]
  - nonethewiser 5mo ago
    But in practice it could just be anyone whose outsourcing vision right? It's basically a free manual vision service.
    - Faaak 5mo ago
      In practice, it's also to help vision impaired people
    - foobiekr 5mo ago
      Are you implying that it might be exploited by unethical tech bros looking to build data sets?
      - 5mo ago

dgllghr 5mo ago

Putting aside the fact that no company should have direct access to anyone's brain, how cool would it be to be building toward VISOR (from TNG) instead of this. If we could translate sensor signals to the neural circuitry of the brain directly, we wouldn't even need an LLM in the mix. But to have it as an overlay, as supplementary data! With the ability to turn it off of course. (Would a person even be able to turn it off? In the same sense as whether someone can "turn off" social media?) If only we had meaningful human rights and institutions that really protected them... I still can't fully give up the techno-optimism that made me love tech in the first place (and TNG for that matter).

dagmx 5mo ago
Brain Control Interface support was already announced last year and afaik is part of iOS already. https://developer.apple.com/documentation/accessibility/brain-computer-interface-hid-reference-for-connecting-to-apple-platforms https://developer.apple.com/documentation/accessibility/brai...

gobdovan 5mo ago

I'm not blind but I sometimes I can't process where things are, even if in front of me. Would be cool to just point to a messy table and see where the keys are. If they offer this as some Vision/Core ML feature, I'd implement the messy table app as soon as these features land. Probably already possible, but simpler if they release this.

MagicMoonlight 5mo ago

And this is why androidlets will never win. They’re too busy selling your data to ever think of disabled people or usability. iOS is just painfully good. I can pause a video, put my finger on text inside the video, and copy it. Until they added it, I didn’t even know how much I needed that.

f33d5173 5mo ago
Until they added it, you didn't need it, then suddenly a phone was unusable without it.
NicuCalcea 5mo ago
I can do that on my Pixel 6.
- LocalPCGuy 5mo ago
  Features don't exist until Apple "invents" them /sarcasm
lern_too_spel 5mo ago
Live captions has existed for more than 6 years on Android. https://ai.googleblog.com/2019/10/on-device-captioning-with-live-caption.html https://ai.googleblog.com/2019/10/on-device-captioning-with-... Selecting text from anything displayed on the screen for more than 7 years. https://www.thurrott.com/mobile/android/165834/android-9-pie-feature-focus-overview-selection https://www.thurrott.com/mobile/android/165834/android-9-pie... I guess if you knew people using Android, you would have known you needed it 7 years ago?

yreg 5mo ago

It's a shame Apple removed the screen reader announcements ("the Apple logo") from the youtube version of the commercial. https://www.youtube.com/watch?v=B3SmsSCvoss https://www.youtube.com/watch?v=B3SmsSCvoss Those made the ad stand out in my opinion.

Washuu 5mo ago
Change the audio language to "English descriptive".

happyPersonR 5mo ago

A lot of us forget it, but things like text to speech, subtitles etc are there for the differently abled Without that, there wouldn’t really be great vlm and conversational models. The AI companies might have paid for the dictation of some videos on their own but voice assistants etc wouldn’t have existed and our ability to have AI that eventually understands the world would be much much harder.

nonethewiser 5mo ago
So we're blaming disabled people now.
- happyPersonR 5mo ago
  lol I’m saying working on accessibility features has helped more than those of us that are sighted. Often times for a lot of us, it’s a drag and comes lower on the priority list, but without it AI, llms etc wouldn’t have the ability to programmatically understand the world. You however…. Maybe need to switch to decaf?
latexr 5mo ago
> A lot of us forget it, but things like text to speech, subtitles etc are there for the differently abled They are there for everyone. You don’t need to have a permanent disability to benefit from accessibility features. A device designed to work one handed is useful to someone without an arm or a person with two arms who is holding a baby. Subtitles are useful to someone who can’t hear or someone lying to a sleeping spouse or in a noisy place. “Accessibility needs can be permanent, temporary or situational.” https://www.coursearc.com/accessibility-content-fundamentals/ https://www.coursearc.com/accessibility-content-fundamentals...

everforward 5mo ago

Seems like everyone skipped over this part, but optical controls for motorized wheelchairs is a cool idea (at least to me, maybe that's an old idea). Full VR hasn't done well, but it does continue to make me wonder if there's a market for a stripped and slimmed device. I'd maybe be interested in a device that does optical controls if it fit in regular-sized glasses. I'd be super interested if it had a HUD system (even a super basic one that can only show a handful of symbols). Better still if it had some basic audio, but maintaining the "regular glasses" form factor is more important to me than the HUD or audio.

willwade 5mo ago
It's been done for a while - follow the links to who they reference. ie https://www.tolt.tech https://www.tolt.tech but it's their integration they've done into the OS is interesting.
- jaybeavers 5mo ago
  Hi Will, thanks for the representation. And an actual link!
hannahstrawbrry 5mo ago
Seems like a pretty strong indicator that AR glasses are still being worked on, this definitely feels like one of those features Apple ships to refine before the proper hardware is ready.
- everforward 5mo ago
  I lack that confidence here, because it doesn't appear to really involve AR/VR. Eye tracking is the only feature this really uses afaict. The AR seems like a net negative here, it's just the only device Apple has that has a consistent and remotely convenient view of your eyes. The device is large and makes the user look weird and non-present, which are net negatives. The only benefit of the AR is showing the directional arrows, but they could get the same thing with much less weird looking non-prescription glasses with arrows sharpied on them. More realistically, anyone really using for mobility probably develops muscle memory for which direction to look to go where and then they don't even need that. At that point it's just a really expensive, really clunky camera.

dawnerd 5mo ago

Didn’t they already have subtitle generation for uncaptioned video? Edit: was thinking about this feature https://support.apple.com/guide/iphone/get-live-captions-of-spoken-audio-iphe0990f7bb/ios https://support.apple.com/guide/iphone/get-live-captions-of-...

abhinav-t 5mo ago

These are pretty helpful features for differently abled people. I think it would be really cool if Apple made AI glasses that could communicate with the iPhone thus eliminating the need to point your phone at everything (especially, if you are moving outdoors or in a crowd).

asadotzler 5mo ago
We use "disabled people" these days. Or, "people with disabilities." There's debate around person first or not, but I'll leave that to you all to read up one. Regardless of where you come down on DP vs PWD, "differently abled" is a thing of the past.

nrmitchi 5mo ago

These are great improvements, it's good to see Apple investing in improvements like this (especially with the Vision Pro) but I can't help but feel that they utility will remain very low until they make the Vision Pro look significantly less distopian than it does. The form-factor is a significant issue for real-world usage, and it's kind of unclear if there is a plan for a future product line given its (pretty abysmal) initial receiption.

brokencode 5mo ago
I don’t think abysmal is the right word. The hardware was widely praised except for being dorky looking and a few other complaints. The price and lack of content and developer interest have been the main problems. And ultimately, people just don’t seem that interested in this product category. Meta ran into the same issue, though at least they targeted gaming where there is a decent niche. VR/AR tech seems cool and futuristic, but hasn’t quite found its killer app yet.
- bigyabai 5mo ago
  Meta did sell over 20 million headsets. The Quest is definitely lower-margin hardware than Vision Pro, but in terms of install base that's an order of magnitude larger audience. Apple really screwed themselves by only supporting WebXR for cross-platform VR experiences. Soon Valve will ship the Steam Frame, which will likely cost a fraction of the Vision Pro and support bog-standard PC games like H3VR, flight simulators and flatscreen PC titles. Meanwhile, AVP owners will have paid $3,500 for a more powerful chip/headset with a fraction of the content library and featureset that Valve and Meta offer. Vision Pro's lack of audience is entirely a self-imposed failure, it seems.
  - brokencode 5mo ago
    Yeah, the gaming market is a decent sized market. It’s not huge, though, and is not growing very fast. It was a strategic mistake for Apple to not focus on gaming. But realistically, the AVP was always going to be way too expensive for basically anything. Maybe if you could pick one up for like $800 and there was a lot of great 3D immersive content, it could take off. But even then, I feel like it’s just not a product category the average person is that excited about.

JeremyHerrman 5mo ago

Apple loves to stealth test new tech in full public view by sneaking it into relatively mundane places, so debuting agentic AI via accessibility is very on brand. A few other examples: - The Touch Bar was much more than an OLED strip, it was Apple’s first move in the transition to Apple Silicon on macs. The Apple T1 chip in the 2016 Touch Bar MacBooks was the first solely Apple-designed processor to appear in a Mac and took over several responsibilities away from intel chipsets like power management, fans, sleep/wake, access to the camera & mic, and the secure enclave powering touch ID. Then the T2 added encryption of the SSD, audio management, image processing for the camera, and prevented tampering with the boot process - The iPhone 3G shipped with a Liquidmetal SIM eject tool, which is made from a strong custom metal alloy which is "practically unbendable by hand unless you want to hurt or cut your fingers." Although Apple hasn’t released anything with the alloy since then, now nearly 20 years later Apple is rumored to be using liquid metal in their upcoming foldable iPhone. - RealityKit had 3D scanning and a lot of other cool AR capabilities for years which didn’t make sense until the Apple Vision Pro was released.

paul7986 5mo ago
And their upcoming smart glasses are the best UX for almost everything they showed the user holding up her phone for. I read their glasses when taking video or pics the lense will light up and or flash more prominently then Metas. Maybe that will help the whole privacy issue and also it's not Meta (do love my Meta or smart glasses as a whole will ditch Metas for Apple quickly as both pair of Metas broke & there's no store for support).
- bookofjoe 5mo ago
  Same for me! Both pairs of Metas are now inoperative because I can't get them back to factory reset. Once you pull that little blue tab out of them when they're new, it's GAME OVER.
  - diseasedyak 5mo ago
    Dang, really? Why would that be? I mean, I believe you, I'm just shocked. I'm glad I read this, as I was thinking of getting some but not now.

deleted 5mo ago

[deleted]

deleted 5mo ago
[deleted]

tonymet 5mo ago

Kudos to apple for providing some of the best accessibility features across their devices. I’ve always appreciated the consistency of reduce transparency, increased contrast, reduced motion, reduced white point, touch areas, color blindness support. And they work well across third party apps. That demands a lot of effort on the API and UI framework to have broad support for something that is mostly a non-sellable feature.

percentcer 5mo ago

Unfortunate thumbnail on that embedded video

fckgw 5mo ago
It's a picture of a blind person. The type of person these features are for.
- mold_aid 5mo ago
  How is it that you know they are blind?

brador 5mo ago

Her phone in the thumbnail has oval camera bumps. It is also extra long. Mine has round camera bumps. Is that a new iphone?

TZubiri 5mo ago

I feel that if we build the UI for blind users first, we would get much more powerful systems, rather than building UI for the seeing users and then slapping a CV to text model of what's shown on the screens. Did not test it yet, but blind users may be more prone to dominate Command Line Interfaces, which are becoming increasingly popular due to its easy integration with LLM

dzonga 5mo ago

actual useful & impactful A.I features not the snake oil being sold daily.

sscaryterry 5mo ago

Luckily the European Accessibility Act has pretty much made PDF/UA a requirement. This should really be the last resort.

RobMurray 5mo ago
Accessible PDFs are quite rare in reality. Especially if there are tables, graphics, maths, forms or anything more than plane text.
- sscaryterry 5mo ago
  Agreed, it is a problem, but it is being legislated in as required for many businesses in the EU going forward.

halapro 5mo ago

Are these already available? I regularly read these announcements and years later I still don't know where to find them, or are not actually functional.

tempodox 5mo ago

Generated subtitles for video does sound useful. Sometimes actors mumble so horribly, I don’t understand a word they’re saying.

dzhiurgis 5mo ago
Wonder if it can translate it too. My biggest gripe with Netflix is that they only have like 3 languages and no auto translation. And even bigger gripe is that it's because of the union racket. They apparently need to pay hundreds of thousands for something computers do for free. Insanity.
- klausa 5mo ago
  It's licensing and/or product decision, not union labor. I guarantee you for most (if not literally all) titles you see, they actually have more subtitles, they just don't / won't show them to _you_. Netflix has gotten pretty good at VPN detection, but you used to be able to see this _very_ clearly by VPNing to a different country in the EU, and the available audio and subtitle languages changed dramatically.

celsoazevedo 5mo ago

Nice. It would be good if Apple could find the time to improve the readability of the white text on green bubbles too. edit: it seems that asking Apple to follow their own accessibility guidelines isn't popular on HN :-(

kps 5mo ago
That wouldn't let them say “AI”.
skiing_crawling 5mo ago
They won't, its literally part of their sales funnel. They've specifically engineered a bad experience for anyone outside the ecosystem by making it all of their friend's problem too. Its very important for their stock price that text messages sent by non apple products are just slightly more difficult to read.

jmyeet 5mo ago

Can Apple “unveil” Touch ID as a “new” accessibility feature because Face ID is an accessibility nightmare?

OhMeadhbh 5mo ago

I miss Apple during it's hey-day. There was a time when Apple was the sine qua non for #a11y and #hci. Then Steve came back.

an_d_rew 5mo ago

As someone slowly and idiopathically losing their hearing, and as someone just... getting older and losing visual acuity... Thank you, Apple, for taking accessibility seriously and dedicating resources towards it. I very much appreciate it, and the work of the entire accessibility team.

aucisson_masque 5mo ago
Have you ever tried the accessibility feature on Android ? Would you recommend one or the other for your use case ? And what about windows (if you use it) ? I think that we should all be concerned by the accessibility feature, we never know what is going to happen in life.
- an_d_rew 5mo ago
  I haven’t used Android or Windows in any meaningful way for years, so I cannot comment on them. I can tell you that the hearing accommodations on the AirPod Pro 2/3 headphones brought literal tears to my eyes because of how fabulous it makes music sound for me. This is a a LOT more work than just adding an equalizer because you have to do multiband real time compression and expansion, in relation to other frequency bands and respecting band-specific sound energy limits. I know I might sound like I’m gushing, and I kinda’ am. They didn’t have to put in the time or energy to do that or maintain it and they did ... and for that, like I said, I am extraordinarily grateful.

jonnyasmar 5mo ago

Different angle from the developer side: Apple's a11y API at the OS level is genuinely good. It's the WebKit-embedded-in-native gap that breaks. Shipped a Tauri app where Monaco editor lived inside WKWebView and found out the hard way that VoiceOver's `accessibilitySupport: auto` mode silently breaks backward text selections in Monaco — only setting it to "off" gave us correct selections. Which meant choosing between functional text selection or VoiceOver support, and the answer was selection. Rock-solid in AppKit/UIKit. Falls over at the embedded-WebView seam where most modern desktop apps actually live.

MYEUHD 5mo ago

This feels like cart before the horse. As of macOS 15 (and I don't think they fixed it in 26), you can only increase the font size of first-party apps on macOS. The global font size setting doesn't apply to third-party apps, even those built using Apple's frameworks.

diogenescynic 5mo ago

Cool, but why is Apple making the new iPhone 17e chips in Israel when Israel just coordinated a mass terrorist attack using pagers? I personally don't want my iPhone being used as a bomb if I say something to criticize Israel. Israel is the last place I want to have anything to do with my phone.

cybercatgurrl 5mo ago

this is exactly how you fight people’s notion that AI is bad. you change their lives with it. make it something indispensable for a subset of users so that being anti-AI is indefensible. for instance, “how dare you threaten people’s ability to navigate the world independently”

commandersaki 5mo ago

Apple accessibility is the #1 reason why I'm in the Apple ecosystem. It does have its shortcomings, but it seems miles ahead of everything else in tech after having tried other ecosystems. It is hard to place a price on these features, but I imagine it is worth significantly more than the base cost of hardware.

Brajeshwar 5mo ago

All of the brilliant video and voice over was expected, I love the final, “The Apple Logo," that is that taking care of the back of the fences. With AI-this-&-AI-that, the human intuition to think of the unnoticable subtle differentiation will be the thing that stands out of your cohort.

jcattle 5mo ago

And here I was, wondering why accessibility features are on top of HN. Of course it's AI.

quotz 5mo ago

I can't think of a worse place to be blind than the NYC subway, especially because even being non-blind is still quite horrible there.