5 ms·
i clipped the 5 mins on apple intelligence here for those interested https://x.com/swyx/status/1833231875537850659 https://x.com/swyx/status/1833231875537850659
by swyx 2y ago
i clipped the 5 mins on apple intelligence here for those interested https://x.com/swyx/status/1833231875537850659 https://x.com/swyx/status/1833231875537850659
notes:
- photos/album search now includes video understanding, which imo seems very good from the first 2 examples they showed. includes scroll to exact time of the moment you describe.
- Mail and Notifications will show summaries instead of str[:x]
- Siri now knows iPhone, becomes the ultimate manual on how to use the increasingly complicated iOS 18. and can read your texts (!) to suggest actions with Personal Context Understanding (also it will try to advertise apple tv shows to you... i'm SURE it will be totally objective and aligned to your preferences amirite)
- new iphone 16 camera control button is PRIME real estate - notice how OpenAI/ChatGPT is now next to Google search, and both are secondary clicks to Apple's visual search, which comes first
- camera adds events to calendar!
- "all done on device" and on cloud (though craig doesnt say that haha)
overall i think insanely good ideas on ai + phone integrations.
- melenaboija 2y ago> overall i think insanely good ideas on ai + phone integrations. Mmmmm, feels disappointing to me if that is all we get from a multi trillion tech company + multi trillion technology
- lofaszvanitt 2y agoThe company needs S. Jobs to be resurrected.
- foreverobama 2y ago[dead]
- schmidtleonard 2y agoThis is good but... all I wanted for Christmas was anti-spam that didn't suck donkey balls. Sigh.
- callalex 2y agoThey don’t prevent spam abuse through their app push notifications, and don’t even offer a way to report bad behavior. They are pro-spam.
- swyx 2y agoi undersatnd i might come across as an apple shill/fanboi here but just pointing out that its fairly easy to punish any abusive app by killing their notifs and apple does sometimes make it easy to do that by letting u go straight to settings to kill the notif (i'm not conscious of when it does that but i think it does?)
- callalex 2y agoNo I can’t. Here’s an example: my security system app sends me push notifications when there is motion detected at times and places where there shouldn’t be motion. I need these critical (“time sensitive” in Apple-ese) notifications. It also helpfully sends me random “SUBSCRIBE TO SOME ADD-ON SERVICE, 50% OFF SALE THIS WEEK” at all hours of the day and night. Sure I could rip out my entire security system and replace it but that’s a lot of physical labor and wasted hardware. Another example is my air filter. It’s nice to get a reminder when it’s time to change the filter since it’s so infrequent, but I can’t get that notification without also getting spammed with “news” about new machines I should buy. Apple actively enables this behavior while claiming that spam is against their terms of service. They are clearly working for the spammers, not the user, because they profit off of their cut.
- samatman 2y agoThis can always be turned off using in-app settings. It's an App Store requirement. Note: in-app settings, not general settings for the app. Check. It's there. > Make sure people can manage their notification settings within your app. In addition to requesting permission to send informational or marketing notifications, you must also provide an in-app settings screen that lets people change their choice. For guidance, see Settings. https://developer.apple.com/design/human-interface-guidelines/managing-notifications https://developer.apple.com/design/human-interface-guideline...
- gffrd 2y agoDid you say spam?? Boy have we got the thing for you! Take a picture of a restaurant? Hey, did you know it's a place you can make a reservation at? Your friend mentions a song offhand? Psst … wanna play it now? Don't forget about how your cousin talked about that AppleTV+ series!
- gffrd 2y agoJoking aside, I'm very curious how (or if) this gets implemented in a way that doesn't isn't suffocating.
- schmidtleonard 2y agoI quite like AppleTV's content choices, but I am not looking forward to getting shilled on it. Again. In between the other spam that they don't block. Ahhhhhg! ("Get an Android?" Yeah, I hear you, I switch ecosystems in anger with every phone, which is why I am so keenly aware of how badly Apple's anti-spam sucks -- but also why I am not over the moon at the prospect of my next switch: I know where the Android skeletons are buried, although hope springs eternal.)
- nashashmi 2y agonow you will get spam that AI can understand.
- thih9 2y ago> "all done on device" and on cloud The real indicator for me is: how many of these features are not available in the EU due to privacy concerns. Do we know that?
- bengale 2y agoIt’s all Apple Intelligence and that’s not coming to the EU currently.
- a13n 2y agolots of the US has similar legislation to GDPR at this point, e.g CCPA
- eptcyka 2y agoPrivacy concerns? They're just trying to bully the EU. There's no legal or technical reason iOS 18's remote access features wouldn't work in EU, but alas, they do not.
- JumpCrisscross 2y ago> no legal or technical reason iOS 18's remote access features wouldn't work in EU How do you know that? (How much of your net worth are you willing to bet on it?) Between GDPR, the DMA and now the AI Act, there is a lot of unique regulatory cross section that Apple has in the EU. It would be surprising if there weren't additional legal checks it had to do, internally and on its suppliers, before launching a product or feature in the EU.
- boringg 2y agoThe part I am most interested in is how much of it is on device and how much of your data is trained only on your own model vs hovered up into a larger model. On device, my own model. Great.
- DidYaWipe 2y agoTotally. I am down for decreased dependence on connectivity and remote servers.
- panarky 2y ago"Insanely good ideas"? With the months of nonstop and over-the-top hype of "Apple Intelligence" these all seem underwhelming. Even in the idealized world of demoware there's no killer feature, not to mention that none of it is even available to real customers yet. I was expecting something only Apple could do with their vast ecosystem of IP and partnerships, the must-have cool new thing that would stimulate a leap in demand for iPhone. It's hard to see how any of this is enough to trigger a new upgrade cycle instead of just waiting until battery life goes to shit or the screen cracks so you have to upgrade.
- teaearlgraycold 2y agoSemantic search was always going to be the killer feature of this AI boom. But it’s just a cherry on top of existing software and requires training users to expand what they think is possible.
- wwweston 2y agoAs far as I can tell, AI is one of several features that have actively made search worse. There are a handful of novel things it does well enough, but I'd rather engage those in their own LLM sandbox than have the facilities formerly available in a searchbox replaced with a guessbox.
- teaearlgraycold 2y agoWhat kind of search has it made worse? I’m not talking about Google search. I’m talking about searching your personal collection of emails or photos or text messages.
- jimbokun 2y agoI would guess is you're probably in the minority with that opinion.
- swyx 2y ago1) i am as ready to shit on apple and siri as the next guy, but imo the HN cynicism is overbearing here. apple isnt building to impress internet neckbeards like us. they are building for the normies. features they understand intuitively and cannot fuck up. maybe wait til it gets in the normies hands to judge. ~none of us here know what its like to work on a hardware/software platform for >1 billion? people. 2) i actually do think dedicated visual intelligence button IS a killer feature. suddenly phone is ai's view into the world. yes probably v1 today will disappoint. but 5 years from now the kids will laugh when we say we had to take photos and upload them to an app back in our day. or manually enter in any information from the real world into our calendars and emails and texts.
- llm_nerd 2y ago"- camera adds events to calendar!" I'm on iOS 18.0 on my iPhone 15 Pro, rather that the 18.1 with the "AI". Recently had an eye doctor appt for one of my children and they scheduled a follow-up, giving me a little card with the details. I was in a hurry so took a picture of it intending to enter the calendar event later, and it immediately pulled up a fully-filled in, completely accurate new calendar event. Little affordance that is just so nice. Indeed, in general the camera is getting...crazy. We have a pair of Maine Coon kittens and every picture is tagged with Maine Coon, despite the cats barely even coming into the breed's features yet. Constantly amazed at the stuff the built in camera is identifying.
- swyx 2y agooh so you're saying this is not an Apple Intelligence feature? its a little hard to tell
- m_mueller 2y agoIt‘s Apple Intelligent, not Apple Intelligence! When are you finally getting it! And no, we don‘t abbreviate it ever! Look, a new button! runs away
- llm_nerd 2y agoThey have their big generative AI push with 18.1, broadly called Apple Intelligence, but they've been building so many AI inference features for a few iterations now, with the camera and Photos getting loads of those additions. Object detection, subject identification, text extraction, etc, though this was the first time I've taken a picture of an appointment card and had it automate extraction like this. I feel like loads of these features just quietly appear to be randomly discovered by users. So it's AI, but not the big hype stuff that gets most of the attention.
- pants2 2y agoI have a hilarious list of animals that my cat has been misclassified as by my iPhone. Everything from a beetle to a nematode. It doesn't give me a lot of confidence in these features!
- laweijfmvo 2y agoAnything that's going to recommend Apple TV shows (or any other product) to me is an automatic turn-off. I'll keep all of Siri off if I have to.
- 015a 2y agoI have this sinking suspicion that the AI segment today was so quick, so high level, and so future-tense ("Siri will be able to..."), identical to the similar segment at WWDC; I have to wonder how many of these features we'll actually see. I'm getting vibes like the old "Facetime will be an open protocol" promise; its like someone is making them do this, their soul isn't in it, and for that reason they still don't have one narrative for why I should care. Instead, they have a quickfire list of twenty features, the titles of twenty epics from their jira. The iPhone 16's camera button will gain the ability to soft-press to lock focus "later this year". Their software teams are so incredibly far behind, the new phone has literally one new feature, a feature medically prescribed to induce drowsiness, yet they can't even get the software for it finished for release.
- zombiwoof 2y agoI think Apple will be the first to realize nobody wants to pay for “ai features”, and they are hella expensive to support
- 015a 2y agoApple will be one of the least impacted once the market corrects to this reality, because they can push off the cost of the inference silicon to customers for many many queries (especially given 3-4 years, as models become more efficient and Apple Silicon becomes more powerful).
- deleted 2y ago[deleted]
- baxuz 2y agoAs someone whose mother tongue isn't English, and whose residence isn't the US, apple intelligence is basically useless to me. Like every other ai assistant gimmick out there.
- swyx 2y ago[flagged]
- robteix 2y agoHonest question: why though? If someone uses a language other than English on their phone, the feature is quite literally useless to them, isn't it? I don't get why they're "self centered" for saying so. I think it's quite telling that one of the main new features of the new iPhones won't even be available until next year for most of the world.
- swyx 2y agobecause obviously its coming and its not a huge technical lift at all. transformers were born to do translation. its just impatient and a waste of space complaining about how we know the world works for already very good if unfair reasons.
- pndy 2y agoHear hear; I'm using Siri to set timers, turn off alarms - everything else without a proper support for my native language is useless. It's been nearly 12 years and the supported language list is still the same.
- jiggawatts 2y agoThe voice and text AIs such as Whisper and ChatGPT are generally multi-lingual, albeit their quality with anything other than English is not the same. In theory at least it should be straightforward to make LLM-based assistants like Siri truly multilingual.
- miguelrochefort 2y agoI predicted those features weeks ago [1], but I still feel they could do so much more. If anything, their implementation feels like a rushed afterthought. In their demos, they use the action button to capture an ambient song for Shazam, the power button to capture a voice command for Siri, and the camera button for an image for Visual Intelligence. All 3 captures should be performed using the same button. And screenshots still require pressing 2 buttons simultaneously. Unless you want to share your screen with Siri in which case it's the power button... People are going to use this button as a voice recorder, and Apple will announce native support next year. [1] https://miguelrochefort.com/blog/capture-button https://miguelrochefort.com/blog/capture-button
- xavdid 2y ago> Mail and Notifications will show summaries instead of str[:x] While it's true that the first x characters of an email don't always have the most important bits, they're never _wrong_. It's always exactly what's in the email. I'm worried about cases like this [0], where a summary is inaccurate or misleading. It may not happen often, but it's possible, so I have to be a little skeptical of every summary. Though I can see where AI summaries are a plus, the move from 100% accurate to <100% feels like a big downgrade. [0]: https://mastodon.macstories.net/@johnvoorhees/112887004822171579 https://mastodon.macstories.net/@johnvoorhees/11288700482217...
- data-ottawa 2y agoI'm curious how Apple Intelligence will manage Duolingo's progressively worried sounding notifications through the day. It feels like an endpoint that will get gamed by apps over time.
- willis936 2y ago>can read your texts Anti-feature. I don't even want my phone listening to me and prompting some awkward speech search engine I didn't ask for.
- sureIy 2y ago> - camera adds events to calendar! The state of the camera app pisses me off. It could do so much but it does so little. Why can’t I snap a picture of a written WiFi password to connect to WiFi? QRs are so 2008. Why can’t it fully parse a menu and show me what each dish might look like? It’s mind-boggling to me that with thousands of engineers on staff they produce so relatively little. Thousands!
- kemayo 2y ago> Why can’t I snap a picture of a written WiFi password to connect to WiFi? You can actually do that, as of last year's iOS release. It's sort of a pain, but it's doable. You need to be at the point where it's asking you for the WiFi password, tap the text field, choose "AutoFill", choose "scan text", and use the camera interface to pick the text. This is extremely undiscoverable, no argument, but it is at least a system-wide standard for how to scan in arbitrary text when you have a text input. (They've picked "AutoFill" as the brand-name for all their get-text-from-somewhere-else features, so it also contains standard password-filling from apps.)
- whywhywhywhy 2y ago> insanely good ideas Bafflingly underwhelming additions from the previous innovation leader who had a last mover advantage in the space. Think we need to be holding Apple to a higher standard, what they did in spaces before with the OG iPhone, iPad, iPod was completely upend the industry and change the game through focusing a product on a few pain points of the existing market. Yet Apple Intelligence feels like a brief struggling to find a reason to be there.