5 ms·
Apple's new AI feature rewords scam messages to make them look more legit
- qintl55 2y agoafter years of teach humans to identify and avoid scams, we now have to teach AI to do this. le sigh
- whatever1 2y agoIs it that bad because it is a smaller model compared to the sota?
- phire 2y agoIt's not related to the size of the model. It's more of a prompting/design issue. The LLM has been told to summarize the message, not to identify it as a scam or not.
- lilyball 2y agoNor should it. Scam identification is a feature of junk mail filtering, not of summarization. Any scam that's being summarized is one that has already made it past junk filtering.
- EForEndeavour 2y agoI'd expect a suite of features explicitly designed to mimic human intelligence to know, like a dutiful assistant, that if I give them access to my inbox and ask them to "summarize" my new messages, and they see a scam message, they notice and delete it rather than pass it through to me like an idiot. We're supposed to be moving away from the era of computers dumbly following explicit instructions and into the era where "AI" finally delivers on the hollow promises of Siri and Alexa.
- deleted 2y ago[deleted]
- phire 2y agoAnd that's a problem with human expectations not lining up with the reality of LLMs. You could actually implement such functionality with current LLMs. Even one small enough to run on a phone. But you can't implement it well enough to be trustworthy. It will make mistakes, and people will quickly stop trusting it. Even if it was as good as a true human assistant (which it's not), humans still make mistakes. We have a tendency to be forgiving of the mistakes that humans make, but we expecit AI to be near perfect and will judge it far more harshly... Hell, just look at how people are blaming it for summarizing a phishing scam. This is before you even consider the potential of prompt injection attacks. If you give the LLM the power to delete emails, it will be vulnerable to people sending emails telling it to delete emails. A job applicant might be able to tell apple intelligence to delete all other applications.
- somebehemoth 2y agoAs it made it past other filtering it is more dangerous. The last possible protection is the user's attention. If the user reads a coherent summary and then opens the message, might they be more primed to treat the message as legitimate? I agree with you about other filtering being critical and more appropriate. I understand Apple made a good feature. I don't actually know if it will have a negative side effect. I do think it is a good idea to examine the feature critically due to the scale of Apple and the trust its users place in the company to protect them from harm. I also do not think the title of this article is very reasonable bc it insinuates something sinister.
- deleted 2y ago[deleted]
- rubslopes 2y agoThis year's marketing trend: crafting copy in a way that it is prioritized by Apple Intelligence.
- kcplate 2y agoI’ve been saying over the last year to our marketing team that we should put some effort into better understanding how AI reads and understands our website copy and email copy so we can optimize there.
- pingers123 2y agothis year's scamming trend: crafting scams in a way that manipulates Apple Intelligence
- lilyball 2y agoIf you enable a feature that summarizes messages, then messages get summarized. Summarizing messages requires rewriting them. Where is the surprise here? Not only that, but the moment you click on the summary, you see the whole thing. Whether that's clicking the email summary to view the email, or clicking a summarized notification stack to expand it and see the notifications. The summary doesn't replace the message, it just lets you know whether it's something you want to click on.
- shard972 2y ago[dead]
- miohtama 2y agoThe surprise is in how to make clickbait titles that sell on the AI fear
- lilyball 2y agoYeah, to be honest, the only reason I wrote the comment was because my understanding is HN stories whose comments/points ratio is higher get downranked and I'm tired of seeing clickbait artificial outrage stories.
- koolala 2y agoA human summary would say " This smells like BS". This article though smells legit. This is an actual problem that it doesn't seem intelligent enough to handle.
- gabeio 2y agoVery few if any of the AIs have reasoning yet. Yes, a human would be able to think through and tell you it’s spam or better yet not even bother telling you about the message _because_ it is spam but we aren’t quite there yet.
- 2y ago
- ytch 2y agoIt's just garbage in, garbage out. AI isn't the safeguard of everything.
- jaredsohn 2y agoSurprised to see no mention of how this impacts people who purposefully write scam messages poorly to filter out people are less likely to get conned. Curious if this starts making traditional scammers less efficient.
- the_snooze 2y agoI feel like the whole "notification summary" use case addresses the wrong problem. It's true that a lot of people are innundated in notifications of varying levels of importance. But I don't think the solution is to condense those diverse notifications and lose a lot of context in the process. A better solution is to reduce notifications altogether, so that the problem remains human-scaled and leaves the user in charge. Maybe an AI (or even a simple statistical model) can suggest group chats to suppress notifications for, based on how frequently the user actually reads and engages with them. Maybe the notifications system can be overhauled altogether so that non-DM uses are severely restricted by default (e.g., a news app can only display one notification at a time). The problem with too many notifications is too many notifications, not the user wishing they had the motivation to read all of them.
- ninkendo 2y agoI think I basically agree. My policy is basically: people can contact me, computers can’t. The only notifications I allow are for apps that are a proxy for human communication: messages, email (with a heavily curated inbox in which I dutifully unsubscribe from everything that’s not something I want), and very little else. The only time I want a notification is if some human being thought to themself “I need to talk to ninkendo right now, and I want him to be notified”. No automated systems, no mass communications. I’ve never found any system to be of any help with this other than just disabling notifications for every single thing other than these 2-3 apps. No “focus modes”, no auto-summaries, etc. None of them are as effective as just not having the notifications in the first place.
- 1123581321 2y agoFocus modes are still nice for letting different groups of people contact you, though.
- cyberge99 2y agoEvery AI would do this if you asked it to.
- sinuhe69 2y agoThey aim for the low-hanging fruits like summarizing, thinking it could not go wrong. Oh boys, how wrong they could be! If they aim high like Jobs always war, Apple Intelligence could actually provide values instead of being an annoyance.
- cadamsau 2y agoSoooo it does what it’s supposed to?
- egberts1 2y agoOriginal: "my dear John Smith. My name is Prince Abiodun Adosina. A fortune is awaiting you but it’s locked in an overseas account" Apple AI: "Dear John Smith, Prince Abiodum Adosina is me. A significant amount of my overseas saving stands to flow into your bank account. All that is required of you to unlock your portion toward you is to ..."
- _boffin_ 2y agoHas anyone extracted the promos for the message summary feature yet?
- bobheadmaker 2y agoWell, that is some good use case they found :P
- Mathnerd314 2y agoIIRC there was something about how the scammers don't bother making their scams that legitimate looking, because if they are too legit, they get a lot of people who waste the scammer's time during the next phases. But yeah, you'd think spam filtering would be more important. I use GMail and I haven't seen a spam message in years, besides when I check my spam folder. Even there, most of the "spam" is false positives.