8 ms·
Somewhat agreed, but I don’t think this demonstrates that kind of failure . If the assumption is that the earlier system is responsible for rejecting spam, the
by dagmx 2y ago
Somewhat agreed, but I don’t think this demonstrates that kind of failure .
If the assumption is that the earlier system is responsible for rejecting spam, then I think it’s reasonable for the AI part to trust the email.
To your point, it should perhaps protect against text that would abuse the user. But in this case the text is very similar to official emails so wouldn’t be distinguishable to an LLM.
I think you’d need a more complex failure case to show the AI bit was failing or succeeding.
- acdha 2y agoI don’t think it’s reasonable to trust the mail server to have perfect filters. Systems like this have to be designed assuming realistic error rates and failures, and part of that has to be asking questions like whether the resulting UI will lend an attacker more credibility than they would have or whether the tool should be surfacing information which people miss such as telling you that you’ve never received emails from that domain before and your normal M365 interactions use Microsoft.com.