This is partly an issue with phone calls as a medium. If the bank only contacted you through the app, this couldn't happen unless the app itself was compromised

This is partly an issue with phone calls as a medium. If the bank only contacted you through the app, this couldn't happen unless the app itself was compromised somehow. E.g., the Monzo app has a chat functionality built in. If, upon a fraud attempt, a notification appeared in the Monzo app, it would certainly be legitimate. If the ease of conversation offered by chatting with voice is necessary, add a link in the chat that has the user call the bank, not the other way around. You can't easily verify that someone is who they say they are over the phone.