Absolutely, but there are various attack vectors that different mitigations are effective against. The program doesn’t even need to be malicious, for a while i

Absolutely, but there are various attack vectors that different mitigations are effective against. The program doesn’t even need to be malicious, for a while it was a pretty common attack vector to trick browsers into uploading random file you could access. Later, a malicious ssh server could read memory of the ssh process, potentially exposing the private key (CVE-2016-0777) Using an agent with an encrypted key protects against that. Using a yubikey/smartcard as well. So it’s strictly a good thing to use it. A yubikey could potentially protect you against a malicious program that wants to open connections if you have set it up to confirm every key operation - but that comes at a cost. You could also use little snitch to see what network connections a program opens, protecting you against a program trying to use your agent to access a server.