I have been circumventing this issue purely by accident it seems. I have always had ssh-keygen write out the keys to a custom path like ~/.ssh/keys/$service/$k

I have been circumventing this issue purely by accident it seems. I have always had ssh-keygen write out the keys to a custom path like ~/.ssh/keys/$service/$key{,.pub} and configured each host's "identity" manually with the ssh config file. I never did this for security purposes or anything, I just thought it made more sense than the default behavior.