If users encrypt their email from the client, there won't be anything meaningful to eavesdrop, will there? I figure there may be a problem with incoming emails

If users encrypt their email from the client, there won't be anything meaningful to eavesdrop, will there? I figure there may be a problem with incoming emails that are not encrypted, but surely people have to fix that on their own (e.g. telling their peers to encrypt emails sent to them). Is it unreasonable not to give up a key that I don't have?