If you're reading files rather than writing them, then you must be decrypting them and in possession of the private key, which means you are permitted access to

If you're reading files rather than writing them, then you must be decrypting them and in possession of the private key, which means you are permitted access to everything; so you could cache all the decryptions. Initial reads might be more expensive to locate, but then free. Also, how many cycles could it take to decrypt the few hundred or few thousand characters that could possibly make up a full filepath?