Caching by tree hash instead of commit SHA makes a lot of sense for docs or non-code commits. How do you handle non-deterministic tests or environment variables

Caching by tree hash instead of commit SHA makes a lot of sense for docs or non-code commits. How do you handle non-deterministic tests or environment variables that aren't captured in the Git tree object?