Ask HN: Language-level package management with docker?

I'm a python user 6 months into using docker heavily and I'm experiencing impedance mismatch between pip / wheel and docker's cache. Are there non-Dockerfile options for creating docker images that can optimize for requirements.txt? Alternatively -- is there a way to apply layers out of order when they don't touch the same files?