The other issue is that it's not like they can go back and run this deduplication after the fact. Image layers are stored as a single gzipped tar of the content

The other issue is that it's not like they can go back and run this deduplication after the fact. Image layers are stored as a single gzipped tar of the contents on the layer. You can't just pull a single file out of that. If you go and reorganize as multiple gzip streams, you'll change the digest of the layer. A new registry could do that reorganization on import (and return the digest), or provide tooling to build the layers in the right format to begin with.