Have you considered content-dependent chunking for your deduplication? At Ronomon, we use https://github.com/ronomon/deduplication https://github.com/ronomon/d

Have you considered content-dependent chunking for your deduplication? At Ronomon, we use https://github.com/ronomon/deduplication https://github.com/ronomon/deduplication