Have you tested Weights & Biases Artifacts[1]? It comes with a smart versioning approach, checks the Δ based on the checksum and has a feature to visualize the

Have you tested Weights & Biases Artifacts[1]? It comes with a smart versioning approach, checks the Δ based on the checksum and has a feature to visualize the lineage. You can also use your existing object store and link it for very large / sensitive data.[2] Disclaimer: I work at W&B. [1]: https://docs.wandb.ai/guides/data-and-model-versioning/model-versioning https://docs.wandb.ai/guides/data-and-model-versioning/model... [2]: https://docs.wandb.ai/guides/artifacts/track-external-files#amazon-s3-gcs-references https://docs.wandb.ai/guides/artifacts/track-external-files#...