I'm one of the creators of Pachyderm (github.com/pachyderm/pachyderm) that might be able to help. We do require an object store of some kind, but if you dont h

I'm one of the creators of Pachyderm (github.com/pachyderm/pachyderm) that might be able to help. We do require an object store of some kind, but if you dont have s3 you can always use Minio in front of whatever storage you do have. Pachyderm will let you mount that data locally in a jupyter notebook so you dont need to constantly copy it around. As many people as you want can do this and there will only ever be one true copy of the data in the centralized system. That central copy also includes version control for your data set so you can make changes to it and get deduplication of the files while still maintaining all your data lineage info.