You can see some related but not directly comparable efforts in HuDI see https://eng.uber.com/uber-big-data-platform/ https://eng.uber.com/uber-big-data-platfor

You can see some related but not directly comparable efforts in HuDI see https://eng.uber.com/uber-big-data-platform/ https://eng.uber.com/uber-big-data-platform/ Or https://iceberg.apache.org/ https://iceberg.apache.org/ Which both keep track of data versioning and management of file based datasets on object stores. Most people today have very ad how approaches to handling data versioning and lineage on Hadoop datasets.