4 ms·
No, data versioning typically does not solve this problem unless you are completely isolated from everything else. Consider the case where you're pulling data
by woeirua 7y ago
No, data versioning typically does not solve this problem unless you are completely isolated from everything else. Consider the case where you're pulling data from a database to use for training a model. Typically DS teams have no control over the backup schedule for that database, how long those backups are maintained, etc. For large systems, if you try to restore a backup 3 or 6 months down the road, you may only have a weekly snapshot (if even that frequent) to use. Certainly that won't give you the same data that you originally had. Admittedly, a team that was strongly focused on reproducibility could try really hard to ensure that their queries are reproducible, but if you want to be productive you typically delegate most of that work to other libraries which may or may not generate reproducible queries.
It seems to me that people working on really large-scale problems always have these external dependencies that they just can't control and subsequently data versioning only goes so far when you cannot make a full-copy of the data for each version.
I agree with you in general that we should strive for reproducibility.
- lewq 7y agoHmm interesting, what if you had database snapshots that could be driven by ML workflows? The primitive behind Dotscience can support lightweight database snapshots quite easily...
- lewq 7y agoTo be clear, that primitive is dotmesh and ZFS - see https://dotscience.com/technology/ https://dotscience.com/technology/
- deleted 7y ago[deleted]