4 ms·
The workflow I would have imagined for versioning data is: 1) Load original data from source into quilt 2) Do transformation 3) Commit transformations to qui
by wodenokoto 9y ago
The workflow I would have imagined for versioning data is:
1) Load original data from source into quilt
2) Do transformation
3) Commit transformations to quilt, with commit message
4) Run experiment
5) Do new transformations
6) Commit to quilt
7) Run experiment
Rinse and repeat.
Looking at the video and documentations, this is not emphasised at all, suggesting that edits to data should be saved as a new package.
- kevinemoore 9y agoWe definitely imagine saving edits as creating new versions of the same package. The most common pattern we've heard is adding more dataframes or files to a package as new results are generated. But, we can certainly imagine other transformations.
- akarve 9y agoYou can absolutely edit in place and that will go into `quilt log` for the package--as long as you are the package owner. Our docs were a bit confusing on this point. I just updated them: https://docs.quiltdata.com/edit-a-package.html https://docs.quiltdata.com/edit-a-package.html