14 ms·

I have a technical question that I'm not at all poised to answer, that might be stupid like all questions not in one's domain: I recently discovered the joy th

I have a technical question that I'm not at all poised to answer, that might be stupid like all questions not in one's domain: I recently discovered the joy that is ZFS and everything that comes with it. I understand that the technical underpinnings of git are actually extremely different (and mathematical) _but_ just how far is a ZFS snapshot from a git commit really? It seems like the gap between the two might not need a huge bridge. Could a copy-on-write filesystem benefit from more metadata that would come from being implemented in a more git-like way?

deepspace 5y ago
I think you are correct. It would not be a huge stretch to turn a snapshotting file system into a VCS. https://en.wikipedia.org/wiki/Versioning_file_system https://en.wikipedia.org/wiki/Versioning_file_system The IBM/Rational Clearcase version control system is an example of building a VCS on top of a versioning file system (MVFS), though MVFS uses an underlying database instead of a copy-on-write snapshot mechanism. https://www.ibm.com/support/pages/about-multiversion-file-system-mvfs https://www.ibm.com/support/pages/about-multiversion-file-sy...
gmueckl 5y ago
Conceptually, the two things are very much related and a birds-eye view shows a lot of similarities. But when you get into the weeds, there are some significant differences. git is optimized to store a great many historic states of files with minor differences between consecutive ones, and it assumes that these are essentially static, immutable snapshots. A COW file system that allows for snapshots is optimized more for allowing mutation of these snapshots (i.e. updating files one way in one snapshot and another way in another one). This, combined with the additional housekeeping required for a file system (disk block allocation, etc. - the actual core features) makes the implementations of the two things very different.
crubier 5y ago
Very close. Actually companies such as https://postgres.ai/ https://postgres.ai/ use ZFS storage to provide git-like features on top of Postgres: Using copy-on-write on the underlying ZFS, you can "fork" a new branch of your DB with all the data, instantly. Then both branches can live their lives independently. But I don't think ZFS has the equivalent of git merge though.
- shakow 5y ago
  Naive question, but what is the advantage compared to a classical DB dump? Faster?
  - withinboredom 5y ago
    I think the keyword was "instantly."
    - qwertox 5y ago
      Is it correct that then the original DB and the snapshotted DB share those blocks on the file system which are unmodified? Assume 1 row per block: Original DB "A" has 2 rows, a snapshot "B" is created, "B" deletes one row and adds a new one. Is it true that the row which "B" took over from "A" and left unmodified resides on the same block for "A" and "B", so that if the block gets corrupted, both databases will have to deal with that corrupt row?
      - Dylan16807 5y ago
        Yes, that's one of the core parts of copy-on-write. It shouldn't matter if you have a reasonable setup. If you depend on other files on the drive to continue working after blocks have started to go corrupt, that's not a good system.
  - crubier 5y ago
    Yes, faster. In my experience it took less a couple seconds at most for a copy of our production database, while a dump took several hours.
- breakingcups 5y ago
  I've been doing this, but with BTRFS as a trial. Is there anyone who can suggest a benefit of using ZFS over BTRFS or the other way around?
  - iheartdata 5y ago
mattnewton 5y ago
At Google, people have built both, and we use a version control system on top of a snapshoting filesytem. The snapshotting is for never losing code/state on your machines, and the version control system is for interfacing with others (code review, merging, etc). While you could use one system for both, having both layered on top makes it easier to change them to each specific workflow.
jhoechtl 5y ago
Came here just to mention Btrfs which does the same as ZFS in the sense that it is also COW by default.
deleted 5y ago
[deleted]
Dylan16807 5y ago
It would be nice if ZFS snapshots were more flexible. And you could say "like git" when talking about the user experience. But it would not be like git in terms of implementation. Git's implementation is not really copy-on-write. It's deduplication. I'd say the git method is actually pretty low in metadata, and the way you'd improve ZFS snapshots doesn't involve making them more like git. If you did get that huge amount of work done, you could then approximate git with snapshots alone. Right now, you'd probably want snapshots and dedup to work together to approximate git using ZFS.
- 0x000000001 5y ago
  doesn't the zfs diff command mostly cover it? You have snapshots, diffs, and clones which is basically equivalent to commits, diffs, and branches. You're missing commit logs and that's it, right?
  - Dylan16807 5y ago
    If you introduce a new file onto multiple ZFS 'branches', there is no way to have it stored copy-on-write. But copying a file from one 'branch' to another is the only way to emulate a cherrypick or merge. So after a while of active use with many branches, you're going to have a lot of redundant copies of files all over. You're no longer making proper use of snapshots, and it becomes less efficient than having a working directory and a directory full of commits that hard link to each other.
    - 0x000000001 5y ago
      Good point, but I bet this could be changed in ZFS.

deepspace 5y ago

I think you are correct. It would not be a huge stretch to turn a snapshotting file system into a VCS. https://en.wikipedia.org/wiki/Versioning_file_system https://en.wikipedia.org/wiki/Versioning_file_system The IBM/Rational Clearcase version control system is an example of building a VCS on top of a versioning file system (MVFS), though MVFS uses an underlying database instead of a copy-on-write snapshot mechanism. https://www.ibm.com/support/pages/about-multiversion-file-system-mvfs https://www.ibm.com/support/pages/about-multiversion-file-sy...

gmueckl 5y ago

Conceptually, the two things are very much related and a birds-eye view shows a lot of similarities. But when you get into the weeds, there are some significant differences. git is optimized to store a great many historic states of files with minor differences between consecutive ones, and it assumes that these are essentially static, immutable snapshots. A COW file system that allows for snapshots is optimized more for allowing mutation of these snapshots (i.e. updating files one way in one snapshot and another way in another one). This, combined with the additional housekeeping required for a file system (disk block allocation, etc. - the actual core features) makes the implementations of the two things very different.

crubier 5y ago

Very close. Actually companies such as https://postgres.ai/ https://postgres.ai/ use ZFS storage to provide git-like features on top of Postgres: Using copy-on-write on the underlying ZFS, you can "fork" a new branch of your DB with all the data, instantly. Then both branches can live their lives independently. But I don't think ZFS has the equivalent of git merge though.

shakow 5y ago
Naive question, but what is the advantage compared to a classical DB dump? Faster?
- withinboredom 5y ago
  I think the keyword was "instantly."
  - qwertox 5y ago
    Is it correct that then the original DB and the snapshotted DB share those blocks on the file system which are unmodified? Assume 1 row per block: Original DB "A" has 2 rows, a snapshot "B" is created, "B" deletes one row and adds a new one. Is it true that the row which "B" took over from "A" and left unmodified resides on the same block for "A" and "B", so that if the block gets corrupted, both databases will have to deal with that corrupt row?
    - Dylan16807 5y ago
      Yes, that's one of the core parts of copy-on-write. It shouldn't matter if you have a reasonable setup. If you depend on other files on the drive to continue working after blocks have started to go corrupt, that's not a good system.
- crubier 5y ago
  Yes, faster. In my experience it took less a couple seconds at most for a copy of our production database, while a dump took several hours.
breakingcups 5y ago
I've been doing this, but with BTRFS as a trial. Is there anyone who can suggest a benefit of using ZFS over BTRFS or the other way around?
- iheartdata 5y ago

mattnewton 5y ago

At Google, people have built both, and we use a version control system on top of a snapshoting filesytem. The snapshotting is for never losing code/state on your machines, and the version control system is for interfacing with others (code review, merging, etc). While you could use one system for both, having both layered on top makes it easier to change them to each specific workflow.

jhoechtl 5y ago

Came here just to mention Btrfs which does the same as ZFS in the sense that it is also COW by default.

deleted 5y ago

[deleted]

Dylan16807 5y ago

It would be nice if ZFS snapshots were more flexible. And you could say "like git" when talking about the user experience. But it would not be like git in terms of implementation. Git's implementation is not really copy-on-write. It's deduplication. I'd say the git method is actually pretty low in metadata, and the way you'd improve ZFS snapshots doesn't involve making them more like git. If you did get that huge amount of work done, you could then approximate git with snapshots alone. Right now, you'd probably want snapshots and dedup to work together to approximate git using ZFS.

0x000000001 5y ago
doesn't the zfs diff command mostly cover it? You have snapshots, diffs, and clones which is basically equivalent to commits, diffs, and branches. You're missing commit logs and that's it, right?
- Dylan16807 5y ago
  If you introduce a new file onto multiple ZFS 'branches', there is no way to have it stored copy-on-write. But copying a file from one 'branch' to another is the only way to emulate a cherrypick or merge. So after a while of active use with many branches, you're going to have a lot of redundant copies of files all over. You're no longer making proper use of snapshots, and it becomes less efficient than having a working directory and a directory full of commits that hard link to each other.
  - 0x000000001 5y ago
    Good point, but I bet this could be changed in ZFS.