4 ms·

You can still think of them as snapshots. Git just does compression on the entire folder of snapshots, including de-duplication of data that doesn't change betw

by goerz 6y ago

You can still think of them as snapshots. Git just does compression on the entire folder of snapshots, including de-duplication of data that doesn't change between snapshots. In fact, when I teach git to students, I don't even bother with the trees/blobs, which in my view are just an implementation detail. I just tell them to think of git zipping up their working directory together with some metadata (commit message, reference to parents), and putting that zip file into its own "compressed" storage inside the .git directory. That seems to be sufficient for a good mental model of how to work with git (independently of the git's somewhat baroque command line interface, which just takes getting used to)

gilbetron 6y ago
So it only stores the difference between the two snapshots? ;)
- dbt00 6y ago
  The implementation-specific compression doesn't store deltas or diffs, it stores unique blocks of text. Git allows for shallow clones, which would be impossible if the protocol or implementation were based solely around diffs.
- detaro 6y ago
  No. If it chooses to compress the commits, which it if I remember correctly not does automatically for each commit, but rather occasionally as a larger step, it uses the difference to whatever it deems to be a good candidate, if it finds one. E.g. if you have a file in commit A, change it massively in later commit B, and then on a different branch create commit C that also changes the file to one very similar to the one in B, git might very well compress C by storing the difference from B to C, despite those having no direct relationship in the commit graph. It can also choose to not use a delta to a different version entirely, and this is 100% an internal implementation detail of the storage system in git (afaik one of those implementation details is that it prefers candidates that are in the same commit chain, but it doesn't have to - and it can easily jump multiple commits if that works better). If you ask git to show you a diff to the previous commit, it does not pull a diff from storage, but pulls two file versions from its storage backend (which if deltas have been used to store will resolve those) and diffs them.
- planckscnst 6y ago
  No, it stores an entirely new set of references to objects, as well as some of those objects themselves (any that are not identical to previously stored objects). You cannot look at a commit on its own and know exactly how it's different from the previous commit, but you do have the complete new state. You have to look at the parent commit's references and do an object-by-object comparison to identify exact changes. On the other hand, when you look at a diff, you can see exactly what has changed, but you cannot produce the version that came before without also having a complete copy of the current version.
redisman 6y ago
I don't know that you need to teach them any of that. Version control is an abstraction. I have no clue what happens under the hood and I don't care.
- shuntress 6y ago
  To some extent, this is true. I don't feel the need to totally understand gits packing logic or the specific mechanics of the various diff/merge algorithms. But some knowledge of how/why your tools work the way they do can be very helpful. Some knowledge of a tools internal working can be fundamental to efficient use of that tool. At the very least it can allow you to understand or derive your useful interactions with that tool rather than simply memorize how it is used.
hmsimha 6y ago
This is the thing though. You're talking about snapshots which actually have duplication removed... in my mind this really fits more with the 'diff' model. I've already done the exploratory diving-into-git-internals thing years ago, so I could develop a better understanding of how things actually work. But for newcomers who want to understand how git is working, it really makes more sense to tell them it's 'like a diff. Not exactly under the hood, but think of it like a diff for now'. This is what I've been telling people as I've mentored a number of people in getting acquainted with git over the years, and if they're curious enough to look under the hood, they'll get a better understanding of the internals. As a programmer, what you're working with is essentially the diff. This is the easiest way to think about things initially. The fact that git is storing blobs under the hood, shallowly deduplicating blobs but still storing large chunks separately that may contain duplicate data, until it generates packfiles which do a deeper deduplication/compression, is really not that helpful. Telling people it's more like zipping is a bit disingenuous because it doesn't really explain how things are compressed more efficiently over the course of many changes. If I have a 1MB code file and make 1000 commits of one-line changes then sure, git is initially storing large blobs representing those, but then will compress over the change set when it generates the packfile. Compared to making a zip of the file for every change (say these are 100KB compressed) and now you have people thinking the 1000 one-line changes generate 100MB in the .git directory. You may think that a 1MB file with many smaller changes is a fabricated example, but consider that dependency lockfiles (package-lock.json I'm looking at you) can easily grow to this size, and contain this many changes.
- ako 6y ago
  Not really: if you do a checkout of a snapshot into an empty directory, you expect the entire state at the time of the snapshot, not just the diffs.
- goerz 6y ago
  It may depend on the background of who you're talking to. Programmers may be very comfortable with diffs, but non-programmers (in my case, physics graduate students) usually aren't. On the other hand, everybody is familiar with snapshots: even high school student will end up with "report_v1.docx", "report_v2.docx", etc, which are snapshots at the file level (and work reasonably well as long as you have a consistent scheme and don't need branches). I've also routinely seen less-technical people organize their research / paper writing by making a weekly snapshot of their work folder ("project-2020-04-1"). Telling these people that git basically does the same thing for them automatically with a tree-like "labeling scheme" that allows for branches tends to go over quite well, in my experience. For actually programmers, I'd be inclined to give them a more technical introduction to git's internals. I'd still point out that git stores compressed snapshots, not diffs (especially if they're older and may have previous SVN experience)

by goerz 6y ago

gilbetron 6y ago

So it only stores the difference between the two snapshots? ;)

dbt00 6y ago
The implementation-specific compression doesn't store deltas or diffs, it stores unique blocks of text. Git allows for shallow clones, which would be impossible if the protocol or implementation were based solely around diffs.
detaro 6y ago
No. If it chooses to compress the commits, which it if I remember correctly not does automatically for each commit, but rather occasionally as a larger step, it uses the difference to whatever it deems to be a good candidate, if it finds one. E.g. if you have a file in commit A, change it massively in later commit B, and then on a different branch create commit C that also changes the file to one very similar to the one in B, git might very well compress C by storing the difference from B to C, despite those having no direct relationship in the commit graph. It can also choose to not use a delta to a different version entirely, and this is 100% an internal implementation detail of the storage system in git (afaik one of those implementation details is that it prefers candidates that are in the same commit chain, but it doesn't have to - and it can easily jump multiple commits if that works better). If you ask git to show you a diff to the previous commit, it does not pull a diff from storage, but pulls two file versions from its storage backend (which if deltas have been used to store will resolve those) and diffs them.
planckscnst 6y ago
No, it stores an entirely new set of references to objects, as well as some of those objects themselves (any that are not identical to previously stored objects). You cannot look at a commit on its own and know exactly how it's different from the previous commit, but you do have the complete new state. You have to look at the parent commit's references and do an object-by-object comparison to identify exact changes. On the other hand, when you look at a diff, you can see exactly what has changed, but you cannot produce the version that came before without also having a complete copy of the current version.

redisman 6y ago

I don't know that you need to teach them any of that. Version control is an abstraction. I have no clue what happens under the hood and I don't care.

shuntress 6y ago
To some extent, this is true. I don't feel the need to totally understand gits packing logic or the specific mechanics of the various diff/merge algorithms. But some knowledge of how/why your tools work the way they do can be very helpful. Some knowledge of a tools internal working can be fundamental to efficient use of that tool. At the very least it can allow you to understand or derive your useful interactions with that tool rather than simply memorize how it is used.

hmsimha 6y ago

This is the thing though. You're talking about snapshots which actually have duplication removed... in my mind this really fits more with the 'diff' model. I've already done the exploratory diving-into-git-internals thing years ago, so I could develop a better understanding of how things actually work. But for newcomers who want to understand how git is working, it really makes more sense to tell them it's 'like a diff. Not exactly under the hood, but think of it like a diff for now'. This is what I've been telling people as I've mentored a number of people in getting acquainted with git over the years, and if they're curious enough to look under the hood, they'll get a better understanding of the internals. As a programmer, what you're working with is essentially the diff. This is the easiest way to think about things initially. The fact that git is storing blobs under the hood, shallowly deduplicating blobs but still storing large chunks separately that may contain duplicate data, until it generates packfiles which do a deeper deduplication/compression, is really not that helpful. Telling people it's more like zipping is a bit disingenuous because it doesn't really explain how things are compressed more efficiently over the course of many changes. If I have a 1MB code file and make 1000 commits of one-line changes then sure, git is initially storing large blobs representing those, but then will compress over the change set when it generates the packfile. Compared to making a zip of the file for every change (say these are 100KB compressed) and now you have people thinking the 1000 one-line changes generate 100MB in the .git directory. You may think that a 1MB file with many smaller changes is a fabricated example, but consider that dependency lockfiles (package-lock.json I'm looking at you) can easily grow to this size, and contain this many changes.

ako 6y ago
Not really: if you do a checkout of a snapshot into an empty directory, you expect the entire state at the time of the snapshot, not just the diffs.
goerz 6y ago
It may depend on the background of who you're talking to. Programmers may be very comfortable with diffs, but non-programmers (in my case, physics graduate students) usually aren't. On the other hand, everybody is familiar with snapshots: even high school student will end up with "report_v1.docx", "report_v2.docx", etc, which are snapshots at the file level (and work reasonably well as long as you have a consistent scheme and don't need branches). I've also routinely seen less-technical people organize their research / paper writing by making a weekly snapshot of their work folder ("project-2020-04-1"). Telling these people that git basically does the same thing for them automatically with a tree-like "labeling scheme" that allows for branches tends to go over quite well, in my experience. For actually programmers, I'd be inclined to give them a more technical introduction to git's internals. I'd still point out that git stores compressed snapshots, not diffs (especially if they're older and may have previous SVN experience)