5 ms·
Git is the leakiest abstraction in the history of abstractions. Diffs are a "natural" object for version control yet got doesn't actually use them and as we ca
by oblio 6y ago
Git is the leakiest abstraction in the history of abstractions.
Diffs are a "natural" object for version control yet got doesn't actually use them and as we can see in this article, multiple git commands leak this implementation detail.
- chubot 6y agoI'd argue that diffs in the presence of renames aren't natural, and git handles them better than other version control systems. That is, git is content-based and not named-based. Name-based systems explicitly track renames with metadata; git does not. git calculates renames dynamically based on content.
- masklinn 6y ago> I'd argue that diffs in the presence of renames aren't natural That is true, at least for the text-based diff(1) formats, though obviously nothing precludes extending the format, or even using something else entirely (aside from the risk of not being compatible with diff(1), but then you could always present diff(1) compatible diffs externally and use something richer internally, for instance). > git handles them better than other version control systems. But that is not. > That is, git is content-based and not named-based. Name-based systems explicitly track renames with metadata; git does not. git calculates renames dynamically based on content. And as a result regularly fucks up tracking renames, or plain refuses to do it without nonsensical workarounds if you copied or moved a file and modified it in the same commit. It also makes browsing histories, blaming, or tracking merges through renames incredibly expensive.
- chubot 6y agoThat could be all be true, but IMO git's handling is still better than the alternatives I've used (Perforce, mercurial, subversion, CVS). (Although honestly I never really have problems with it messing up. If you approximately separate renaming from editing, it seems to work very reliably for me.) There's also nothing stopping anyone from writing a git history browser that caches the calculation. The point is that it's ephemeral / derived state, not authoritative state. One problem with the name-based systems is that developers don't actually use the VCS rename operation. They just delete and add, and then you've lost the diff under every system I know of. Another problem is importing from other VCSes -- the metadata is often messed up subtlely. Git keeps it simple. And they can and have improved the algorithm over time without making your repo data obsolete / incompatible.
- tsimionescu 6y agoGit rename tracking is better than P4 rename tracking with colleagues that don't use Rename. But if people are disciplined and DO use Rename/Move (and Copy), then P4 is much nicer to use. In particular, since many languages require file contents to be change depending on dir structure, Git often forces you to commit invalid files just to try to ensure that it will pick up the changes. With P4, you can move the files around, adjust everything until it compiles, and only then start telling P4 about what you actually did (you can even start from Reconcile Offline Work, which will try to use exactly Git's logic to identify moved/renamed/copied files based on content - but only one time, on your own machine, not every time someone looks at the history).
- im3w1l 6y agoYou could have the commit command do rename-detection and store the result.
- layer8 6y ago> One problem with the name-based systems is that developers don't actually use the VCS rename operation. That’s a matter of tooling. In languages and environments where using an IDE is the norm, use of VCS operations happens automatically, simply by virtue of performing all source file operations through the IDE.
- pmeunier 6y ago> I'd argue that diffs in the presence of renames aren't natural, and git handles them better than other version control systems. Maybe this is because I designed that part in Pijul, but I disagree. Pijul has renames commute with other types of edits, and renames are faithfully modeled in Pijul patches, not just "guessed" after the fact based on contents.
- smcameron 6y agoAnd good thing it's leaked, as anyone who's ever maintained, say linux kernel drivers for multiple distros knows. Easy to maintain a stack of commits on Linus's kernel and patch them into the various distro kernels. Well, for some value of "easy". But it would be a hell of a lot harder if you didn't have access to diffs and what I think of as "patch arithmetic." (i.e. source - patch3 + patch1 + patch2 == source + patch2 + patch1 - patch3). Patches are commutative (barring conflicts).
- gnufx 6y agoThat's a significant bar, which doesn't apply to Darcs and Pijul.
- pmeunier 6y ago> Patches are commutative (barring conflicts). That isn't quite true, even without conflicts. The same example that shows that Git merge is not associative (https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html) can be used to show that it also isn't commutative.
- epistasis 6y agoI can't agree, because I don't work on diffs, I work on whatever the current state of the file hierarchy is. When I make changes to a file, it's not enough to know just the few lines before, I need to know what versions of code are operating dozens of lines away and in different files. Further, diffs only make sense if you have the full version of the files, pre-diff. Without that full file state, diffs could operate in all sorts of incorrect ways. I think this is one thing that git actually does correctly: save full checkpoints of a directory structure. I want the same thing when I'm dealing with, say, edits of an essay or book. I want the saved full state, and may produce a diff for convenience, but I wouldn't want to save the diff as the fundamental object of interest.
- the_gipsy 6y agoYour output, however, should be a patch. That's all I care about when it's time to review and integrate.
- epistasis 6y agoYou had better care that the patch is coming from the same starting state, or else you could be wasting you time on a patch that makes no sense.
- pmeunier 6y ago> I can't agree, because I don't work on diffs, I work on whatever the current state of the file hierarchy is. When I make changes to a file,… This is somewhat contradictory. Of course, everybody works on states. But as you write yourself, you make changes to a file, you don't create a full new version from scratch every time you add a semicolon. So, if you "make changes", you actually do "work on diffs". > Further, diffs only make sense if you have the full version of the files, pre-diff. Without that full file state, diffs could operate in all sorts of incorrect ways. That actually is an argument against Git: because Git doesn't model diffs, yet tries to simulate them, Git merges and rebases do all sort of incorrect things. For example, even the following simple example is not correctly handled in Git: https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html (see the figure "git merge vs pijul merge").
- tener 6y agoSnapshots and diffs are dual. Most version control systems operate on snapshots, but there is at least one that doesn't: Darcs. Written in Haskell, it bases it's operation on an algebra of patches. Unfortunately, while usable (and neat!) for smaller projects, Darcs has serious performance problems (due to inherent algorithmic complexity) and gets slower with certain operations. Still, one can hope it will inspire a next generation of version control systems.
- gnufx 6y agoDarcs is quite usable in my experience, for medium-sized projects, at least. Say ~50MB of source with ~10 years of history. Darcs has inspired the next version of itself (version 3), but it's not clear how that will compare with Pijul.
- pmeunier 6y agoThis is not inherent to the patch model, just to Darcs' algorithm. Pijul (pijul.org) doesn't have these problems, and is indeed quite fast. The main limitation at the moment is merging very large histories, which has the same complexity as a Git rebase at the moment (this will be fixed).
- im3w1l 6y agoI wonder if you could do the same thing video codecs does. Diffs, with the occasional keyframe.
- pdonis 6y ago> Diffs are a "natural" object for version control Diffs are natural objects for evaluating commits--answering questions like "does this new version of the code make sense?"--but they are not the natural objects for storing commits.
- oblio 6y agoTo expand, since we agree. Diffs are "natural" objects for users of source control. Diffs are not natural objects for storing anything since that's not the job of the users, it's the job of the version control system. It can use ponies to store the "controlled versions" for all I (and probably other users) care.
- pdonis 6y ago> It can use ponies This I would like to see. :-) Particularly the part about defining a protocol for sending messages over the Internet using ponies (there already is one for pigeons...)
- pmeunier 6y agoI don't know about ponies, but for snapshots, I disagree: snapshots don't model diffs properly, and many operations in Git are essentially trying to simulate patches: - Merge and rebase both try to "replay" patches. But because Git doesn't work with patches, it actually only simulates that, using heuristic algorithms such as 3-way merge. This works most of the time, but not always (see https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html). - Conflict resolutions in Git cannot be modeled as patches, and you need `git rerere` to simulate that. This is not anecdotal, since conflicts is the one situation where you need the best and most intuitive tool. - Short-lived branches are another poor simulation of patches: if Git were really working on patches, you would be able to create a branch "after the fact", meaning that you would work (possibly on multiple features at the same time), and push different parts of your work separately, without having to worry about branches before you start.
- Gibbon1 6y ago
- geofft 6y agoI don't think diffs are the "natural" object - if you talk to someone who isn't using an automated version control system but still doing revision control, what they have is "Resume 12-1-2020.doc," "Resume 12-2-2020.doc," "Resume reviewed by Joe.doc," "Resume for Contoso final.doc," "Resume for Contoso final final USE THIS VERSION.doc," etc. Those are snapshots of individual versions of the files, not diffs. Or think about your favorite wiki (Wikipedia or your corporate wiki) - if you hit the "History" page, what you see is a list of the versions of the page, and then some UI to compare any two given versions. While there's a button for computing a diff between any version and the previous, that's not what it appears to treat as the natural object. Diffs are an emergent object when you have some mechanism to automatically create them and apply them. (This is generally a programmatic mechanism, but there have long been cases where the mechanism is manual effort - think laws that amend other laws by saying "After section 3 insert... and remove section 5".) But the ultimate goal is to produce a version of the file. If you want a diff management system, play around with quilt, which treats diffs (patch files) as the first-class object. Just about everyone I know who's tried using quilt for more than the tiniest amount of work vastly prefers just importing the applied patches into git and working with them in git. This is largely because quilt is empirically much more leaky - it's very sensitive to the current state of your directory, because a diff is a second-class object which requires something to be applied to in order to make sense, and it's too easy for the contents of your directory to not quite line up. (Put another way - there's a reason we call it "version control" / "revision control". The thing it tracks is versions/revisions, which are concrete instances of the file in time. Quilt is a patch management system, and very few people are interested in those.)
- oblio 6y agoYou're right but I think what you're describing the absolute bare minimum of version control. Yes, people want to ostensibly see the different versions/states. But for anything past the simplest setups, they immediately want to see changes, since you're actually after change management, not version control. Viewing different versions scales very poorly. Even regular user oriented tools such Word offer "Track changes", which is a diff view. Wikipedia also has a "Compare versions" button quite prominently on the page you mention. If anything, most users I've exposed even to simple diff tools immediately appreciate the value. It's just that mainstream tools rarely support diffing since it's very complex from a technical point of view (end user tools and apps rarely work with plain text, they use binary formats, marked up text, etc), so it's not a mainstream paradigm as a result, and people are less familiar with it.
- yodsanklai 6y agoI'm not sure I understand what you mean. Snapshots and diffs are different views of the same thing. You need both. It doesn't really matter which one is more natural or primitive in the implementation. When I git push, I upload my commits (i.e. snapshots). And when I git cherry-pick, I apply a diff. Both views are needed, I don't see why it's a leak.
- pmeunier 6y ago> Snapshots and diffs are different views of the same thing. Not completely: think of conflict resolutions for example. In patch-based systems such as Darcs and Pijul, you don't need a "rerere" command to handle that special case. And I'm sure you'll agree that conflicts is the one critical situation where we need the best possible tool. > When I git push, I upload my commits (i.e. snapshots). And when I git cherry-pick, I apply a diff That's not how Git does it. Instead, Git runs 3-way merge, which doesn't do what you expect, see https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html.
- jamesdsadler 6y agoGit does use diffs extensively its algorithms, just not in its representation of history. Git's "domain model" is blobs, trees, commits, branches and tags. At a lower layer than that (the pack files) it does use binary diffs to store similar objects as deltas to save on disk space. TL;DR diffs play an important part in git's machinery at different levels of abstraction. They just aren't part of how it represents its domain model.