9 ms·
The Architecture of Git (2012)
- devy 8y agoVery informative. This explains well about why Linus once said signing every single Git commit is unnecessary and it only means you have automated the signing process. [1] [1]: https://news.ycombinator.com/item?id=12290873 https://news.ycombinator.com/item?id=12290873
- ArchTypical 8y ago> To understand Git's design philosophy better it is helpful to understand the circumstances in which the Git project was started in the Linux Kernel Community. I don't understand this sentiment. It's not helpful to know the history at all. At best, it romanticize the choices made. Stating the goals would be an intro that shows some level of analysis.
- mycall 8y agoI am fine with having extra sentences to read.
- gmueckl 8y agoNo, the author is right. Git us in its core a database for managing patches. Understanding the needs of Linus Torvalds as his role of Kernel maintainer is about the only good way to understand why git is so strangely designed.
- avar 8y agoDarcs is a database for managing patches. Nothing in Git inherently cares about patches. To a first approximation it's a database for managing full snapshots of trees of files.
- gmueckl 8y agoGit only needs lists of files because it needs entry points into its lists of patch fragments that make up the file and to assign file names to them. Other than that, a changeset is just another name for a patch that can be added, altered, rewritten or removed. That makes git a patch database in my book. Darcs feels more like a research project to me. The developers try to find a theoretical foundation in which they can base a VCS, but they have not managed to make their theory work with the level of perfection that they want. But if they eventually get it right, it will probably have the provably best text-based merge tool possible.
- avar 8y agoThis is not how Git's data model works. You may be thinking of delta-compression which during "git gc" and purely as an optimization step does delta-compression across content in the repository. But that's purely an optimization that has nothing to do with the intrinsic data model. There's no point at which the patch output you see with "git diff/show" is actually stored as-is in Git. It's computed on-the-fly. This separates Git from many other SCMs where patches or other deltas are permanently stored at the time of commit in a way that can't modified afterwards. The distinction matters because those systems generally have storage that doesn't compress as well, since they need to compute and store a diff at the time, whereas a system like Git can keep finding better delta candidates as history progresses. This goes all the way back to the likes of RCS. The Subversion FSFS backend also works like this, and I believe Mercurial to some extent, and certainly Darcs since storing a history of patches is what it's for.
- mbfg 8y agocurious why you say it's strangely designed. seems pretty rational to me.
- gmueckl 8y agoThere are three things in git that I consider design errors: - The staging area/index/cache should not even exist. That the same construct has three interchangeable names is already a sign that something is wrong. The fact that that construct is used to confusingly stage snapshots of files for committing as well as moerge operations makes it an unwieldy thing that has probably teleported too many lines of code into the digital nirvana already. - Branches should be immutable properties of changesets instead of flimsy, easily deleted tags with special flags. Deleting a branch after a merge makes it impossible to tell which branch in the history was the master and which the feature branch. - Gits graph of changesets is also too lightweight and is missing forward references. This is the reason why deleting branches irreversibly deletes their entire history. The reflog is only a crude hack around that and exists only because the crude data structures require taking stock of the entire set of internal references to figure out that a certain part of it (an "object", but essentially a file in the repository) is actually no longer referenced and can be removed. I can probably come up with more reasons why git is very flawed. But this is enough fuel for the fire for one post.
- mbfg 8y agoMaybe -- but the index is ridiculously useful. being able to commit some of your changes is part of what makes git so much more useful than something like mercurial. To be honest you may have a complaint for 2 and 3, but i'm not sure what it is, as i've never had any of the issues you bring up.
- krupan 8y agomercurial and git both have commands that allow you to select part of a change to commit, and both allow you to amend an existing commit. No index necessary.
- deleted 8y ago[deleted]
- dahart 8y agoThe goals are stated, just a few lines down in the same section you quoted. What makes you think design goals and philosophy aren't formed by history? How does knowing the history of a project romanticize it? Personally, I'd assume exactly the opposite, that knowing history is the only way to understand the goals and philosophy, and that romaticizing only happens when a history is not understood and/or the story is changed or told through a tinted lens.
- alkonaut 8y agoIf it had been started in e.g the game development scene there would have been some other design decisions made. The idea that the source tree is rather small and the history is short and there are few binary assets managed are definitely showing in the design.
- jayd16 8y agoBy understanding the real world motivation you can reason about both the stated and unstated goals of the project.
- frou_dh 8y agoGit is the VCS most suitable for bottom-up learners. Which is the VCS most suitable for top-down learners?
- WorldMaker 8y agoDarcs, probably? Darcs has one of the easiest UIs in the DVCS game and was always very easy to teach people how to use. But then of course if you want to learn the details of how darcs works under the hood you wind up in Haskell, and feeling a need to read a lot of academia papers.
- pritambaral 8y agoDropbox, perhaps.
- MordodeMaru 8y agoPlastic SCM.
- blattimwind 8y agoCopy/paste.
- reificator 8y agoC:/Users/Billy/Desktop/Backup/New Folder (39)/IMPORTANT/My_Project (OLD)/My_Project.sln.bak
- crispyambulance 8y ago> Which is the VCS most suitable for .... Git. It doesn't matter what your question is. :-) I mean, come on, it's not like you have a choice. You gotta use whatever your teammates/coworkers/organization is using. That said, yeah, there's a lot of pedagogic problems with git stemming from the INSANE inconsistency of command-line. The only redeeming quality? It works and it's popular.
- reificator 8y agoIt works and it's popular and the underlying model is easy to understand. I think step one to replacing git will be to launch a tool that completely replaces the git CLI but uses the same model below. I've not looked into EasyGit much, but I don't think it goes far enough.
- nine_k 8y agoThe best (known to me) informal intro into the architecture of Git is The Git Parable: http://tom.preston-werner.com/2009/05/19/the-git-parable.html http://tom.preston-werner.com/2009/05/19/the-git-parable.htm...
- svat 8y agoAlso greatly worth reading: “Git from the Bottom Up”: * HTML version: https://jwiegley.github.io/git-from-the-bottom-up/ https://jwiegley.github.io/git-from-the-bottom-up/ * PDF version: http://ftp.newartisans.com/pub/git.from.bottom.up.pdf http://ftp.newartisans.com/pub/git.from.bottom.up.pdf (via http://newartisans.com/2008/04/git-from-the-bottom-up/ http://newartisans.com/2008/04/git-from-the-bottom-up/) The great thing is that after reading and understanding these, one's mental model matches the reality of the Git program, so one can both try bolder things, and get unstuck from any mess.
- freshhawk 8y agoI recommend this constantly, it is excellent. Not to outright beginners, but anyone past that point should read this. It is a great job of clearly presenting an accurate mental model that helps you use git. If you are already a git expert this is just a good thing to learn from in general and with how to explain and teach git usage in particular. The only documentation I know of that can turn people from cargo cult git users to people who just do the version control things they need done with the parts of git they need. That is damn useful.
- paulddraper 8y ago> After using branches for a while you notice that they can serve two purposes. The first use case allows you to keep track of ongoing development, things like “Release Maintenance”. The second case is useful for labeling points of interest, like “Version 1.0” and “Version 1.0.1”. > For the sake of clarity and elegance, you decide to create another file called tags to contain pointers of the second type. Keeping these two inherently different pointers in separate files will help you from accidentally treating a branch as a tag or vice versa. Ah, ha ha ha ha ha ha ha ha ha ha ha. If "tags" and "branches" were the same thing, but just conventionally separate uses, that would be great. If only that was the way git treated them.
- dabei 8y agoThe whole website is a treasure trove for learning about system design.
- xrd 8y agoI still prefer my book's history of Git in which I compare Linus rejection of Monotone to the movie "Back to the Future": https://github.com/xrd/BuildingToolsWithGithubBook/blob/ca7f2c0b24496019a56fb1a1e3d81c752d601082/old/chapter-bootstrapping-git.asciidoc#monotone https://github.com/xrd/BuildingToolsWithGithubBook/blob/ca7f...
- agbell 8y agoInteresting, what am I reading?
- xrd 8y agoThis (a book about building tools with Git and the GitHub API) was published by O'Reilly in 2016, and we just released it under creative commons. You can read it completely free at https://buildingtoolswithgithub.teddyhyde.io https://buildingtoolswithgithub.teddyhyde.io or get the repo of the book contents above.
- adamkl 8y agoI've found this online course by Paolo Perrotta to be the best introduction to Git's architecture, by far: https://www.pluralsight.com/courses/how-git-works https://www.pluralsight.com/courses/how-git-works He takes you pretty much from first principles all the way up to how remote repositories are tracked. Its been far more useful to me than simply trying to learn the CLI. I realize its behind a paywall, but I'd recommend signing up for the free trial just to watch this course (if you're curious about the inner workings of Git).
- royalghost 8y agoI still recommend people to watch this video from the man himself, Linus Torvalds, if they really want to understand the architecture and data structure behind git - https://www.youtube.com/watch?v=4XpnKHJAok8 https://www.youtube.com/watch?v=4XpnKHJAok8 If I remember correctly, he mentioned that he wrote the MVP in 4 weeks and the data structures he used are quite simple. Never got a chance to look at the source code but I guess they are in github (at least the mirror copy.)
- nickthemagicman 8y agoIt only took 4 weeks plus 30 years of programming experience and a genius mind.
- sfescape 8y agoI'm surprised that it's not mentioned in the article that one of the most interesting architectural aspects of git is that it's a blockchain system.
- beagle3 8y agoIt was written before “blockchain” was a well known term.
- sfescape 8y agogit predates the creation of bitcoin and 13 years after the invention of blockchain.
- erwan 8y agoThat's arguably not the most interesting architectural aspect of Git. Or does any back-linked tree data-structure becomes interesting if the nodes keep a hash of their parent instead of a raw reference? I don't think that's the case. It might be a bit heretical but I don't think Git has a super interesting internal architecture. I'm not downplaying the fact that Git was very innovative, especially considered the landscape of SVMs at the time. The tool as a whole is great and has desirable properties but its internals don't strike me as particularly innovative. It's a clever composition of solutions to well established problem domains. And in that aspect it is a beautiful engineering solution although there is room for a lot of improvement in terms of UX. And in addition to that, I would argue that it would be a very weak definition of "blockchain". The innovation in Bitcoin is the incorporation of proof-of-work and resulting alignment of incentives such that it can achieve probabilistic consensus in an adverse setting and with some degree of asynchronicity. The underlying structure of the data is an obvious choice because it is simple and "captures" the idea of aggregate global state, but it's also hardly an important innovation. UTXOs are more significant. And also, recall that the textbook example for state-machine replication is always an append-only log for example. So that's not the crux of it, or of blockchain, in my humble opinion.
- sfescape 8y ago
- 50 8y agoNot sure if it’s of any relevance but I recently discovered https://gitup.co https://gitup.co and must say, it’s pretty cool.
- dilap 8y agoFrickin' amazing UI design, and some great pieces of functionality (edit the commit graph directly; ability to undo most operations). Unfortunately development seems mostly paused, and it still has some big gaps... Still tho by far my favorite git client. (#2 would be Sublime Merge.)
- findjashua 8y agoFork (git-fork.com) is pretty neat as well
- dmoreno 8y agoI loved the AOSA books, but there's been sometime that they don't publish a new one. Do anybody know if there is something new in the works? And of any other similar books?
- emmanueloga_ 8y agoHave you already read all of them? I was calculating how long it would take me to read and understand every chapter of every book. I usually like to write small snippets of code or look for references and stuff for books like these. Anyway, if I read, say, one chapter every Monday, it would take me about two years to complete the books! Probably not a problem that there's "nothing new" since the books seem to be more about timeless design principles and less about novelty.
- emmanueloga_ 8y agoOk, time to contradict myself. Other two books in my back burner in the same ballpark: * Beautiful Code * Programming Pearls
- macspoofing 8y ago"The Architecture of Open Source Applications" is great. I highly recommend their other articles, as they are all interesting and informative. If you can, throw a few bucks their way by buying the PDF or paperback versions.
- chmaynard 8y agoThe fourth paragraph in section 6.2 refers to "BitMover" with no previous context. Copy-paste error?
- deleted 8y ago[deleted]
- sitzkrieg 8y agothere is context in the intro (see bitkeeper)
- vram22 8y agoI had come across this gem about understanding the vi editing model a while ago: "Your problem with Vim is that you don't grok vi." It's the top answer on this StackOverflow question: What is your most productive shortcut with Vim? https://stackoverflow.com/questions/1218390/what-is-your-most-productive-shortcut-with-vim https://stackoverflow.com/questions/1218390/what-is-your-mos... Wonder if there is any such post about Git that cuts to the chase, even for a part [1] of the Git model, and explains it clearly [2]. [1] I had also come across a StackOverflow post that explains a part of git very clearly, like the vi example I quoted. I think it was about how to roll back accidental changes using "git reset --hard" and variants. Saved it, but don't have it handy right now. [2] Note: I said "clearly", not necessarily "simply". I like to quote the (probably out-of-print) book by Abbe Dimnet [3] called The Art of Thinking, in which he said something like this (while deploring the trend of books that try to make things artificially simple, a.k.a. dumbed down): "French grammar cannot made simple. It can be made clear." [3] https://en.wikipedia.org/wiki/Ernest_Dimnet https://en.wikipedia.org/wiki/Ernest_Dimnet He also wrote a book on that same topic (French grammar made clear). Googled the former book recently and saw about the second book. I've read the first, long ago, which I found in a second-hand bookshop. Good book. Apparently it was a best-seller at the time it came out, according to Wikipedia. Quotes by him: https://en.wikiquote.org/wiki/Ernest_Dimnet https://en.wikiquote.org/wiki/Ernest_Dimnet