9 ms·
Supercharging the Git Commit Graph
- s2g 8y agoThis is very cool. Makes me sad I'll never do anything like this.
- slobotron 8y agoInteresting tidbit: > The developers making Microsoft Windows use Git
- xkcdefgh 8y agoYeah, i believe it was a huge effort some time ago to migrate windows to git
- LyalinDotCom 8y agoYep we use Git to build windows! here is the story that was told about our efforts. https://arstechnica.com/gadgets/2018/03/building-windows-4-million-commits-10-million-work-items/ https://arstechnica.com/gadgets/2018/03/building-windows-4-m...
- LyalinDotCom 8y agooh and also we did an interview with the DevOps lead for both source control and build system around Windows, so check that out too if you want: https://www.youtube.com/watch?v=nsXiKLLaH4M https://www.youtube.com/watch?v=nsXiKLLaH4M
- 01100011 8y agoKinda surprising. I know everyone seems to love git these days, but I find it's really better suited towards distributed and/or smaller projects. After feeling the pain of a megarepo system at work, I'm pushing to switch to a monorepo(well, like 4 repos instead of 200). git sort of sucks for monorepos. Also, even after learning a fair amount of git, I still find I spend a noticeable amount of time dealing with it. I don't remember spending a lot of time on any of the VCS systems I've used in the past. They just stayed out of my way and let do my thing.
- rtcoolaid 8y agoYou are actually right and down voters have no clue (and this is what I hate about HN, if you don't comment then don't downvoye) To answer your question though - Microsoft has a lot of git extensions that they are slowly submitting upstream. Hence git is usable for their megarepo. Look up GVFS on GitHub, very cool work. Same for Twitter (they are not submitting upstream though as far as I know) Similar for fb/mercurial
- adrianN 8y agoI do a lot of things that aren't really possible in older VCS with git, which is why I spent more time with it.
- gmueckl 8y agoI don't do new things in git, but since we transitioned, I spend about 3 to 4 times longer wrestling the VCS than with anything that I have used previously. This is merely for routine stuff, because it takes more steps
- adrianN 8y agoThe additional steps enable workflows that weren't possible before, so it's a tradeoff.
- gmueckl 8y agoThis argument is just wrong. I have fewer steps with less mental load on other VCS implementations, yet I can get better workflows at the same time with zero loss of functionality. Prime example: the git staging area/cache/index needs to die. Git would be half as difficult to use with fewer code shredding surprises. This abomination is a prime example of badly exposed internal structure. Everything feature that is crammed into this whatever-the-hell-that-is could be replaced by a vastly superior solution which does not require anything like it.
- 8y ago
- yoklov 8y agoDang, this made `git log` inside the repo I use at work (which is enormous with a stupid number of commits) nearly instantaneous. Great work. `git status` still takes over a second for me though, oh well...
- fanf2 8y agoHave you tried the fsmonitor hook feature that was added in git 2.17? https://blog.github.com/2018-04-05-git-217-released/ https://blog.github.com/2018-04-05-git-217-released/
- yoklov 8y agoYep, as well as the untracked cache. I also was using a split index for a while, but it didn’t play nicely with some tools...
- erikb 8y agoEverybody who seriously uses git will also want to see the graph somehow. Here's how I do it: https://github.com/erikbgithub/dot-files/blob/master/.gitconfig#L38 https://github.com/erikbgithub/dot-files/blob/master/.gitcon...
- bluebluetimes 8y agoTo enable the commit-graph feature in your repository, run git config core.commitGraph true. Then, you can update your commit-graph file by running ‘git show-ref -s | git commit-graph write —stdin-commits' how do you automatically update this?
- rakoo 8y agoIt's not perfect but doing it in a pre-push hook can be useful. Unfortunately there's no post-receive hook on client-side, which could have been useful in this situation...
- masklinn 8y ago> It's not perfect but doing it in a pre-push hook can be useful. That's completely unnecessary and way too frequent. Some where else (reddit I think?) the authors noted that they'd like to have it run alongside GCs in the next version. So running it as pre-auto-gc is a better idea.
- rakoo 8y agoI was thinking that you need to do that everytime refs change, but it's just a boost and I presume you can have some refs in the commit-graph and some not in there and it will still work, so you're probably right.
- stolee 8y agoWe are working to make this run automatically in the future. https://public-inbox.org/git/20180627132447.142473-1-dstolee@microsoft.com/T/#u https://public-inbox.org/git/20180627132447.142473-1-dstolee... You don't need to run this with every commit, but maybe once after a big fetch or right after a new clone.
- pmarin 8y agoThis is basically what Fossil call timeline or I'm missing something? https://www.fossil-scm.org/index.html/timeline https://www.fossil-scm.org/index.html/timeline
- masklinn 8y agoIt's not a UI feature, the UI feature has existed forever (it's the log and log graph). This is a cache for the graph so that it does not have to be rebuilt every time it's displayed.
- rakoo 8y agoYet another file in the .git directory. The work is impressive and certainly helpful, but I can already hear Fossil proponents say "just use SQLite", which is getting more and more true.
- rtcoolaid 8y agoIs that what you got from the article? Or you're the guy that knows the names of all he latest tools but understands the underlying concept of none and comments this making HN a waste of time to actually learn anything.
- naturalgradient 8y agoI think you should refrain from making such comments and follow the best-intention assumption on HN.
- Aissen 8y agoI had to bite… The sqlite repository has only 20795 commits since May 2000 at the time of this writing: https://www.sqlite.org/src/timeline?udc=1&ss=m&n=100000&y=ci https://www.sqlite.org/src/timeline?udc=1&ss=m&n=100000&y=ci This is the amount of commits that goes into Linux every ~5months. Has anyone done any meaningful performance comparison between fossil and git?
- farresito 8y agoTo save people some time, an alias for your .gitconfig create-graph = "!f() { git show-ref -s | git commit-graph write --stdin-commits ; } ; f"
- masklinn 8y agoHow does it save time, given this command will rarely be run, and next version will automatically run it on GC? Instead of copy/pasting a command you now have to copy-paste that same command with more gunk added and run it separately.
- farresito 8y agoYou have to run it on every repository you want this in, don't you? At least, that's what I understood.
- masklinn 8y agoThis is mostly for large repositories where building the revisions graph takes a long time (aka 5+ figures revisions). I have two of those from $dayjob, most of the stuff I work with/on/for doesn't come even remotely close. Running this on a repo with 5 commits is all but useless. And even then you still really only need run it once per repository, you can just cd/paste/return; cd/paste/return; … Hell, you'd probably have an easier time writing a script which looks for all git repositories and runs the command versus having to manually visit each and do so.
- deleted 8y ago[deleted]
- wscott 8y ago"Before I joined Microsoft, I was a mathematician working in computational graph theory. I spent years thinking about graphs every day, so it was a habit that was hard to break. " As a former BitKeeper developer, this is a key person for Microsoft to have on hand to improve git. BitKeeper got about 10X faster after it was used on the Linux kernel and many of the key performance wins were due to better graph traversal algorithms. Rick was a wizard at that sort of thing. The other was memory layout optimizations and caching. (my contribution) So Rick made sure we walked the graph as little as necessary and I made sure the graph had an extremely compact representation containing as little information as possible and then once a target commit is found it is looked up in another store. However, this quote was disturbing. "There are a few Git features that don’t work well with the commit-graph, such as shallow clones, replace-objects, and commit grafts. If you never use any of those features, then you should have no problems!" The joy of not having to write commercial software! Reading between the lines it appears they changed the default output order for 'git log' or some internal API and then didn't bother to fix the cases that depending on the old order.
- stolee 8y agoSorry for the worrying note about the experimental feature. One issue when working in open source is that contributors don't have control over the release cycle, and review requires smaller series than having the feature be delivered all at once. These interactions with grafts, replace-objects, and shallow clones are one reason 2.18 does not create and manage this file automatically. The commit-graph file works by representing the commit relationships in a new file, and if that file exists, we treat that as the truth. The commit grafts, replace-objects, and shallow clones use another set of special files to track commits whose parents have been modified in special ways. If you would like to see our progress on integrating these features together, please see this thread on the Git mailing list: https://public-inbox.org/git/20180531174024.124488-1-dstolee@microsoft.com/T/#u https://public-inbox.org/git/20180531174024.124488-1-dstolee...
- wscott 8y ago> One issue when working in open source is that contributors don't have control over the release cycle, and review requires smaller series than having the feature be delivered all at once. Yes, my reply was unnecessarily disparaging. Overall this looks like a cool feature. Perhaps a stopgap solution if for those commands to just delete your cache. But then your repository will get mysteriously slower. I just need to think of this as a technology demonstration.
- tomfloyer 8y agoI wish i had those amazing algorithm skills.
- xvilka 8y agoWould be nice to have support of it in tig too.
- yebyen 8y agoOne of the best things I ever did for my git usage was installing this ridiculous thing in my .gitconfig aliases: [alias] l = log --date-order --date=iso --graph --full-history --all --pretty=format:'%x08%x09%C(red)%h %C(cyan)%ad%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08%x08 %C(bold blue)%aN%C(reset)%C(bold yellow)%d %C(reset)%s' It lets me type 'git l' and get this kind of commit graph: * 8264cf2a 2018-06-28 yebyen (origin/dev, dev) ... |\ | * df6e4c49 blabla commit messages | * 195d8924 commit messages |/ | * 96e74a5c (branch-label) commit message | |\ | | * b2786c07 blabla | | * 85721288 blabla This is semi-related but not nearly as interesting from a technical perspective. But it's one of the missing features in Git, to visually help people understand why it's helpful to rebase occasionally, and keep your commit history clean. I think I found it here[1] [1]: https://stackoverflow.com/a/16735971/661659 https://stackoverflow.com/a/16735971/661659
- WorldMaker 8y agoI had a similar alias for a while at a past job, but realized I was just using `gitk` anyway, and `gitk` is installed by default everywhere and doesn't need an alias setup, so it's also especially easier to teach to junior developers. Also, I don't encourage rebasing, especially to junior developers. I realize everyone has different preferences, but a messy graph is useful and there are other tools like `git log --first-parent` for getting "clean" baselines from the graph. The great thing about using a graph in the first place is there are a lot of traversal options and you don't have to micromanage where the trees are to still see the forest.
- yebyen 8y agoYes! I agree with you mostly, except that we try to keep from having junior developers for too long. Nobody is really a junior developer, and everyone should get a turn as release manager. The main benefit of this is that people can see directly how their work style impacts the release engineering process, if they do know exactly what that process involves and actually get a turn at it. (You have to stub your own toe in order to know how bad it hurts.) Our team is actually really small, and we like to make sure everyone knows about the hassles involved in putting together a release and doing a complete code review when it's needed. The main reason I encourage rebasing is because it helps avoid a messy graph, and a messy graph makes it immediately much harder to do a rebase across any number of merges, or any non-trivial span of time. So in other words, I like to selfishly expect the other developers on my team to do rebases at the appropriate times, in order to preserve my own capability to easily do rebases when needed. (We also learned last week that git rebase has a --preserve-merges option that I feel foolish to not have known about sooner. We've wiped out so many merge commits unnecessarily.) We treat the master branch as carved in stone like anyone should, but other branches should clearly flow their merges only one way (features into releases, or features into environments), and if a branch hasn't actually been included in a release tag, it's considered fair game for rebasing. It helps us to prevent our three developers' sometimes too many concurrent trains of thought from resulting in equally too many confusing merges, or an intractable number of HEADs to manage and organize into releases. One of the places we struggle is that we're not all-the-way onboard with CI/CD processes, but we do the best we can so that whenever support for that kind of thing materializes, we will mostly not need to change our processes at all and can obtain the benefits of that kind of tooling as quickly as possible.
- claytonjy 8y agoI wonder if any of these folks will have a hand in improving the Github commit graph, now that Microsoft owns it? As great as Github is, I've never understood why they still have such a hard-to-read, horizontal commit graph while competitors like Stash/bitbucket/gitlab have all had beautiful vertical graphs (like shown in this article) for as long as I can remember. I think this is especially valuable for newbies who are less inclined to get similar viz at the command line, but still useful for vets when they (inevitably) end up in weird branch situations.