7 ms·
High Performance Git
- normie3000 6mo ago> LFS adds its own operational overhead. Seemingly seconds on every remote-touching command, even on a very small repo.
- Hendrikto 6mo agoWhat is worse is that for about half a year or so, I now have to authenticate my ed25519-sk key with my Yubikey thrice (!) when using LFS. On every push.
- fragmede 6mo agoThat they didn't go with git annex was such a fit of NIH of a mistake.
- Hendrikto 6mo agoBoth have their advantages and disadvantages. git-annex is not strictly better, LFS just chose different tradeoffs.
- wadefletch 6mo agoted nyman: #1 most knowledgable college football fan in sf and also git which makes more sense i guess
- mitchellh 6mo agoOf most things, really, he was on Jeopardy for a reason! https://thejeopardyfan.com/tag/ted-nyman https://thejeopardyfan.com/tag/ted-nyman
- snthpy 6mo agoI've been wanting to ask this: Why isn't git clone --depth 1 ... the default? I would guess that for at least 90% of the repos I clone, I just want to install something. Even for the rest, I might hack on the code but seldom look into the history. If I do then I could do a `git fetch` at that point and save the bandwidth and disk space the rest of the time.
- dwattttt 6mo agoA question: why is git involved at all in this? You don't want a repository.
- skydhash 6mo agoThis! The default was to have a link to download a tarball of the source. And if the user wanted to contribute (or check the devel version), you would add a link to the vcs.
- eddythompson80 6mo agoI think gitignore solves a problem that is hard to solve with the traditional tarball approach. Downloading a tarball and running ./configure or make, editing a config file here or there, etc then running `make install` is the most common flow. Now days I find myself frequently editing the Dockerfile to make it to my liking. With a git repo, the owners of the repo have excluded all the local files, build caches, etc and you can keep pulling to get updates stashing and reapplying your local changes. With tarballs, you have to figure it out all over again. Lose your build cache (language dependent maybe), lose a change you made here or there, etc.
- kingstnap 6mo agoGrabbing git repos instead of just tarballs is useful. A) You can update them, because you can git pull to fetch changes. B) If you want to apply patches on top, its better to have version control so you can keep track of what you changed, especially useful if you want to rebase.
- skydhash 6mo agoA) only valid if you want to stay with the devel version B) See A I use OpenBSD and before that, I was on alpine, debian, and arch. Of it was a software I want to try, I downloaded the tarball. if it’s something I wanted to keep for longer, I created a port or a custom packages.
- deleted 6mo ago[deleted]
- anitil 6mo agoI'm only on to chapter two and already it's explained some plumbing details that I somehow have missed all these years. This is great
- tnm 6mo agoThanks much. I hoped that chapter would stand well on its own. That and the packfile chapter were the first I wrote, partly as a reference for myself!
- nikhilpareek13 6mo ago[flagged]
- ruuda 6mo agoThe text reads like an LLM was involved in this.
- nananana9 6mo agoGit is industry standard, because for what it give you it's a remarkably robust and simple program to use. We're all vaguely aware that the internals are complex, but the UX is clean and usable enough that the complexity usually doesn't leak out. But the day this breaks down and I have to deal with bloom filters, packfiles, maintaining the git garbage collector or rerere cleanup, is the day I switch our codebase to a centralized VCS. This stuff is cool to learn about; but it's 5 layers removed from anything I want to be thinking about in my day to day work.
- thfuran 6mo agoI'm pretty sure git is industry standard almost entirely entirely because GitHub exists. And I very much disagree that the UX is clean. The cli is more than a bit of a mess.
- stingraycharles 6mo ago> I'm pretty sure git is industry standard almost entirely entirely because GitHub exists. Nah, I remember that time vividly, Github became a thing about a year or two after it was already very much taking the lead. GitHub became GitHub because git was the winner. There were alternative hubs that supported bazaar and mercurial and whatnot, but git won because for most people, Linus and the kernel team being behind it was reason enough to trust it. (and I say this as someone who liked hg more than git)
- embedding-shape 6mo agoI mean, I don't think anyone can say for sure if "GitHub became GitHub because git was the winner" or "Git became mainstream because GitHub won the developer mindshare", pretty much everyone I knew used GitHub for everything besides the actual VCS protocol, although a lot of us early users were users of GitHub especially because of git. Most people just wanted to collaborate on the platform other people were on, and where the popular projects were, that it used git was just an implementation detail at that point for most I think.
- 6mo ago
- aa-jv 6mo agoI've always wanted to see a book that describes git for the common man and gives them tons of examples for how to use it to do productive things. Even for a small office, git can be immensely useful. Entire production line workflows can be implemented with git .. if only folks would learn to use it productively. Its not just for development. Writers can use it productively. Accountants too. It always kind of irks me that Git hasn't just been folded into the OS front-end UI by any of the OS vendors .. it'd be so revolutionary to give common folks an easy way to manage the timeline/history of their computer use using git.
- awesan 6mo agoThe obvious reason is that most file formats used by writers, accountants, etc. are binary files which do not very much benefit from git.
- fragmede 6mo agoMicrosoft Office files are zipped XML these days, there's a standard and everything.
- deleted 6mo ago[deleted]
- aa-jv 6mo agoSo? Doesn't matter. Git in that case still provides valuable historical archiving and versioning that is still more useful than the option, without it. Plus, its chicken and egg. If the OS had a great interface to Git as part of its responsibilities in the Explorer/Finder interface, folks would be more inclined to use text-based file format standards that are coherent with the Git methodology.
- jcranmer 6mo agoThe thing is that change tracking and sometimes even full-on version control are actually integrated with a lot of the tools that people use, like your document editor. The incremental benefit of git then would largely not be automatic version tracking but only the interactive history browsing which git itself is kinda meh at (and is of questionable value for a lot of workflows). And the cost of this transition is forcing people to use a different tool from their regular workflow, one that wants to be used in an environment they're not comfortable in, and also one that is not conducive to handling anything other than plain text. Rather, programmers should learn from how other software handles version control and incorporate those ideas into git instead. For example, perhaps we should automatically create a commit every time we build the project so that we can roll back or forward to previous builds and not rely on the programmer to remember to make commits so frequently.
- alxgsv 6mo agoI never faced git performance issues when working with code. Guess my repos weren't bit. But when I tried to use git as a versioned database of changes in my pet project, I learned a lot about indexes, compacting, etc. Article covers a lot and is very helpful!
- WolfeReader 6mo agoGit is noticeably slow on Windows. Git is built to run on top of Unix commands, which work great on Linux and Mac. For Windows, the commands have to be installed separately, and there's a performance penalty for each call. Individual Git commands are usually fine, but anything that calls several steps in sequence will visibly drag. (WSL gets around this entirely.)
- retired 6mo agoI hardly see developers with Windows anymore, I guess this is one of the many reasons.
- WolfeReader 6mo agoI have to use Windows for work. With WSL, it's actually perfectly fine! Which is really more of an endorsement of Linux than Windows.
- retired 6mo agoI last developed on Windows in the time where I had to fiddle with Git Bash, Cygwin and manually setting my PATH. It was not a fun time.
- zokier 6mo agoafaik far bigger factor is that windows file io is just generally much slower than linux. both of these are further exacerbated by av solutions which are ubiquitous in windows. that is why ms introduced "dev drive" in windows few years back which in their own benchmarks showed biggest gain specifically with git: https://blogs.windows.com/windowsdeveloper/2023/06/01/dev-drive-performance-security-and-control-for-developers/ https://blogs.windows.com/windowsdeveloper/2023/06/01/dev-dr...
- hmpc 6mo agoSimilarly, if not performance-focused, I can wholeheartedly recommend Building Git[0], which walks you through building your own git clone in Ruby (although the language is immaterial). [0]: https://shop.jcoglan.com/building-git/ https://shop.jcoglan.com/building-git/
- ergl 6mo agoSurprise, surprise, another piece of LLM-generated slop on the front page of HN. From chapter 1: > When Git slows down, engineers adapt in bad ways. They stop asking questions the history could answer. They batch work to avoid sync cost. They keep messy branches alive longer, postpone cleanup, and treat the repository like something slightly dangerous. From https://gitperf.com/epilogue.html https://gitperf.com/epilogue.html > Once machines start producing code at machine cadence, the model from this book does not break. What changes is the pace: more branches, more commits, more automation, and more surrounding metadata. The traffic gets louder, and the features that keep Git legible under pressure move from "nice to have" to "essential." > These stop looking like side optimizations. They are what keep machine-scale Git traffic usable.
- redditor98654 6mo agoI had the same thought. TBH there is nothing in those individual sentences that read like AI but when you read them all together I could see it too. I dunno what it is, only way I can describe it is that it does not sound like a normal human but rather a monologue from a character trying to sound impressive with each successive sentence.
- ergl 6mo agoThe author works at OpenAI, so it's no surprise that they've stopped noticing how grating this kind of structure is to read.
- alchemist1e9 6mo agoI think it’s likely there will be methods to fix this soon, some de-slop algorithms, or is there a deep reason it will always be detectable? Perhaps there are some PhD linguists who have figured out how to quantify the “slop” effect and are writing their thesis on it. Once that is done it will be possible to smooth it away. The book is definitely LLM assisted authoring yet it also has great content, so not sure we can immediately jump to shaming it entirely for being slop.
- tnm 6mo agoSo nice to see this get picked up, and honestly surprised to see the interest in what I think of as an extremely esoteric area. Few things: - Just released an Edition 1.1 that fixed some small errors, amended a few chapters content, and removed some general bluster. I'm going to try and, well, version these. - New things are coming to Git, and I suspect I'll be talking about Git Futures or A Post-Git World soon enough. - There's now a free PDF, https://gitperf.com/pdf.html https://gitperf.com/pdf.html - I'll have a couple more highly practical chapters coming soon, focused on pragmatic organizational adoption, e.g., on wrapping the git CLI to best practices
- lthi747 6mo agoThe only thing I miss in git is an uncomplicated sparse checkout, or how svn’s allows to just check specific path.
- john-tells-all 6mo agoGit's data structure is shockingly simple. Here's an article with diagrams on the four basic data types. Also shows why "HEAD" is different from a "commit". https://eagain.net/articles/git-for-computer-scientists/ https://eagain.net/articles/git-for-computer-scientists/
- packetlost 6mo agoIs HEAD not just a ref to a commit? There's basically only two "things" in git, refs and objects. Git internals are so easy that IMO people should start off by running through this tutorial [0] instead of learning the basics of git porcelain, it makes understanding what's going on so much easier. [0]: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
- skydhash 6mo agoObjects should be split into trees and blobs to make some operations clearer, especially checkout and rename detections.
- packetlost 6mo agoThere's also commits and tags. Commits are important for understanding how branches and histories work. I was just trying to be brief, the types of objects are important and covered in that tutorial.
- WorldMaker 6mo agoHEAD in most operations is usually a ref to a branch, which makes it somewhat unique as a ref type (it's a ref to a ref, double pointer). When it is a ref to a commit, that's a detached HEAD state. Plus HEAD to the CLI can also mean the family of refs under refs/heads/* that relate to the HEADS of each branch (which depending on fetch status may not be the same as the branch ref) and traversal into the reflog.
- zokier 6mo agoIt annoys me greatly that the megacorps have all these fancy devtools that we plebs have no hope of touching. One example being Metas Sapling, which afaik is their way of making git-compatible vcs perform decently well at their scale. Sure it is nominally open-source, but the readme has enough notes like "Not yet supported publicly, OSS is buildable for unsupported experimentation." to dissuade any casual users. And afaik the others have their own things, mostly even less public. https://github.com/facebook/sapling https://github.com/facebook/sapling
- WorldMaker 6mo agoI appreciate that of such companies Microsoft seems the most dedicated to upstreaming what they can directly into git. Most of those heavy performance features are off-by-default, but they are there to `git config` (or `git maintenance` or `git clone`) yourself.