6 ms·

Git from the Inside Out (2015)

195 points by dpeck 7y ago

troughway 7y ago
>Git from the Inside Out https://github.com/git/git https://github.com/git/git https://www.aosabook.org/en/git.html https://www.aosabook.org/en/git.html https://codewords.recurse.com/issues/two/git-from-the-inside-out https://codewords.recurse.com/issues/two/git-from-the-inside... http://gitimmersion.com/ http://gitimmersion.com/
- neves 7y ago
  Why these links are better than the original article?
  - Mijka 7y ago
    Complementary links from some list i guess. Third one matches with current article.
akkartik 7y ago
> Notice how just `git add`ing a file saves its content to the objects directory. Its content will still be safe inside Git if the user deletes data/letter.txt from the working copy. Holy crap, how do I not know this in 14+ years of working with git?! The `git add --help` manpage seems to make no reference to this feature, it just talks about adding the file to the index.
- coldpie 7y ago
  It's not really a feature, more of a side-effect. Git-add causes git to record the state of added files. You can see this because if you make changes to an added (but uncommitted) file, you can see the diff between that uncommitted index and the state on disk. That index state must exist somewhere. Where it exists is in the object dir, just like everything else Git knows about. (The article is slightly incorrect in that I think Git will eventually delete unreferenced state files during git-fsck; it's not stored forever. But there's a lot of heuristics during fsck to help keep data that could be valuable if the user messed up.)
  - akkartik 7y ago
    I always assumed the state of the index is stored in the binary file `.git/index`, and that mutations of the index overwrite this file. Is this not accurate?
    - akkartik 7y ago
      I just `git add`ed a 100MB file to a test repo, and `.git/index` only grew to 104 bytes. So it seems to only contain metadata. TIL.
      - coldpie 7y ago
        And if you go dig into `objects/`, you'll find your (possibly compressed) 100 MB object under its hash, and can view it with "git cat-file -p $hash" without ever having committed it :) Edit: And I bet if you dig around your index file enough, you'll find that hash someplace in there.
        chimeracoder 7y ago
        > And if you go dig into `objects/`, you'll find your (possibly compressed) 100 MB object under its hash, and can view it with "git cat-file -p $hash" without ever having committed it :) > > Edit: And I bet if you dig around your index file enough, you'll find that hash someplace in there. And if you want to know how that works, there's an article in the next issue of Code Words that goes into that as well: https://codewords.recurse.com/issues/three/unpacking-git-packfiles https://codewords.recurse.com/issues/three/unpacking-git-pac...
    - coldpie 7y ago
      I admit I'm getting out of my knowledge here (it's been a while since I read gitcore-tutorial), but I think that's the "to-be-committed" commit object. So the index file "points to" the object file that stores the state of the object you added.
    - tux1968 7y ago
      The index doesn't contain objects themselves, every added file (whether it is committed or not) is held as a regular object. The index simply holds a list of object ID's. So yes, if you change a file and re-add it, the index will be overwritten and the original object will become dangling without any references to it.
- frenchyatwork 7y ago
  It's certainly an understandable point of confusion. It's not clear to me if the current behavior was actually intentional, or just an byproduct of implementation. If you add a file, then modify it, and then commit it, you're old version gets committed. That caused me a bit of confusion back in the day.
  - boustrophedon 7y ago
    The behavior is pretty intentional - you can use `git add -p` explicitly to only add parts of a given file to the index.
  - tonyedgecombe 7y ago
    A mistake I seem to make fairly often. That or not adding new files in the first place.
- skrebbel 7y ago
  The staging area^W^Windex^Wcache is a terribly designed mess, that's why. The best way to think about it is it's just a half-finished commit, i.e. one without a message and an author and a date. But otherwise git treats the index like any other commit. Adding stuff to the staging area is like amending that commit. Actually committing it is like amending that commit again, but without changing the files, only editing the metadata (message, author, etc). And then moving the current branch to it. You could totally simulate the cache by doing exactly this, i.e. a series of `git commit -a --amend` commands (just make sure you don't push halfway). The idea behind the staging area is that you obviously need this all the time, because reasons, so let's force you to go through the hassle for every commit you might want to make. Because it's just a commit that hates your guts, it has all the same side effects as making a real commit has.
  - minitech 7y ago
    > The idea behind the staging area is that you obviously need this all the time, because reasons, so let's force you to go through the hassle for every commit you might want to make. I do need it all the time, and simulating it with commits would be horribly unsuited to getting work done. (The funny thing is that what you mentioned – `git commit -a` – doesn’t accomplish that, but it does skip the hassle you just said was forced.) It’s also a clean place to handle conflict resolution state, because that shouldn’t go in commits.
    - skrebbel 7y ago
      > (The funny thing is that what you mentioned – `git commit -a` – doesn’t accomplish that, but it does skip the hassle you just said was forced.) I wish it did, but it doesn't stage deletions, therefore also messing up renames, which is exactly never what you want.
      - minitech 7y ago
        It does commit deletions. It doesn’t commit untracked files. If you really want to track every new file unprompted, an alias can be made for `git add --all && git commit`.
- james-skemp 7y ago
  While confusing for some, I love this part of `git add`. While branching is already easy enough, I'll regularly get to a point where I may want to spend a few minutes going down a path. I'll either be happy where it leads me, or realize it was a bad idea and scrap it. I'll `git add` the current state, make the changes I want, test it out, and then either revert back to what's staged, or like where I'm at and `git add` the rest in. That and `git add -p` also mean that I rarely do a `git commit -a` or the like; stage it, then commit it.
  - reificator 7y ago
    That sounds like the intended workflow of `git stash`.
    - akkartik 7y ago
      What I like about `git add` is that it creates a backup I can restore from in extremis but without cluttering up various dashboard views like `git status` or `git branch` or `git stash list`.
    - james-skemp 7y ago
      Then I did a bad job of explaining my workflow. :) Stash is 'I want to temporarily keep track of where I'm at and roll back to a previous state, likely so I can do something else.' You could do a commit/branch, but it's temporary/not finished. My add is 'I have stuff that I'm working on and want to try going in another direction for a bit; I'll either add it in if I like where I went, or go back to what I've staged.' Example from a few hours ago: We've lost the primary on a project that is doing ... interesting things with Grunt. The packages are three years out of date, which is what I'm tackling first. I'm opting to break these into commit based upon related groupings of packages. So, `npm upgrade package1`, test things out, then `git add package-lock.json && git add package.json -p`. Now I upgrade another package and after testing determine that this is a pretty significant change, even though it should have been easy. Since I haven't staged my last npm upgrade I can easily discard the changes and still have all of my `npm upgrade package1` modifications. Now I can choose to commit those staged changes or try a different, but related, package upgrade. Simple example, but easy to expand this to something that touches a handful or more of files. The alternative would be to commit each individual upgrade, or roll back the last upgrade/thing(s) you did. Another common use case is if I'm writing some code and realize I'd like to refactor a bit before I commit it. `git add` the working code, refactor, and if it's getting to look like it's a commit onto itself I can always `git commit` what I have staged, versus having to undo. (`git commit --amend` would work in this case too, but I do a lot of work with third-party code and am never quite sure if I'll want to keep something for historical purposes/an alternative way to do something.)
sdan 7y ago
How did they generate those nice looking graphs?
- maryrosecook 7y ago
  Hiya! Author of the article here. I used OmniGraffle.
  - sdan 7y ago
    Interesting. Wish there was a way to auto-generate these graphs with my git history. I like them!
    - coldpie 7y ago
      Have you tried "gitk"?
      - drharby 7y ago
        I am personally embarrassed and excited to discover this just now. For that I thank you because this makes my life that much better.
    - tcoff91 7y ago
      Lots of git clients like Fork have pretty graphs for browsing your history. Even the terminal client can generate graphs in the git log if you give it the right params.
    - fit2rule 7y ago
      I regularly use 'gource' to get a lovely picture of my repos: https://gource.io https://gource.io
  - tomca32 7y ago
    Hey Mary, Tomislav here from the 2013 summer batch. Just wanted to let you know it was an absolute pleasure learning from you at HS. You're a wonderful teacher and I learned a ton. Thanks!
dang 7y ago
A thread from 2016: https://news.ycombinator.com/item?id=12802949 https://news.ycombinator.com/item?id=12802949 A bit from 2015: https://news.ycombinator.com/item?id=9793069 https://news.ycombinator.com/item?id=9793069 Discussed at the time: https://news.ycombinator.com/item?id=9272249 https://news.ycombinator.com/item?id=9272249
shadykiller 7y ago
I gave a similar talk “Inside Git Guts with Ruby” at RubyConf India 2013 - https://m.youtube.com/watch?v=lPlwkxrG2NM https://m.youtube.com/watch?v=lPlwkxrG2NM I had to learn a lot of git internals and it was super fun

195 points by dpeck 7y ago

>Git from the Inside Out https://github.com/git/git https://github.com/git/git https://www.aosabook.org/en/git.html https://www.aosabook.org/en/git.html https://codewords.recurse.com/issues/two/git-from-the-inside-out https://codewords.recurse.com/issues/two/git-from-the-inside... http://gitimmersion.com/ http://gitimmersion.com/

neves 7y ago
Why these links are better than the original article?
- Mijka 7y ago
  Complementary links from some list i guess. Third one matches with current article.

akkartik 7y ago

> Notice how just `git add`ing a file saves its content to the objects directory. Its content will still be safe inside Git if the user deletes data/letter.txt from the working copy. Holy crap, how do I not know this in 14+ years of working with git?! The `git add --help` manpage seems to make no reference to this feature, it just talks about adding the file to the index.

coldpie 7y ago
It's not really a feature, more of a side-effect. Git-add causes git to record the state of added files. You can see this because if you make changes to an added (but uncommitted) file, you can see the diff between that uncommitted index and the state on disk. That index state must exist somewhere. Where it exists is in the object dir, just like everything else Git knows about. (The article is slightly incorrect in that I think Git will eventually delete unreferenced state files during git-fsck; it's not stored forever. But there's a lot of heuristics during fsck to help keep data that could be valuable if the user messed up.)
- akkartik 7y ago
  I always assumed the state of the index is stored in the binary file `.git/index`, and that mutations of the index overwrite this file. Is this not accurate?
  - akkartik 7y ago
    I just `git add`ed a 100MB file to a test repo, and `.git/index` only grew to 104 bytes. So it seems to only contain metadata. TIL.
    - coldpie 7y ago
      And if you go dig into `objects/`, you'll find your (possibly compressed) 100 MB object under its hash, and can view it with "git cat-file -p $hash" without ever having committed it :) Edit: And I bet if you dig around your index file enough, you'll find that hash someplace in there.
      - chimeracoder 7y ago
        > And if you go dig into `objects/`, you'll find your (possibly compressed) 100 MB object under its hash, and can view it with "git cat-file -p $hash" without ever having committed it :) > > Edit: And I bet if you dig around your index file enough, you'll find that hash someplace in there. And if you want to know how that works, there's an article in the next issue of Code Words that goes into that as well: https://codewords.recurse.com/issues/three/unpacking-git-packfiles https://codewords.recurse.com/issues/three/unpacking-git-pac...
  - coldpie 7y ago
    I admit I'm getting out of my knowledge here (it's been a while since I read gitcore-tutorial), but I think that's the "to-be-committed" commit object. So the index file "points to" the object file that stores the state of the object you added.
  - tux1968 7y ago
    The index doesn't contain objects themselves, every added file (whether it is committed or not) is held as a regular object. The index simply holds a list of object ID's. So yes, if you change a file and re-add it, the index will be overwritten and the original object will become dangling without any references to it.
frenchyatwork 7y ago
It's certainly an understandable point of confusion. It's not clear to me if the current behavior was actually intentional, or just an byproduct of implementation. If you add a file, then modify it, and then commit it, you're old version gets committed. That caused me a bit of confusion back in the day.
- boustrophedon 7y ago
  The behavior is pretty intentional - you can use `git add -p` explicitly to only add parts of a given file to the index.
- tonyedgecombe 7y ago
  A mistake I seem to make fairly often. That or not adding new files in the first place.
skrebbel 7y ago
The staging area^W^Windex^Wcache is a terribly designed mess, that's why. The best way to think about it is it's just a half-finished commit, i.e. one without a message and an author and a date. But otherwise git treats the index like any other commit. Adding stuff to the staging area is like amending that commit. Actually committing it is like amending that commit again, but without changing the files, only editing the metadata (message, author, etc). And then moving the current branch to it. You could totally simulate the cache by doing exactly this, i.e. a series of `git commit -a --amend` commands (just make sure you don't push halfway). The idea behind the staging area is that you obviously need this all the time, because reasons, so let's force you to go through the hassle for every commit you might want to make. Because it's just a commit that hates your guts, it has all the same side effects as making a real commit has.
- minitech 7y ago
  > The idea behind the staging area is that you obviously need this all the time, because reasons, so let's force you to go through the hassle for every commit you might want to make. I do need it all the time, and simulating it with commits would be horribly unsuited to getting work done. (The funny thing is that what you mentioned – `git commit -a` – doesn’t accomplish that, but it does skip the hassle you just said was forced.) It’s also a clean place to handle conflict resolution state, because that shouldn’t go in commits.
  - skrebbel 7y ago
    > (The funny thing is that what you mentioned – `git commit -a` – doesn’t accomplish that, but it does skip the hassle you just said was forced.) I wish it did, but it doesn't stage deletions, therefore also messing up renames, which is exactly never what you want.
    - minitech 7y ago
      It does commit deletions. It doesn’t commit untracked files. If you really want to track every new file unprompted, an alias can be made for `git add --all && git commit`.
james-skemp 7y ago
While confusing for some, I love this part of `git add`. While branching is already easy enough, I'll regularly get to a point where I may want to spend a few minutes going down a path. I'll either be happy where it leads me, or realize it was a bad idea and scrap it. I'll `git add` the current state, make the changes I want, test it out, and then either revert back to what's staged, or like where I'm at and `git add` the rest in. That and `git add -p` also mean that I rarely do a `git commit -a` or the like; stage it, then commit it.
- reificator 7y ago
  That sounds like the intended workflow of `git stash`.
  - akkartik 7y ago
    What I like about `git add` is that it creates a backup I can restore from in extremis but without cluttering up various dashboard views like `git status` or `git branch` or `git stash list`.
  - james-skemp 7y ago
    Then I did a bad job of explaining my workflow. :) Stash is 'I want to temporarily keep track of where I'm at and roll back to a previous state, likely so I can do something else.' You could do a commit/branch, but it's temporary/not finished. My add is 'I have stuff that I'm working on and want to try going in another direction for a bit; I'll either add it in if I like where I went, or go back to what I've staged.' Example from a few hours ago: We've lost the primary on a project that is doing ... interesting things with Grunt. The packages are three years out of date, which is what I'm tackling first. I'm opting to break these into commit based upon related groupings of packages. So, `npm upgrade package1`, test things out, then `git add package-lock.json && git add package.json -p`. Now I upgrade another package and after testing determine that this is a pretty significant change, even though it should have been easy. Since I haven't staged my last npm upgrade I can easily discard the changes and still have all of my `npm upgrade package1` modifications. Now I can choose to commit those staged changes or try a different, but related, package upgrade. Simple example, but easy to expand this to something that touches a handful or more of files. The alternative would be to commit each individual upgrade, or roll back the last upgrade/thing(s) you did. Another common use case is if I'm writing some code and realize I'd like to refactor a bit before I commit it. `git add` the working code, refactor, and if it's getting to look like it's a commit onto itself I can always `git commit` what I have staged, versus having to undo. (`git commit --amend` would work in this case too, but I do a lot of work with third-party code and am never quite sure if I'll want to keep something for historical purposes/an alternative way to do something.)

sdan 7y ago

How did they generate those nice looking graphs?

maryrosecook 7y ago
Hiya! Author of the article here. I used OmniGraffle.
- sdan 7y ago
  Interesting. Wish there was a way to auto-generate these graphs with my git history. I like them!
  - coldpie 7y ago
    Have you tried "gitk"?
    - drharby 7y ago
      I am personally embarrassed and excited to discover this just now. For that I thank you because this makes my life that much better.
  - tcoff91 7y ago
    Lots of git clients like Fork have pretty graphs for browsing your history. Even the terminal client can generate graphs in the git log if you give it the right params.
  - fit2rule 7y ago
    I regularly use 'gource' to get a lovely picture of my repos: https://gource.io https://gource.io
- tomca32 7y ago
  Hey Mary, Tomislav here from the 2013 summer batch. Just wanted to let you know it was an absolute pleasure learning from you at HS. You're a wonderful teacher and I learned a ton. Thanks!

dang 7y ago

A thread from 2016: https://news.ycombinator.com/item?id=12802949 https://news.ycombinator.com/item?id=12802949 A bit from 2015: https://news.ycombinator.com/item?id=9793069 https://news.ycombinator.com/item?id=9793069 Discussed at the time: https://news.ycombinator.com/item?id=9272249 https://news.ycombinator.com/item?id=9272249

shadykiller 7y ago

I gave a similar talk “Inside Git Guts with Ruby” at RubyConf India 2013 - https://m.youtube.com/watch?v=lPlwkxrG2NM https://m.youtube.com/watch?v=lPlwkxrG2NM I had to learn a lot of git internals and it was super fun