11 ms·

Write yourself a Git (2018)

684 points by adamnemecek 8y ago

nukeop 8y ago
If you find these kinds of "build your own X" articles interesting, there's a repo on Github that aggregates them, sorted in various categories: https://github.com/danistefanovic/build-your-own-x https://github.com/danistefanovic/build-your-own-x
- jyriand 8y ago
  I think I found my learning projects for the next 10 years.
- kiuraalex 8y ago
  Thank you!
bibyte 8y ago
This is such a great tutorial to learn Git from the bottom up. I always thought the "back end" part of the Git is pretty complex but this tutorial makes it look so easy.
deleted 8y ago
[deleted]
blastbeat 8y ago
Seems to be a great opportunity to learn some Git and Python at the same time. I like those kind of approaches very much. Thanks!
dakra 8y ago
I recommend "Git from the bottom up"[1] for a nice and relatively simple tutorial that shows you the inner design of a git repository. [1] https://jwiegley.github.io/git-from-the-bottom-up/ https://jwiegley.github.io/git-from-the-bottom-up/
mcbain 8y ago
Just be aware that it is a good start but isn’t as complete as the command list would indicate - runs up to about the point of index files, so not quite at ‘git add’.
seleniumBubbles 8y ago
This is great, thanks for sharing. People in this thread might also appreciate this essay: https://maryrosecook.com/blog/post/git-in-six-hundred-words https://maryrosecook.com/blog/post/git-in-six-hundred-words And the more expanded version: https://maryrosecook.com/blog/post/git-from-the-inside-out https://maryrosecook.com/blog/post/git-from-the-inside-out It really helped me comprehend Git enough to start understanding the more complex work flows.
- JustSomeNobody 8y ago
  I really enjoy the way MRC explains topics. I, too, would recommend her essay.
zadwang 8y ago
This is simply excellent. I already know some basics of the internals of git at a conceptual level but this tutorial makes the knowledge so much more concrete. Wonderful.
Milank 8y ago
I always thought the best way to learn something is by doing it. The only problem now is the time.
ssivark 8y ago
I have nothing to directly comment on the tutorial. Just a tangential mention regarding the tedious argument parsing boilerplate in Python, I have found Python Fire to be much more convenient: https://github.com/google/python-fire https://github.com/google/python-fire It would have shaved off another 15-20 lines from the 503 line example ;-)
- lapinot 8y ago
  Yeah, i couldn't help but notice the first piece of code is an ugly "switch case". There's a python idiom for this, it's putting your functions in a dictionary and doing something like `cmds_dict.get(args.command, default)(args)`. I guess we all have our religious habits for argument parsing (more of a docopt-er myself).
  - rob74 8y ago
    I guess the author wanted the code to also look friendly to people who are not that familiar with Python - everybody understands a switch, while what you are describing would probably puzzle some people...
- bibyte 8y ago
  Python Fire looks much more concise. But do you know of any other languages that handles argument parsing better then Python ?
  - agigao 8y ago
    Clojure.
    - bibyte 8y ago
      Can you please post a hello world example ?
  - vbsteven 8y ago
    Installed by default I cannot find any. But pretty much any language with a package manager has sane and easy to use libraries to bang out a small CLI app with subcommands and option parsing in minutes. For example PicoCli in Java, Thor in Ruby and Clap in Rust.
MordodeMaru 8y ago
It is indeed a great (and time consuming approach). Other VCSs have gone a little further than Git though, simplifying their workflows or the GUIs and CLI commands to make their use more intuitive like https://www.plasticscm.com/ https://www.plasticscm.com/ and others.
ansible 8y ago
It might be fun do try this in Rust as well.
- Vogtinator 8y ago
  Or in Haskell. Or in C++.
chrislo 8y ago
Folks interested in this may also be interested in a new book from James Coglan "Building Git": https://shop.jcoglan.com/building-git/ https://shop.jcoglan.com/building-git/ It also takes a ground-up "build it yourself" approach and has tons of interesting detail.
speter 8y ago
This free ebook is also a good resource if you're just starting with Git: https://www.git-tower.com/learn/git/ebook/en/command-line/introduction https://www.git-tower.com/learn/git/ebook/en/command-line/in... There is a video course as well: https://www.git-tower.com/learn/git/videos#episodes https://www.git-tower.com/learn/git/videos#episodes
Chia 8y ago
Good job, thanks for sharing. I'm just written a simple git dumper tool (https://github.com/owenchia/githack https://github.com/owenchia/githack) a few days ago. Learn by doing is a very good way and I really enjoy it.
cabalamat 8y ago
I have an admission to make: I don't understand git. By this I mean I have a few simple commands I use (status/add/commit/push/pull) and if I try to do anything more complicated it always ends up with lots of complex error messages that I don't understand and me nuking the repository and starting again. So I think: there must be a better way. I have often thought about implementing a VCS. The idea behind one doesn't seem particularly complex to me (certainly it's simpler than programming languages). If I did I would quite probably use WYAG as a starting point. My first step would be to define the user's mental model -- i.e. what concepts they need to understand such that they can predict what the system will do. Then I would build a web-based UI that presents the status of the system to the user in terms of that model.
- cmroanirgo 8y ago
  I'm like you. I use SourceTree to get a 'visual grasp' on what I find to be the noise of git commands. However, if you're into command line, you can try fossil: it's got lots going for it. Your idea of a "user's mental model" might land you into trouble though, because all of us come from different backgrounds (subversion, SSafe, git, HG...) and they all maddeningly redefine terms in different ways (eg branch, forks, commits, checkout).
  - cabalamat 8y ago
    > Your idea of a "user's mental model" might land you into trouble though If I do do this, I will explicitly lay out the user's metal model in the documentation at the start. Then it will be the user's fault if they can't be bothered to read it.
    - huffstler 8y ago
      While gits documentation might not lay out the model at the start, they certainly have documentation showing how everything works, complete with diagrams and such. As such... > Then it will be the user's fault if they can't be bothered to read it. can be applied to you about git technically. I think that's just me being a "little" pedantic though. It sounds like the issue you actually have is that the documentation isn't easily readable in one or two sittings, and you don't have the time (or can't be bothered) to go through it and learn it. Which I totally understand, everyone has different things they need to spend time on, most of the time learning Git isn't one of them.
driusan 8y ago
Having written my own git client, I can tell you that "the most complicated part will be the command-line arguments parsing logic" doesn't go away. I wouldn't be surprised to wake up one day and find someone published a proof that NP != P, and the proof involved trying to parse the git command line.
- Jupe 8y ago
  Case in point: https://git-man-page-generator.lokaltog.net/ https://git-man-page-generator.lokaltog.net/ :)
  - peteradio 8y ago
    lol --mumble-hire-file
  - driusan 8y ago
    Doesn't seem that realistic. I reloaded a bunch of times, and didn't get a single one where the same command can mean 3 different context-sensitive things that each take different arguments but are all named the same command.
  - skocznymroczny 8y ago
    git-massage-head - massage some non-cleaned remote heads next to various imported remotes
dprophecyguy 8y ago
Does any body know any more write your own type tutorials for any other projects ? Please point them out here
- inetsee 8y ago
  Doing a search on HN for "write your own" returns a lot of answers, including 'Ask HN: “Write your own” or “Build your own” software projects' https://news.ycombinator.com/item?id=16591918 https://news.ycombinator.com/item?id=16591918 from a year ago.
sransara 8y ago
Thanks for sharing. I too agree on this: "... Git is complex is, in my opinion, a misconception... But maybe what makes Git the most confusing is the extreme simplicity and power of its core model. The combination of core simplicity and powerful applications often makes thing really hard to grasp..." If I may do a self plug, I had recently written a note on "Build yourself a DVCS (just like Git)"[0]. The note is an effort on discussing reasoning for design decisions of the Git internals, while conceptually building a Git step by step. [0] https://s.ransara.xyz/notes/2019/build-yourself-a-distributed-version-control-system-just-like-git/ https://s.ransara.xyz/notes/2019/build-yourself-a-distribute...
- linkmotif 8y ago
  What makes it hard is that it’s taught wrong. All this pull/checkout/commit/push whereas for me it took a long time to discover that fetch/rebase/show-branch/reset/checkout —amend, and especially the interactive -p variants, are the core tools that really make it a pleasure to use. They give you flexibility and let you write and rewrite your story, whereas the commands you’re introduced with provide no control to the user. It’s remarkable the number of users who think you can’t rewrite a Git branch.
  - ectospheno 8y ago
    The rebase command is only safe if you never share a branch. Most people use a distributed revision control system to work with others and if you do work with others then rebase is dangerous and should not be used. Almost every rebase user I've spoken with has no idea what the danger is despite it being clearly discussed in the manual page for rebase and despite rebase being listed as dangerous every time it is mentioned in any manual page. For the sake of new users everywhere, please stop recommending rebase.
    - anyzen 8y ago
      I strongly disagree. Git rebase is a very useful command, but of course you need to know what you are doing. Git becomes very weird if you try to avoid its core concepts... as far as I'm concerned, use rebase, get yourself cut (or better yet, learn about it before using it) and try not to repeat mistakes. That said, git porcelain is simply awful. It is inconsistent and full of dangers - there is no way I would dare try new commands without reading up on them, because the names are often misleading. Sometimes I really wish GitHub / GitLab used Mercurial as their foundation. I think the world of programming would be much easier....
asdkhadsj 8y ago
On the note of Git being difficult, I'm really curious to see if Pijul[1] ends up being easier to understand than Git. [1]: https://pijul.org https://pijul.org
- thomascgalvin 8y ago
  I really like the way Pijul "thinks". Unfortunately, I can't see myself using it (or Fossil, for that matter) for anything except toy code, because I have contract requirements to store everything in GitLab.
- nemetroid 8y ago
  From my limited knowledge (mostly based on jneems article series[1]), I think Pijul is more powerful, but for the same reason also considerably more difficult to understand than Git. In particular, Pijul supports (and depends on) working with repository states that are, in Git terms, not fully resolved. In addition, those states are potentially very difficult to even represent as flat files (see e.g. [2]). Git is simpler in that it mandates that each commit represents a fully valid filesystem state. That said, I still think Pijul might have a place, if it turns out that it supports superior workflows that aren't possible in Git. But the "VCS elitism" would probably become worse than it is today. [1]: https://jneem.github.io/merging/ https://jneem.github.io/merging/ [2]: https://jneem.github.io/cycles/ https://jneem.github.io/cycles/
vkaku 8y ago
Great! Thank you for writing an iterative one.
raju 8y ago
If it helps anyone, I wrote a two-parter on Git internals a while back https://looselytyped.com/blog/2014/08/31/gits-guts-part-i/ https://looselytyped.com/blog/2014/08/31/gits-guts-part-i/ https://looselytyped.com/blog/2014/10/31/gits-guts-part-ii/ https://looselytyped.com/blog/2014/10/31/gits-guts-part-ii/ Disclaimer — This is my blog Update - Fixed formatting / Clarified post
andrewshadura 8y ago
A bit of a plug, I know, but here’s a tool I wrote to make selective committing a bit easier and more interactive: https://github.com/andrewshadura/git-crecord/ https://github.com/andrewshadura/git-crecord/ Those familiar with Mercurial will surely notice it is, in fact, a port of a Mercurial’s interactive commit functionality, previously a separate extension called crecord.

684 points by adamnemecek 8y ago

If you find these kinds of "build your own X" articles interesting, there's a repo on Github that aggregates them, sorted in various categories: https://github.com/danistefanovic/build-your-own-x https://github.com/danistefanovic/build-your-own-x

jyriand 8y ago
I think I found my learning projects for the next 10 years.
kiuraalex 8y ago
Thank you!

bibyte 8y ago

This is such a great tutorial to learn Git from the bottom up. I always thought the "back end" part of the Git is pretty complex but this tutorial makes it look so easy.

deleted 8y ago

[deleted]

blastbeat 8y ago

Seems to be a great opportunity to learn some Git and Python at the same time. I like those kind of approaches very much. Thanks!

dakra 8y ago

I recommend "Git from the bottom up"[1] for a nice and relatively simple tutorial that shows you the inner design of a git repository. [1] https://jwiegley.github.io/git-from-the-bottom-up/ https://jwiegley.github.io/git-from-the-bottom-up/

mcbain 8y ago

Just be aware that it is a good start but isn’t as complete as the command list would indicate - runs up to about the point of index files, so not quite at ‘git add’.

seleniumBubbles 8y ago

This is great, thanks for sharing. People in this thread might also appreciate this essay: https://maryrosecook.com/blog/post/git-in-six-hundred-words https://maryrosecook.com/blog/post/git-in-six-hundred-words And the more expanded version: https://maryrosecook.com/blog/post/git-from-the-inside-out https://maryrosecook.com/blog/post/git-from-the-inside-out It really helped me comprehend Git enough to start understanding the more complex work flows.

JustSomeNobody 8y ago
I really enjoy the way MRC explains topics. I, too, would recommend her essay.

zadwang 8y ago

This is simply excellent. I already know some basics of the internals of git at a conceptual level but this tutorial makes the knowledge so much more concrete. Wonderful.

Milank 8y ago

I always thought the best way to learn something is by doing it. The only problem now is the time.

ssivark 8y ago

I have nothing to directly comment on the tutorial. Just a tangential mention regarding the tedious argument parsing boilerplate in Python, I have found Python Fire to be much more convenient: https://github.com/google/python-fire https://github.com/google/python-fire It would have shaved off another 15-20 lines from the 503 line example ;-)

lapinot 8y ago
Yeah, i couldn't help but notice the first piece of code is an ugly "switch case". There's a python idiom for this, it's putting your functions in a dictionary and doing something like `cmds_dict.get(args.command, default)(args)`. I guess we all have our religious habits for argument parsing (more of a docopt-er myself).
- rob74 8y ago
  I guess the author wanted the code to also look friendly to people who are not that familiar with Python - everybody understands a switch, while what you are describing would probably puzzle some people...
bibyte 8y ago
Python Fire looks much more concise. But do you know of any other languages that handles argument parsing better then Python ?
- agigao 8y ago
  Clojure.
  - bibyte 8y ago
    Can you please post a hello world example ?
- vbsteven 8y ago
  Installed by default I cannot find any. But pretty much any language with a package manager has sane and easy to use libraries to bang out a small CLI app with subcommands and option parsing in minutes. For example PicoCli in Java, Thor in Ruby and Clap in Rust.

MordodeMaru 8y ago

It is indeed a great (and time consuming approach). Other VCSs have gone a little further than Git though, simplifying their workflows or the GUIs and CLI commands to make their use more intuitive like https://www.plasticscm.com/ https://www.plasticscm.com/ and others.

ansible 8y ago

It might be fun do try this in Rust as well.

Vogtinator 8y ago
Or in Haskell. Or in C++.

chrislo 8y ago

Folks interested in this may also be interested in a new book from James Coglan "Building Git": https://shop.jcoglan.com/building-git/ https://shop.jcoglan.com/building-git/ It also takes a ground-up "build it yourself" approach and has tons of interesting detail.

speter 8y ago

This free ebook is also a good resource if you're just starting with Git: https://www.git-tower.com/learn/git/ebook/en/command-line/introduction https://www.git-tower.com/learn/git/ebook/en/command-line/in... There is a video course as well: https://www.git-tower.com/learn/git/videos#episodes https://www.git-tower.com/learn/git/videos#episodes

Chia 8y ago

Good job, thanks for sharing. I'm just written a simple git dumper tool (https://github.com/owenchia/githack https://github.com/owenchia/githack) a few days ago. Learn by doing is a very good way and I really enjoy it.

cabalamat 8y ago

I have an admission to make: I don't understand git. By this I mean I have a few simple commands I use (status/add/commit/push/pull) and if I try to do anything more complicated it always ends up with lots of complex error messages that I don't understand and me nuking the repository and starting again. So I think: there must be a better way. I have often thought about implementing a VCS. The idea behind one doesn't seem particularly complex to me (certainly it's simpler than programming languages). If I did I would quite probably use WYAG as a starting point. My first step would be to define the user's mental model -- i.e. what concepts they need to understand such that they can predict what the system will do. Then I would build a web-based UI that presents the status of the system to the user in terms of that model.

cmroanirgo 8y ago
I'm like you. I use SourceTree to get a 'visual grasp' on what I find to be the noise of git commands. However, if you're into command line, you can try fossil: it's got lots going for it. Your idea of a "user's mental model" might land you into trouble though, because all of us come from different backgrounds (subversion, SSafe, git, HG...) and they all maddeningly redefine terms in different ways (eg branch, forks, commits, checkout).
- cabalamat 8y ago
  > Your idea of a "user's mental model" might land you into trouble though If I do do this, I will explicitly lay out the user's metal model in the documentation at the start. Then it will be the user's fault if they can't be bothered to read it.
  - huffstler 8y ago
    While gits documentation might not lay out the model at the start, they certainly have documentation showing how everything works, complete with diagrams and such. As such... > Then it will be the user's fault if they can't be bothered to read it. can be applied to you about git technically. I think that's just me being a "little" pedantic though. It sounds like the issue you actually have is that the documentation isn't easily readable in one or two sittings, and you don't have the time (or can't be bothered) to go through it and learn it. Which I totally understand, everyone has different things they need to spend time on, most of the time learning Git isn't one of them.

driusan 8y ago

Having written my own git client, I can tell you that "the most complicated part will be the command-line arguments parsing logic" doesn't go away. I wouldn't be surprised to wake up one day and find someone published a proof that NP != P, and the proof involved trying to parse the git command line.

Jupe 8y ago
Case in point: https://git-man-page-generator.lokaltog.net/ https://git-man-page-generator.lokaltog.net/ :)
- peteradio 8y ago
  lol --mumble-hire-file
- driusan 8y ago
  Doesn't seem that realistic. I reloaded a bunch of times, and didn't get a single one where the same command can mean 3 different context-sensitive things that each take different arguments but are all named the same command.
- skocznymroczny 8y ago
  git-massage-head - massage some non-cleaned remote heads next to various imported remotes

dprophecyguy 8y ago

Does any body know any more write your own type tutorials for any other projects ? Please point them out here

inetsee 8y ago
Doing a search on HN for "write your own" returns a lot of answers, including 'Ask HN: “Write your own” or “Build your own” software projects' https://news.ycombinator.com/item?id=16591918 https://news.ycombinator.com/item?id=16591918 from a year ago.

sransara 8y ago

Thanks for sharing. I too agree on this: "... Git is complex is, in my opinion, a misconception... But maybe what makes Git the most confusing is the extreme simplicity and power of its core model. The combination of core simplicity and powerful applications often makes thing really hard to grasp..." If I may do a self plug, I had recently written a note on "Build yourself a DVCS (just like Git)"[0]. The note is an effort on discussing reasoning for design decisions of the Git internals, while conceptually building a Git step by step. [0] https://s.ransara.xyz/notes/2019/build-yourself-a-distributed-version-control-system-just-like-git/ https://s.ransara.xyz/notes/2019/build-yourself-a-distribute...

linkmotif 8y ago
What makes it hard is that it’s taught wrong. All this pull/checkout/commit/push whereas for me it took a long time to discover that fetch/rebase/show-branch/reset/checkout —amend, and especially the interactive -p variants, are the core tools that really make it a pleasure to use. They give you flexibility and let you write and rewrite your story, whereas the commands you’re introduced with provide no control to the user. It’s remarkable the number of users who think you can’t rewrite a Git branch.
- ectospheno 8y ago
  The rebase command is only safe if you never share a branch. Most people use a distributed revision control system to work with others and if you do work with others then rebase is dangerous and should not be used. Almost every rebase user I've spoken with has no idea what the danger is despite it being clearly discussed in the manual page for rebase and despite rebase being listed as dangerous every time it is mentioned in any manual page. For the sake of new users everywhere, please stop recommending rebase.
  - anyzen 8y ago
    I strongly disagree. Git rebase is a very useful command, but of course you need to know what you are doing. Git becomes very weird if you try to avoid its core concepts... as far as I'm concerned, use rebase, get yourself cut (or better yet, learn about it before using it) and try not to repeat mistakes. That said, git porcelain is simply awful. It is inconsistent and full of dangers - there is no way I would dare try new commands without reading up on them, because the names are often misleading. Sometimes I really wish GitHub / GitLab used Mercurial as their foundation. I think the world of programming would be much easier....

asdkhadsj 8y ago

On the note of Git being difficult, I'm really curious to see if Pijul[1] ends up being easier to understand than Git. [1]: https://pijul.org https://pijul.org

thomascgalvin 8y ago
I really like the way Pijul "thinks". Unfortunately, I can't see myself using it (or Fossil, for that matter) for anything except toy code, because I have contract requirements to store everything in GitLab.
nemetroid 8y ago
From my limited knowledge (mostly based on jneems article series[1]), I think Pijul is more powerful, but for the same reason also considerably more difficult to understand than Git. In particular, Pijul supports (and depends on) working with repository states that are, in Git terms, not fully resolved. In addition, those states are potentially very difficult to even represent as flat files (see e.g. [2]). Git is simpler in that it mandates that each commit represents a fully valid filesystem state. That said, I still think Pijul might have a place, if it turns out that it supports superior workflows that aren't possible in Git. But the "VCS elitism" would probably become worse than it is today. [1]: https://jneem.github.io/merging/ https://jneem.github.io/merging/ [2]: https://jneem.github.io/cycles/ https://jneem.github.io/cycles/

vkaku 8y ago

Great! Thank you for writing an iterative one.

raju 8y ago

If it helps anyone, I wrote a two-parter on Git internals a while back https://looselytyped.com/blog/2014/08/31/gits-guts-part-i/ https://looselytyped.com/blog/2014/08/31/gits-guts-part-i/ https://looselytyped.com/blog/2014/10/31/gits-guts-part-ii/ https://looselytyped.com/blog/2014/10/31/gits-guts-part-ii/ Disclaimer — This is my blog Update - Fixed formatting / Clarified post

andrewshadura 8y ago

A bit of a plug, I know, but here’s a tool I wrote to make selective committing a bit easier and more interactive: https://github.com/andrewshadura/git-crecord/ https://github.com/andrewshadura/git-crecord/ Those familiar with Mercurial will surely notice it is, in fact, a port of a Mercurial’s interactive commit functionality, previously a separate extension called crecord.