11 ms·

Introducing Git protocol version 2

547 points by robmaceachern 8y ago

GauntletWizard 8y ago
Interesting that they took to the Google blog to announce this; is there a corresponding LKML post?
- jkaplowitz 8y ago
  Why LKML? Despite Git's origins from and use by the Linux project, it isn't especially tied to it now. LKML would presumably be the place for Linux to announce when they adopt this. The Google open source blog is among the several credible options for this post, since Google employs much of the core Git team, and this post discusses their experience deploying Git protocol v2 at Google. As noted in the blog text, it's not in a released version of Git yet, just Git master branch. So maybe it'll appear on a dedicated Git announcement list, if any, once that happens.
  - joatmon-snoo 8y ago
    Junio posts on the list when there's a new release, e.g. https://public-inbox.org/git/xmqqwoxw6kkk.fsf@gitster-ct.c.googlers.com/ https://public-inbox.org/git/xmqqwoxw6kkk.fsf@gitster-ct.c.g... It seems that https://groups.google.com/forum/#!forum/git-packagers https://groups.google.com/forum/#!forum/git-packagers is the closest thing to a formal announcement list that there is.
    - jkaplowitz 8y ago
      Okay, I guess that tie continues for historical reasons. At least people who don't otherwise subscribe to LKML can still receive Git release announcements via the second link. I presume Git 2.18 (the first release supporting protocol v2) will be announced via both channels once it's out.
- Analemma_ 8y ago
  > support for v2 was recently merged to Git's master branch and is expected to be part of Git 2.18 Not yet, but presumably there will be a post like this: https://lkml.org/lkml/2018/4/2/425 https://lkml.org/lkml/2018/4/2/425 when it is released. It is strange that the Google Blog is the first place to announce it through.
- tgummerer 8y ago
  As mentioned in another comment, protocol v2 was implemented by a Google employee, and they decided to write a blog post about it. This is not an official git announcement.
- u801e 8y ago
  I found a mention of it in a "what's cooking" post on the git mailing list (Message-ID <xmqqvabm6csb.fsf@gitster-ct.c.googlers.com>). But I can't find a direct link on gmane.com right now.
ksec 8y ago
I thought google uses hg, have they switched over to git as well?
- seabrookmx 8y ago
  For all the "big" Google projects they use a proprietary system called piper. I think all their open-source stuff (Angular, GoLang, Android) uses git (and sometimes Gerrit). Although given Google's scale, I'm sure there's some teams/projects that use Mercurial.
  - pjmlp 8y ago
    Go started on Mercurial and then eventually moved into Git.
    - jordigh 8y ago
      And they like neither. They really want a versioned filesystem.
      - dilap 8y ago
        That's interesting. Do you have more details / refs?
        jordigh 8y ago
        I understand this was private communication from Rob and Russ.
        dilap 8y ago
        well dang! :-)
  - ngoldbaum 8y ago
    In fact, developers are allowed to use whichever VCS tool they want on their local machine (or on the online coding in the cloud CitC environment). Some opt to use hg. The canonical repo is in piper though, so the hg commits or git commits get converted before they land.
Ericson2314 8y ago
This is disgusting. So little forsight in the past... At least the outcome of quite useful.
- needz 8y ago
  IIRC, they were in a serious time crunch when they drafted/made git. I can't remember the whole story...
  - mc42 8y ago
    I thought Linus Torvalds was almost wholly responsible for the initial development of git? Even so, everything, especially software, is easier in hindsight....
    - needz 8y ago
      Sorry, I was using the 'singular' they.
  - tytso 8y ago
    At the time Linus was the sole author/contributor of git, and he needed a replacement for BitKeeper in a hurry. BitKeeper had been made unavailable for Linux kernel development because the proprietor of BitKeeper was really unhappy that Tridge had reverse engineered the protocol and created an open-source client[1] which could talk to the Bitkeeper server. Linus created the first version of git sufficient to do a kernel commit in ten days[2]. [1] https://sourceforge.net/projects/sourcepuller/ https://sourceforge.net/projects/sourcepuller/ [2] https://www.linuxfoundation.org/blog/10-years-of-git-an-interview-with-git-creator-linus-torvalds/ https://www.linuxfoundation.org/blog/10-years-of-git-an-inte... (As Tridge tells the story[3], he telnet'ed to the bk port and typed "help" so it wasn't that much of a reverse engineering effort. :-) [3] https://lwn.net/Articles/132938/ https://lwn.net/Articles/132938/
  - wereHamster 8y ago
    s/they/he/. It was Linus himself who alone created git, within a few weeks (two or three). At first it was just a handful of shell scripts, but it was self-hosting pretty early on.
gpvos 8y ago
Git didn't have a proper version number or extensibility field in its protocol? That's quite a bit of hubris.
- zeroxfe 8y ago
  Or, more likely, an oversight.
  - gpvos 8y ago
    Hmm, I haven't designed very many data formats or wire protocols, and I won't claim I got it right any of those times, but I included some kind of extension possibility every time.
    - gchpaco 8y ago
      I damn near released a (private) message protocol without a version field a couple months ago, and I know better. Fortunately stopped myself and added it before any actual data got released.
    - shakna 8y ago
      Git was a 10-day urgent project. Given the timeframe, it's done remarkably well.
buckminster 8y ago
AIUI, the git ssh protocol is just the git protocol tunnelled through ssh. So why do they need different mechanisms for signalling V2?
- _wmd 8y ago
  Deploying Git over SSH entails locking the precise command line executable by the public key you use to authenticate. Locking SSH SendEnv down is mandatory too, otherwise thousands of people would have shell access to GitHub.com! This isn't even theoretical, there was an environment-related bug not 5 years ago involving Git. At least BitBucket was impacted, I think GitHub were patched before it was announced
  - simias 8y ago
    I don't think that answers the parent's question, if the update was in the git protocol itself (encapsulated in the SSH session) then you wouldn't have to change anything at the SSH level. As you point out selectively allowing a new environment variable could open a can of worms for shared hosts like github if they mess up their implementation.
- xyzzyz 8y ago
  Because if you tunnel through ssh, you can signal v2 using ssh mechanism of setting environment variables. If you don't tunnel, you don't have this option. This is clearly described in the article.
  - buckminster 8y ago
    Yes, but once you've updated the git protocol, ssh support comes for free. Having one mechanism is simpler than having two. And as your sibling notes, setting env vars from ssh has disadvantages. So why bother?
  - deathanatos 8y ago
    I think what the person you're replying to is asking is why not, in the case of ssh, use the signaling in the git protocol, since it will be there anyways. That is, if you don't tunnel, you must signal w/ the git protocol. If you do tunnel, why use a different mechanism, since the signal in the git protocol must be there? I think that this is because the SSH protocol isn't just encapsulating the Git protocol directly (the initial assumption of ssh "just" encapsulating the git protocol is not fully correct), and one of the parts that differs is this particular part. (Since on the git protocol side, we need to select a "service": > a single packet-line which includes the requested service (git-upload-pack for fetches and git-receive-pack for pushes) which in SSH would be done not by transmitting that packet-line but by instructing SSH to run that particular executable. > This is clearly described in the article. It really isn't, IMO; if you don't have precise knowledge of the protocols involved, I don't think anything in the article particularly spells this out.
cpburns2009 8y ago
Wait, why was this posted by Google? I thought Git was made by Linus Torvalds.
- Semaphor 8y ago
  > Linus Torvalds said in 2012 that one of his own biggest successes was recognizing how good a developer Hamano was on Git, and trusting him to maintain it.
- hk__2 8y ago
  Git was created by Linus Torvalds, but out of the 50k+ commits on the repo, only 250 or so are from him, with only 14 in the past 6 years. [1] [1] https://github.com/git/git/graphs/contributors?from=2012-03-11&to=2018-05-10&type=c https://github.com/git/git/graphs/contributors?from=2012-03-...
  - sdesol 8y ago
    Here is a more detailed analysis, which shows all contributors: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26t%3Dcontributors%26rg%3Dhistory https://public.gitsense.com/insight/github?r=git/git#b%3Dgit... These are contributions by Linus: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26q%3Dauthor%3Atorvalds%40g5.osdl.org%5E_%5Eauthor%3Atorvalds%40linux-foundation.org%5E_%5Eauthor%3Atorvalds%40osdl.org%5E_%5Eauthor%3Atorvalds%40ppc970.osdl.org%5E_%5Ewindow%3A10000%26t%3Dcommits https://public.gitsense.com/insight/github?r=git/git#b%3Dgit... and as you can see, his contributions, really tapered off after 2010, while contributions from Hamano remained steady from 2008 to present date, as shown below: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26q%3Dauthor%3Agitster%40pobox.com%5E_%5Ewindow%3A10000%26t%3Dcommits0 https://public.gitsense.com/insight/github?r=git/git#b%3Dgit....
- euyyn 8y ago
  Opensource though.
- tgummerer 8y ago
  It's because a Google employee implemented protocol v2, and wrote a post about it.
oneeyedpigeon 8y ago
The current (and pretty much only, ever, despite Linus having been the creator) maintainer of git is a google employee [1], in case anyone else was wondering. [1] https://en.m.wikipedia.org/wiki/Junio_Hamano https://en.m.wikipedia.org/wiki/Junio_Hamano
- Retroity 8y ago
  Non-Mobile link for those on desktop: https://en.wikipedia.org/wiki/Junio_Hamano https://en.wikipedia.org/wiki/Junio_Hamano
  - Cynddl 8y ago
    The mobile version of Wikipedia works perfectly fine in a browser. I personally prefer it for readability.
    - esaym 8y ago
      I don't...
    - tinus_hn 8y ago
      If you link the normal desktop version it the reader will automatically be redirected to the place they prefer.
- chrisseldo 8y ago
  >"Linus Torvalds said in 2012 that one of his own biggest successes was recognizing how good a developer Hamano was on Git, and trusting him to maintain it."
  - vga805 8y ago
    wow! what an accolade
    - Dragonai 8y ago
Someone1234 8y ago
Too bad they didn't make Git LFS part of Version 2[0]. Most vendors[2] support LFS already but because it isn't required, some still lack it and its support cannot be assumed. [0] https://git-lfs.github.com/ https://git-lfs.github.com/ [1] https://github.com/git-lfs/git-lfs/wiki/Implementations https://github.com/git-lfs/git-lfs/wiki/Implementations
- xenomachina 8y ago
  > but because it is required Just to confirm, but you meant "because it is not required", right?
  - Someone1234 8y ago
    Right. Edited.
- 49bc 8y ago
  I wouldn't be so sure. They said one of the motivations was to "unblocking the path to more wire protocol improvements in the future".
- icebraining 8y ago
  I say that's LFS' fault. Why do you even need a custom server? It should just be able to use any ol' file server or S3-API compatible service, and do everything on the client side. I find git-annex a much better solution, it's a shame everyone went with LFS.
  - rspeer 8y ago
    My experience with git-annex is that it seems heavily designed for individuals and not for projects. The places it looks for files are often just a computer you were once developing on, and it sometimes expects you to go find that computer. It never forgets about any crazy place your files have been. It was very hard to use in asymmetric cases where different people have different credentials, such as where one person has access to a computer and others don't, or where a couple of core developers have authenticated R/W access to a file server or an S3 bucket and everyone else just has HTTP.
simias 8y ago
Let that be a reminder to all the coders out there: if you ever design a protocol or file format to communicate between machines always remember to add a version field or some other way to allow for updates and revisions later without breaking everything. Having a way to specify extensions in a backward-compatible way is nice too.
- cesarb 8y ago
  > if you ever design a protocol or file format to communicate between machines always remember to add a version field or some other way to allow for updates and revisions later without breaking everything Also, somehow make sure no servers, clients, or third-party middleboxes break when the version field is incremented. The TLS protocol designers had to give up on the version field; it's now going to forever be stuck at "TLS 1.2", since too much would break otherwise.
  - icebraining 8y ago
    It's an universal truth: if you want to keep something from jamming up, you need to exercise it. It's true for the human body, for machine parts and for protocol features.
    - stingraycharles 8y ago
      HTTP was at 1.1 for a very long time but it appears the upgrade to version 2 is going fine. What is the difference here? The protocol version exchange mechanism?
      - sp332 8y ago
        Because it's up to the client to request HTTP 2 if they support it. https://http2.github.io/http2-spec/#discover-http https://http2.github.io/http2-spec/#discover-http
        stingraycharles 8y ago
Confiks 8y ago
It's quite a comedy that this feature has not been implemented for at least 6 years, solely because the raw git:// protocol's parameter handling was severely broken, and feature detection by disconnecting and retrying [1] was ultimately deemed far too dirty. [1] https://public-inbox.org/git/CAJo=hJtZ_8H6+kXPpZcRCbJi3LPuuF7M1U8YsjAp-iWvut9oMw@mail.gmail.com/ https://public-inbox.org/git/CAJo=hJtZ_8H6+kXPpZcRCbJi3LPuuF...
avar 8y ago
The specification of the v2 protocol is here: https://github.com/git/git/blob/master/Documentation/technical/protocol-v2.txt https://github.com/git/git/blob/master/Documentation/technic... One of the more exciting things is that it can now be extended to arbitrary new over-the-wire commands. So e.g. "git grep" could be made to execute over the network if that's more efficient in some cases. This will also allow for making things that now use side-transports part of the protocol itself if it made sense. E.g. the custom commands LFS and git-annex implement, and even more advanced things like shipping things like the new commit graph already generated from the server to the client.
- xorcist 8y ago
  If you are to link to a git repo, don't link to some unoffical mirror. That would just confuse search engines. The specificiation of the v2 protocol is here: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/technical/protocol-v2.txt https://git.kernel.org/pub/scm/git/git.git/tree/Documentatio... (There are a couple of repos listed as official mirrors, such as the googlesource.com one, but the one you linked to isn't one of them.)
  - avar 8y ago
    The repository I linked to is official. See https://public-inbox.org/git/xmqqindt6g1r.fsf@gitster.mtv.corp.google.com/ https://public-inbox.org/git/xmqqindt6g1r.fsf@gitster.mtv.co... What list are you referring to? If it doesn't list the one on GitHub it needs to be fixed.
    - xorcist 8y ago
      I had a release announcement open for other reasons and noted the different URLs, but I was wrong and the link is absolutely fine. Thanks. If I could edit the post I would, so I'll just let this sit here in case anyone is confused (as I was).
rwmj 8y ago
Is there a git protocol variant that allows the client to avoid downloading objects that it already has stored locally in another repository or cache? For example: I have the Linux kernel already cloned in some directory. I clone a second repo which has the Linux kernel as a submodule. Can I clone the second repo straightforwardly without having to download Linux a second time? (Well yes, but only by manual intervention before doing the git submodule update - it'd be nice if objects could be shared in a cache across also repos somehow).
- andrewaylett 8y ago
  I'm assuming from your comment that you're already aware of --reference but it doesn't completely meet your needs? The only other thing I can think of would be to use the 'insteadOf' configuration to tell Git to use your local clone instead of the remote one. Search 'git help config' for 'url.<base>.insteadOf'.
- Boulth 8y ago
  Have you seen alternates? https://stackoverflow.com/questions/36123655/what-is-the-git-alternates-mechanism#36123844 https://stackoverflow.com/questions/36123655/what-is-the-git...
- nothrabannosir 8y ago
  You could literally link the two object directories? I just tried this and it seems to work: git clone git://github.com/git/git mkdir git2 cd git2 git init cd .git/ rm -rf objects ln -s ../../git/.git cd ../ git remote add origin git://github.com/git/git git fetch # returned without downloading anything git checkout master ls # etc. If you seriously want to use this, you'll probably want to hard link the contents, instead. But iirc git clone from local disk already does that, for you? In short: clone your local copy and taking it from there?
  - falsedan 8y ago
    You can also use alternates: echo ../../../git/.git/objects >> git2/.git/objects/info/alternates or use the original as a reference: git clone --reference git git://github.com/git/git git2 This sets up the alternates for you.
Boulth 8y ago
> Server-side filtering of references I wonder if this will be somehow exposed by git daemon. It could be used for easy per ref access controls. For example Git Switch [0] that uses Macaroons had to clone the repository to implement per ref ACL. [0]: https://github.com/rescrv/gitswitch https://github.com/rescrv/gitswitch
hartator 8y ago
Is Git a Google project now?
- jkaplowitz 8y ago
  No, but many of the core contributors are employed by Google and spend time on it as part of their day job (with Google's knowledge and permission). This post straddles both the open source part of their jobs and the "Git deployment at Google" part.
- s2g 8y ago
  BRB switching to Mercurial
s2g 8y ago
oh neat, and it's on a google blog. That's great. Another subtle reminder that this ad company has way too much control.
lsiebert 8y ago
I'm a very (very) minor contributor to git. If you are at all interested in hacking on Git, it's not that difficult. Knowing C and portable shell scripting for writing tests are the big things. One sticking point, you need to submit patches to the mailing list, you can't just do a github pull request. See https://github.com/git/git/blob/master/Documentation/SubmittingPatches https://github.com/git/git/blob/master/Documentation/Submitt... I still see github pull requests rather frequently, even though they have never been allowed. All discussion AND patches go through the mailing list, much like the linux kernel.
- pm215 8y ago
  It's unfortunate that github doesn't let a project disable the on-website UI for pull request submission; as it is it's easy for somebody to end up wasting their time trying to submit a change that way. (QEMU has that issue too.)
  - ImJasonH 8y ago
    Totally agree! I made nopullrequests.com to help solve this.
    - lsiebert 8y ago
      And that's nice, but I'd also love to see a bot that formats the pull request into a patch email for you.
newscracker 8y ago
Sometime in the far future, someone will write an interesting story about how a double null byte came into existence in the git request protocol, and it will be amusing and interesting to look back. As the saying goes, hindsight is always 20/20. I'm glad that they found ways to maintain backward compatibility, at only a minor cost to understanding things.

547 points by robmaceachern 8y ago

GauntletWizard 8y ago

Interesting that they took to the Google blog to announce this; is there a corresponding LKML post?

jkaplowitz 8y ago
Why LKML? Despite Git's origins from and use by the Linux project, it isn't especially tied to it now. LKML would presumably be the place for Linux to announce when they adopt this. The Google open source blog is among the several credible options for this post, since Google employs much of the core Git team, and this post discusses their experience deploying Git protocol v2 at Google. As noted in the blog text, it's not in a released version of Git yet, just Git master branch. So maybe it'll appear on a dedicated Git announcement list, if any, once that happens.
- joatmon-snoo 8y ago
  Junio posts on the list when there's a new release, e.g. https://public-inbox.org/git/xmqqwoxw6kkk.fsf@gitster-ct.c.googlers.com/ https://public-inbox.org/git/xmqqwoxw6kkk.fsf@gitster-ct.c.g... It seems that https://groups.google.com/forum/#!forum/git-packagers https://groups.google.com/forum/#!forum/git-packagers is the closest thing to a formal announcement list that there is.
  - jkaplowitz 8y ago
    Okay, I guess that tie continues for historical reasons. At least people who don't otherwise subscribe to LKML can still receive Git release announcements via the second link. I presume Git 2.18 (the first release supporting protocol v2) will be announced via both channels once it's out.
Analemma_ 8y ago
> support for v2 was recently merged to Git's master branch and is expected to be part of Git 2.18 Not yet, but presumably there will be a post like this: https://lkml.org/lkml/2018/4/2/425 https://lkml.org/lkml/2018/4/2/425 when it is released. It is strange that the Google Blog is the first place to announce it through.
tgummerer 8y ago
As mentioned in another comment, protocol v2 was implemented by a Google employee, and they decided to write a blog post about it. This is not an official git announcement.
u801e 8y ago
I found a mention of it in a "what's cooking" post on the git mailing list (Message-ID <xmqqvabm6csb.fsf@gitster-ct.c.googlers.com>). But I can't find a direct link on gmane.com right now.

ksec 8y ago

I thought google uses hg, have they switched over to git as well?

seabrookmx 8y ago
For all the "big" Google projects they use a proprietary system called piper. I think all their open-source stuff (Angular, GoLang, Android) uses git (and sometimes Gerrit). Although given Google's scale, I'm sure there's some teams/projects that use Mercurial.
- pjmlp 8y ago
  Go started on Mercurial and then eventually moved into Git.
  - jordigh 8y ago
    And they like neither. They really want a versioned filesystem.
    - dilap 8y ago
      That's interesting. Do you have more details / refs?
      - jordigh 8y ago
        I understand this was private communication from Rob and Russ.
        dilap 8y ago
        well dang! :-)
- ngoldbaum 8y ago
  In fact, developers are allowed to use whichever VCS tool they want on their local machine (or on the online coding in the cloud CitC environment). Some opt to use hg. The canonical repo is in piper though, so the hg commits or git commits get converted before they land.

Ericson2314 8y ago

This is disgusting. So little forsight in the past... At least the outcome of quite useful.

needz 8y ago
IIRC, they were in a serious time crunch when they drafted/made git. I can't remember the whole story...
- mc42 8y ago
  I thought Linus Torvalds was almost wholly responsible for the initial development of git? Even so, everything, especially software, is easier in hindsight....
  - needz 8y ago
    Sorry, I was using the 'singular' they.
- tytso 8y ago
  At the time Linus was the sole author/contributor of git, and he needed a replacement for BitKeeper in a hurry. BitKeeper had been made unavailable for Linux kernel development because the proprietor of BitKeeper was really unhappy that Tridge had reverse engineered the protocol and created an open-source client[1] which could talk to the Bitkeeper server. Linus created the first version of git sufficient to do a kernel commit in ten days[2]. [1] https://sourceforge.net/projects/sourcepuller/ https://sourceforge.net/projects/sourcepuller/ [2] https://www.linuxfoundation.org/blog/10-years-of-git-an-interview-with-git-creator-linus-torvalds/ https://www.linuxfoundation.org/blog/10-years-of-git-an-inte... (As Tridge tells the story[3], he telnet'ed to the bk port and typed "help" so it wasn't that much of a reverse engineering effort. :-) [3] https://lwn.net/Articles/132938/ https://lwn.net/Articles/132938/
- wereHamster 8y ago
  s/they/he/. It was Linus himself who alone created git, within a few weeks (two or three). At first it was just a handful of shell scripts, but it was self-hosting pretty early on.

gpvos 8y ago

Git didn't have a proper version number or extensibility field in its protocol? That's quite a bit of hubris.

zeroxfe 8y ago
Or, more likely, an oversight.
- gpvos 8y ago
  Hmm, I haven't designed very many data formats or wire protocols, and I won't claim I got it right any of those times, but I included some kind of extension possibility every time.
  - gchpaco 8y ago
    I damn near released a (private) message protocol without a version field a couple months ago, and I know better. Fortunately stopped myself and added it before any actual data got released.
  - shakna 8y ago
    Git was a 10-day urgent project. Given the timeframe, it's done remarkably well.

buckminster 8y ago

AIUI, the git ssh protocol is just the git protocol tunnelled through ssh. So why do they need different mechanisms for signalling V2?

_wmd 8y ago
Deploying Git over SSH entails locking the precise command line executable by the public key you use to authenticate. Locking SSH SendEnv down is mandatory too, otherwise thousands of people would have shell access to GitHub.com! This isn't even theoretical, there was an environment-related bug not 5 years ago involving Git. At least BitBucket was impacted, I think GitHub were patched before it was announced
- simias 8y ago
  I don't think that answers the parent's question, if the update was in the git protocol itself (encapsulated in the SSH session) then you wouldn't have to change anything at the SSH level. As you point out selectively allowing a new environment variable could open a can of worms for shared hosts like github if they mess up their implementation.
xyzzyz 8y ago
Because if you tunnel through ssh, you can signal v2 using ssh mechanism of setting environment variables. If you don't tunnel, you don't have this option. This is clearly described in the article.
- buckminster 8y ago
  Yes, but once you've updated the git protocol, ssh support comes for free. Having one mechanism is simpler than having two. And as your sibling notes, setting env vars from ssh has disadvantages. So why bother?
- deathanatos 8y ago
  I think what the person you're replying to is asking is why not, in the case of ssh, use the signaling in the git protocol, since it will be there anyways. That is, if you don't tunnel, you must signal w/ the git protocol. If you do tunnel, why use a different mechanism, since the signal in the git protocol must be there? I think that this is because the SSH protocol isn't just encapsulating the Git protocol directly (the initial assumption of ssh "just" encapsulating the git protocol is not fully correct), and one of the parts that differs is this particular part. (Since on the git protocol side, we need to select a "service": > a single packet-line which includes the requested service (git-upload-pack for fetches and git-receive-pack for pushes) which in SSH would be done not by transmitting that packet-line but by instructing SSH to run that particular executable. > This is clearly described in the article. It really isn't, IMO; if you don't have precise knowledge of the protocols involved, I don't think anything in the article particularly spells this out.

cpburns2009 8y ago

Wait, why was this posted by Google? I thought Git was made by Linus Torvalds.

Semaphor 8y ago
> Linus Torvalds said in 2012 that one of his own biggest successes was recognizing how good a developer Hamano was on Git, and trusting him to maintain it.
hk__2 8y ago
Git was created by Linus Torvalds, but out of the 50k+ commits on the repo, only 250 or so are from him, with only 14 in the past 6 years. [1] [1] https://github.com/git/git/graphs/contributors?from=2012-03-11&to=2018-05-10&type=c https://github.com/git/git/graphs/contributors?from=2012-03-...
- sdesol 8y ago
  Here is a more detailed analysis, which shows all contributors: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26t%3Dcontributors%26rg%3Dhistory https://public.gitsense.com/insight/github?r=git/git#b%3Dgit... These are contributions by Linus: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26q%3Dauthor%3Atorvalds%40g5.osdl.org%5E_%5Eauthor%3Atorvalds%40linux-foundation.org%5E_%5Eauthor%3Atorvalds%40osdl.org%5E_%5Eauthor%3Atorvalds%40ppc970.osdl.org%5E_%5Ewindow%3A10000%26t%3Dcommits https://public.gitsense.com/insight/github?r=git/git#b%3Dgit... and as you can see, his contributions, really tapered off after 2010, while contributions from Hamano remained steady from 2008 to present date, as shown below: https://public.gitsense.com/insight/github?r=git/git#b%3Dgithub%3Agit%2Fgit%3Amaster%26q%3Dauthor%3Agitster%40pobox.com%5E_%5Ewindow%3A10000%26t%3Dcommits0 https://public.gitsense.com/insight/github?r=git/git#b%3Dgit....
euyyn 8y ago
Opensource though.
tgummerer 8y ago
It's because a Google employee implemented protocol v2, and wrote a post about it.

oneeyedpigeon 8y ago

The current (and pretty much only, ever, despite Linus having been the creator) maintainer of git is a google employee [1], in case anyone else was wondering. [1] https://en.m.wikipedia.org/wiki/Junio_Hamano https://en.m.wikipedia.org/wiki/Junio_Hamano

Retroity 8y ago
Non-Mobile link for those on desktop: https://en.wikipedia.org/wiki/Junio_Hamano https://en.wikipedia.org/wiki/Junio_Hamano
- Cynddl 8y ago
  The mobile version of Wikipedia works perfectly fine in a browser. I personally prefer it for readability.
  - esaym 8y ago
    I don't...
  - tinus_hn 8y ago
    If you link the normal desktop version it the reader will automatically be redirected to the place they prefer.
chrisseldo 8y ago
>"Linus Torvalds said in 2012 that one of his own biggest successes was recognizing how good a developer Hamano was on Git, and trusting him to maintain it."
- vga805 8y ago
  wow! what an accolade
  - Dragonai 8y ago

Someone1234 8y ago

Too bad they didn't make Git LFS part of Version 2[0]. Most vendors[2] support LFS already but because it isn't required, some still lack it and its support cannot be assumed. [0] https://git-lfs.github.com/ https://git-lfs.github.com/ [1] https://github.com/git-lfs/git-lfs/wiki/Implementations https://github.com/git-lfs/git-lfs/wiki/Implementations

xenomachina 8y ago
> but because it is required Just to confirm, but you meant "because it is not required", right?
- Someone1234 8y ago
  Right. Edited.
49bc 8y ago
I wouldn't be so sure. They said one of the motivations was to "unblocking the path to more wire protocol improvements in the future".
icebraining 8y ago
I say that's LFS' fault. Why do you even need a custom server? It should just be able to use any ol' file server or S3-API compatible service, and do everything on the client side. I find git-annex a much better solution, it's a shame everyone went with LFS.
- rspeer 8y ago
  My experience with git-annex is that it seems heavily designed for individuals and not for projects. The places it looks for files are often just a computer you were once developing on, and it sometimes expects you to go find that computer. It never forgets about any crazy place your files have been. It was very hard to use in asymmetric cases where different people have different credentials, such as where one person has access to a computer and others don't, or where a couple of core developers have authenticated R/W access to a file server or an S3 bucket and everyone else just has HTTP.

simias 8y ago

Let that be a reminder to all the coders out there: if you ever design a protocol or file format to communicate between machines always remember to add a version field or some other way to allow for updates and revisions later without breaking everything. Having a way to specify extensions in a backward-compatible way is nice too.

cesarb 8y ago
> if you ever design a protocol or file format to communicate between machines always remember to add a version field or some other way to allow for updates and revisions later without breaking everything Also, somehow make sure no servers, clients, or third-party middleboxes break when the version field is incremented. The TLS protocol designers had to give up on the version field; it's now going to forever be stuck at "TLS 1.2", since too much would break otherwise.
- icebraining 8y ago
  It's an universal truth: if you want to keep something from jamming up, you need to exercise it. It's true for the human body, for machine parts and for protocol features.
  - stingraycharles 8y ago
    HTTP was at 1.1 for a very long time but it appears the upgrade to version 2 is going fine. What is the difference here? The protocol version exchange mechanism?
    - sp332 8y ago
      Because it's up to the client to request HTTP 2 if they support it. https://http2.github.io/http2-spec/#discover-http https://http2.github.io/http2-spec/#discover-http
      - stingraycharles 8y ago

Confiks 8y ago

It's quite a comedy that this feature has not been implemented for at least 6 years, solely because the raw git:// protocol's parameter handling was severely broken, and feature detection by disconnecting and retrying [1] was ultimately deemed far too dirty. [1] https://public-inbox.org/git/CAJo=hJtZ_8H6+kXPpZcRCbJi3LPuuF7M1U8YsjAp-iWvut9oMw@mail.gmail.com/ https://public-inbox.org/git/CAJo=hJtZ_8H6+kXPpZcRCbJi3LPuuF...

avar 8y ago

The specification of the v2 protocol is here: https://github.com/git/git/blob/master/Documentation/technical/protocol-v2.txt https://github.com/git/git/blob/master/Documentation/technic... One of the more exciting things is that it can now be extended to arbitrary new over-the-wire commands. So e.g. "git grep" could be made to execute over the network if that's more efficient in some cases. This will also allow for making things that now use side-transports part of the protocol itself if it made sense. E.g. the custom commands LFS and git-annex implement, and even more advanced things like shipping things like the new commit graph already generated from the server to the client.

xorcist 8y ago
If you are to link to a git repo, don't link to some unoffical mirror. That would just confuse search engines. The specificiation of the v2 protocol is here: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/technical/protocol-v2.txt https://git.kernel.org/pub/scm/git/git.git/tree/Documentatio... (There are a couple of repos listed as official mirrors, such as the googlesource.com one, but the one you linked to isn't one of them.)
- avar 8y ago
  The repository I linked to is official. See https://public-inbox.org/git/xmqqindt6g1r.fsf@gitster.mtv.corp.google.com/ https://public-inbox.org/git/xmqqindt6g1r.fsf@gitster.mtv.co... What list are you referring to? If it doesn't list the one on GitHub it needs to be fixed.
  - xorcist 8y ago
    I had a release announcement open for other reasons and noted the different URLs, but I was wrong and the link is absolutely fine. Thanks. If I could edit the post I would, so I'll just let this sit here in case anyone is confused (as I was).

rwmj 8y ago

Is there a git protocol variant that allows the client to avoid downloading objects that it already has stored locally in another repository or cache? For example: I have the Linux kernel already cloned in some directory. I clone a second repo which has the Linux kernel as a submodule. Can I clone the second repo straightforwardly without having to download Linux a second time? (Well yes, but only by manual intervention before doing the git submodule update - it'd be nice if objects could be shared in a cache across also repos somehow).

andrewaylett 8y ago
I'm assuming from your comment that you're already aware of --reference but it doesn't completely meet your needs? The only other thing I can think of would be to use the 'insteadOf' configuration to tell Git to use your local clone instead of the remote one. Search 'git help config' for 'url.<base>.insteadOf'.
Boulth 8y ago
Have you seen alternates? https://stackoverflow.com/questions/36123655/what-is-the-git-alternates-mechanism#36123844 https://stackoverflow.com/questions/36123655/what-is-the-git...
nothrabannosir 8y ago
You could literally link the two object directories? I just tried this and it seems to work: git clone git://github.com/git/git mkdir git2 cd git2 git init cd .git/ rm -rf objects ln -s ../../git/.git cd ../ git remote add origin git://github.com/git/git git fetch # returned without downloading anything git checkout master ls # etc. If you seriously want to use this, you'll probably want to hard link the contents, instead. But iirc git clone from local disk already does that, for you? In short: clone your local copy and taking it from there?
- falsedan 8y ago
  You can also use alternates: echo ../../../git/.git/objects >> git2/.git/objects/info/alternates or use the original as a reference: git clone --reference git git://github.com/git/git git2 This sets up the alternates for you.

Boulth 8y ago

> Server-side filtering of references I wonder if this will be somehow exposed by git daemon. It could be used for easy per ref access controls. For example Git Switch [0] that uses Macaroons had to clone the repository to implement per ref ACL. [0]: https://github.com/rescrv/gitswitch https://github.com/rescrv/gitswitch

hartator 8y ago

Is Git a Google project now?

jkaplowitz 8y ago
No, but many of the core contributors are employed by Google and spend time on it as part of their day job (with Google's knowledge and permission). This post straddles both the open source part of their jobs and the "Git deployment at Google" part.
s2g 8y ago
BRB switching to Mercurial

s2g 8y ago

oh neat, and it's on a google blog. That's great. Another subtle reminder that this ad company has way too much control.

lsiebert 8y ago

I'm a very (very) minor contributor to git. If you are at all interested in hacking on Git, it's not that difficult. Knowing C and portable shell scripting for writing tests are the big things. One sticking point, you need to submit patches to the mailing list, you can't just do a github pull request. See https://github.com/git/git/blob/master/Documentation/SubmittingPatches https://github.com/git/git/blob/master/Documentation/Submitt... I still see github pull requests rather frequently, even though they have never been allowed. All discussion AND patches go through the mailing list, much like the linux kernel.

pm215 8y ago
It's unfortunate that github doesn't let a project disable the on-website UI for pull request submission; as it is it's easy for somebody to end up wasting their time trying to submit a change that way. (QEMU has that issue too.)
- ImJasonH 8y ago
  Totally agree! I made nopullrequests.com to help solve this.
  - lsiebert 8y ago
    And that's nice, but I'd also love to see a bot that formats the pull request into a patch email for you.

newscracker 8y ago

Sometime in the far future, someone will write an interesting story about how a double null byte came into existence in the git request protocol, and it will be amusing and interesting to look back. As the saying goes, hindsight is always 20/20. I'm glad that they found ways to maintain backward compatibility, at only a minor cost to understanding things.