6 ms·

I've never seen an adequate answer to this but it boggles my mind that this problem wasn't foreseen. When Git came out we'd already seen a move from MD5 hashes

• masklinn 6y ago
  > So why wasn't the protocol and the repository designed to have multiple (hash algorithm, hash) pairs? yagni? The design without a real use case could well have been unsuitable, and git has managed to live for 15 years before this issue became a real concern. That’s a pretty good run.
• corty 6y ago
  It would have been possible to include the ability to change algorithms in a handshake (agility). But general current consensus in the cryptography and security community (which I don't agree with) is: "Agility is bad, hmkay". The reasoning is largely because there have been plenty of attacks based on downgrading protocols like TLS to broken crypto by an attacker. Also, the handshake mechanism necessarily enlarges the attack surface of your software, usually in a pre-auth part which is very problematic. The counter-argument is of course that a lack of agility leads to lots of pain later on, when you need agility because your one chosen algorithm is broken. There is also the added argument for not using SHA-1 in git: SHA-1 was already starting to "smell" when git was first implemented. Picking something else back then would at least have moved the current problems farther into the future, when SHA-2 might start to be problematic.
  • hannob 6y ago
    There are good reasons why agility is bad. It never works the way people imagine. The fantasy is people think "in case of a sudden algorithm break we can quickly switch to a better algorithm". This is wrong because: a) there are no sudden algorithm breaks. You always know way in advance before something is broken. (In the case of SHA-1 there was more than a decade between early warnings and the first attack.) In the case of Git the warnings about SHA-1 are older than Git itself. The problem wasn't lack of agility, the problem was choosing an algorithm that was already known to be bad. b) if you have agility it means people will continue supporting bad algorithms and use agility as an excuse. b) happened exactly in TLS. Padding oracles were known, but people chose to ignore it. In TLS 1.2 they included better algorithms (GCM), but chose to keep the bad ones (CBC/HMAC). Then more padding oracle attacks happened. Then people said "oh we have agility, we can disable CBC and ... well nobody supports GCM yet... let's use this other broken thing we have called RC4". Then attacks on RC4 came in. If by the time people designed TLS 1.2 they would've said "we know RC4 is weak, we know CBC/HMAC (as used in TLS) is weak, we will just support GCM" none of that mess would've happened.
    • upofadown 6y ago
      >The fantasy is people think "in case of a sudden algorithm break we can quickly switch to a better algorithm". That isn't really what people are afraid of. They are afraid of the time, money and wasted effort that comes from having to completely rewrite everything. These cryptographic hashes are sort of a worst case. They often end up being part of a defined interface of some sort like a cryptographic identity. So starting over from scratch can have a high social cost as well. I think that when considering the lessons from the TLS case that people are forgetting that many applications do not work at all like TLS. You have to have the wisdom to throw out wisdom that does not apply when designing these things. GIT is an excellent example of such an application.
      • cyphar 6y ago
        You don't have to rewrite everything though. The rest of Git doesn't need to be rewritten if you change hash functions, nor does the rest of WireGuard if you change encryption or key exchange constructions. Newer constructions of similar crypto primitives can be used in the same way to solve the same problem. The key question is should a single version be able to handle both the old (broken) construction and the new (not broken) construction? There is fairly good evidence that allowing that leads to lots of potential for security issues due to all sorts of downgrade tricks that force users to switch to the broken version (or even worse, use the fact there are two versions to break the crypto as happened with DROWN). However, not allowing it has obvious consequences for usability (Python 3 comes to mind as a non-crypro example where having a hard cutover really didn't work).
    • corty 6y ago
      The changeover to TLS1.2 only ever could have worked because of agility. Without at least protocol version agility (which implies cryptographic agility), one couldn't talk TLS and its successor on the same port, because there wouldn't be any other way to distinguish them. A new TCP port assignment would have been possible, but a total and utter disaster. Countless firewalls, appliances, routers, middleboxes, rulesets and similar stuff assume that HTTPS is on port 443, changing that would at least take a decade. That the allowed set of ciphers didn't change enough with TLS1.2 may be right, but doesn't matter for why we do need agility. Agility is necessary and TLS is the poster-child for why we need agility in such protocols. Whether the same applies for git, where the network protocol isn't that important and ingrained is of course debatable, I think you are right there. But what git has to do now is create agility by cramming it in somewhere, using cludges, hacks and whatever might be possible without breaking too much. It remains to be seen if the cludge will be free of security problems.
• hannob 6y ago
  This was pointed out to Linus right at the beginning of GIt development [1]. SHA-256 was available and SHA-1 weaknesses were known. Unfortunately he chose to ignore it. [1] https://www.metzdowd.com/pipermail/cryptography/2017-February/031623.html https://www.metzdowd.com/pipermail/cryptography/2017-Februar...
• zamalek 6y ago
  SHA-1 wasn't intended to be cryptographic in Git. In Git, SHA is used to facilitate "content addressable storage". It probably would have been better if something like Murmur were used, because it would have been very clear that the function was not being used in a cryptographic sense. There was, and is, no need for it to change. The correct solution is to sign commits with something designed for that (PGP). Switching to SHA-256 is a dangerous change, because people will believe that it is secure in ways that it is not. SHA-256 will reduce the likelihood of collisions. However, if SHA-1 is used for CAS, there is a higher chance of a meteorite falling through your roof than finding a collision through normal use.
  • cesarb 6y ago
    > SHA-1 wasn't intended to be cryptographic in Git. In Git, SHA is used to facilitate "content addressable storage". It probably would have been better if something like Murmur were used, because it would have been very clear that the function was not being used in a cryptographic sense. For "content addressable storage" using hashes to work, you cannot have different files with the same hash, otherwise getting that file (through its contents hash) could return a different file. The only way to make sure there are no different files with the same hash is to use a cryptographic hash, otherwise some joker will create two files with different contents but the same hash (as happened with md5, and later with sha1; luckily, these sha1 files did not have the internal blob header git uses, and git was changed to detect and reject files created through that technique). > The correct solution is to sign commits with something designed for that (PGP). Git commits contain only the hash of the tree, the hash of the parent(s), and some other information like dates, authors, and commit message (see it for yourself with "git cat-file commit HEAD"). That is, signing a commit gains nothing if the hash of the tree (and all hashes nested within it) is not a cryptographic hash. You might think the answer is "just sign all the tree contents", but that would be really slow (a checkout of the Linux kernel tree seems to be over one gigabyte nowadays; compare with a commit, which can be as small as 250 bytes).
• robto 6y ago
  I don't think you are, and the IPFS ecosystem has already run into this problem. In IPFS, content is addressed by its hash, but different protocols are better suited to different encodings - browsers speak http, which is case insensitive. And so they've worked out self-describing hashes! https://multiformats.io/multihash/ https://multiformats.io/multihash/ The basic idea is that you include the name of the algorithm as part of the hash. This allows changing the hash algorithm without breaking backwards compatibility. It's a cool standard with implementations[0] for many languages. I don't know if it was considered for git, but it does seem likely that this issue will come up again before the end of time :) [0]https://github.com/multiformats/multihash https://github.com/multiformats/multihash