8 ms·

Git was created 16 years ago. The impending breakage of SHA-1 was known even at that time, just like how MD5 had been broken before it. I'm honestly still shoc

by cletus 5y ago

Git was created 16 years ago. The impending breakage of SHA-1 was known even at that time, just like how MD5 had been broken before it. I'm honestly still shocked that updating the hashing algorithm wasn't built into Git from day one. I really wonder why. Did people think this wouldn't happen? Were they so in love with the performance of C/C++ being able to pass around 20 byte hashes on the stack without worrying about a more complicated structure (eg a collection of variable length hashes)? It just seems like such a massive and foreseeable oversight that's going to cause pain for some time to come for really no reason at all.

tantalor 5y ago
> going to cause pain What kind of attack on a git repo are you worried about?
- snovv_crash 5y ago
  Subrepo hash substitution could enable a supply-chain attack
renewiltord 5y ago
It is a law of nature that all successful products find themselves not accounting for some predictable problems.
alerighi 5y ago
SHA-1 will still work fine for the purpose of git. It is just no longer considered secure for cryptographic operations, such as digital signature, that doesn't mean that you can't use it for other purposes, like git does. Using it is still fine and will ever be fine. Making the hashing algorithm exchangeable would have introduces complexity in a software that is already complex, and also less efficient (one of the reasons git was created was speed for large project like the Linux kernel) for no real purpose. If you want to change the algorithm, given that you will break compatibility with all existing repositories, tools, and clients, you make a fork of git because you are changing too much. I don't see why migrating to SHA-256. The collisions are still very unlikely to generate accidentally, and sure, if you want to do damage to a repository you can create one on purpose, as you can as well commit some hooks that contains malware, or alter the git history in any way you want, so what's the point?
- madars 5y ago
  Collisions definitely do matter for git security: many people pin explicit git hashes for their dependancies, and thus they can be tricked in running malicious forks. This requires placing a chosen commit in the git repo (so unlike second preimage break it does not mean that you could attack repos you have no control over) but that's not an unrealistic threat model overall.
  - y4mi 5y ago
    What is the thread model though? I don't think it's possible to create a collision that's also executeable code which adds a security hole or anything. So what exactly would they achieve with the collision? And how do they push these gigantic files that have the hash collisions to a server? The upload time would be significant.
    - madars 5y ago
      The files don't need to be gigantic. You could, for example, have a binary config file which in one colliding version encodes a potentially dangerous debugging setting, e.g., "allow_unauthenticated_rpc = false" but in other has it to "true".
    - noway421 5y ago
      The possible attack is to prepare 2 versions of a commit, both resulting in the same commit id. Then later on, after the project is successful/etc, swap out the commit with the second version, while keeping the other commits intact. Granted, the file that the commit touches would need to be not touched in other commits. That's not out of question in a typical software project - maybe a file in the utils folder which is only written once and never changed? > I don't think it's possible to create a collision that's also executeable code You can include an unreadable binary blob in the commit. Tweak the blob to find the collision while keeping the code the way attack requires.
      - deleted 5y ago
        [deleted]
elfchief 5y ago
Git was not intended (AFAIK) to be cryptographically secure. Being unsuitable for crypto != being unsuitable for other uses.
- wcoenen 5y ago
  Surely signed tags and signed commits in git are supposed to be cryptographically secure? [0] Doesn't the security of those signatures depend on the security of the SHA-1 hashes that are being signed? [0] https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work
  - danachow 5y ago
    No - after all, the most common use of digital signatures is to sign documents that can be easily tampered. All the security is in the signature, not the content being signed.
    - lvh 5y ago
      This makes no sense. Sure: you can generate an X.509 certificate that says whatever you want, but the point is that you can validate the signature and see that it's a forgery. In the case of a hash-addressed system like git, the problem is that the signature is over a collision, so it no longer certifies the thing its supposed to certify. Git uses the hash as a shorthand for a revision, including its entire history--so yes, it is using the hash that way. By that logic, would MD5 be fine? MD4? CRC32?
    - wongarsu 5y ago
      A digital signature usually signs (~encrypts) a hash of the content. So asking what exactly signing a commit or tag entails is a very valid question. I would expect that signing a tag is only as strong as SHA-1, since a tag is essentially a label for a commit hash. For signing commits I have no clue, but would be quite interested as well.
formerly_proven 5y ago
Git is designed around content-addressed storage and while you can cater for hash migrations, it gets pretty messy design-wise. Gits core data structures were also designed really quickly to patch a very urgent need of the kernel hackers. I doubt it has anything to do with being in love passing 20 byte strings on the stack. The fine git folks have produced a rather detailed document about the hash migration (https://git-scm.com/docs/hash-function-transition/ https://git-scm.com/docs/hash-function-transition/) and it is not particularly simple to do this.
umvi 5y ago
"We note that classical collisions and chosen-prefix collisions do not threaten all usages of SHA-1."
ori_b 5y ago
For someone to be able to break your repo using sha1 collisions, they need to be able to commit to it. If you don't trust someone, don't let them commit to your repo. > Were they so in love with the performance of C/C++ being able to pass around 20 byte hashes on the stack without worrying about a more complicated structure (eg a collection of variable length hashes)? The hashes show up everywhere. They're how every object in git is identified. They make it onto disk. They're not just commit ids. Changing hashes changes the disk format.
- lamontcg 5y ago
  Yeah I don't think any of these proposed attacks work without write access to the repo, at which point the game is already pretty much over already.
  - seoaeu 5y ago
    There's plenty of workflows where untrusted people have their proposed changes audited and then written to a git repo
    - lamontcg 5y ago
      How do they get through the auditing? If I'm looking at a PR which contains a hash collision with code that does a `sudo rm -rf --no-preserve-root /` and I don't accept the malicious code, the hash collision becomes irrelevant.
- kragen 5y ago
  They might not need to commit to your repo to break it using SHA-1 collisions. They might, for example, compute a collision, commit the malicious version to a private GitHub repository, and then send you a patch with the non-malicious version via email, or by sending you a merge request on GitLab. When you accept and merge in their changes and then push to GitHub, perhaps GitHub will recognize the hash of one of your blobs as a hash that it already knows the blob for (from the private commit), and so it will ignore whatever data you send it for that blob. Maybe that particular scenario won't work — I haven't tried it — but there are hundreds of possible angles of attack. Fundamentally the security of Git is based on the idea that hashes don't collide.
dec0dedab0de 5y ago
Linus posted about it on google plus in 2017. I haven't re-read it yet, but I remember one of the ideas floating around hn at the time was to just have two hashes per commit. That is, two insecure hashes may be secure together for git's purposes. Even though we can generate collisions for md5, and sha1, it would be much more difficult to have a file generate an arbitrary collision for both at the same time. Here is a link to Linus's post on the internet archive https://web.archive.org/web/20170717192607/https://plus.google.com/+LinusTorvalds/posts/7tp2gYWQugL https://web.archive.org/web/20170717192607/https://plus.goog... And here are some hn posts around the same time https://news.ycombinator.com/item?id=13733481 https://news.ycombinator.com/item?id=13733481 https://news.ycombinator.com/item?id=13719368 https://news.ycombinator.com/item?id=13719368
- bawolff 5y ago
  > That is, two insecure hashes may be secure together for git's purposes. Generally that's not true for Merkle–Damgård (e.g. sha1, sha2) hashes - afaik the difficulty of finding a collision in the more difficult hash is the same (up to big-oh) as finding it in both hashes.
- zzo38computer 5y ago
  In git I don't know, but in fossil, it already does that if the file that the collision is generated for is not the manifest file. The manifest file and all other files are identified by the SHA-1 hash (in newer versions SHA3-256 is also supported), although the manifest file also has a R card which is the MD5 hash of the all of the other files. This means that if there is a collision, it will be detected and the file would be rejected.
deleted 5y ago
[deleted]
ploxiln 5y ago
> I'm honestly still shocked that updating the hashing algorithm wasn't built into Git from day one. I really wonder why. The first basically functional version of Git was created in less than a month by Linus Torvalds, immediately after Bitkeeper revoked the license/permission for linux kernel developers to use that proprietary source control tool for free. Linus took a look at Monotone and Mercurial IIRC, but they were not nearly fast enough at that time, so he created Git to just do what the linux kernel development community needed. This isn't the first time Linus did something that academics said was a terrible mistake, and then his creation took over the freaking world, because it works so freaking well. (And it still does today, 16 years later. Maybe tomorrow it won't, I dunno. But it probably will.)
- ezst 5y ago
  > Linus took a look at Monotone and Mercurial IIRC, but they were not nearly fast enough at that time That's incorrect for Mercurial, it was started by another kernel developer (Matt Mackall) at the same time, for the same reasons (but they ended-up with different trade-offs, which makes mercurial still very interesting and relevant in a post-github world).
throwaway984393 5y ago
> I'm honestly still shocked that updating the hashing algorithm wasn't built into Git from day one. The name literally means stupid person.
oshiar53-0 5y ago
FYI signed git commits and tags weren't a thing back then.

by cletus 5y ago

tantalor 5y ago

> going to cause pain What kind of attack on a git repo are you worried about?

snovv_crash 5y ago
Subrepo hash substitution could enable a supply-chain attack

renewiltord 5y ago

It is a law of nature that all successful products find themselves not accounting for some predictable problems.

alerighi 5y ago

SHA-1 will still work fine for the purpose of git. It is just no longer considered secure for cryptographic operations, such as digital signature, that doesn't mean that you can't use it for other purposes, like git does. Using it is still fine and will ever be fine. Making the hashing algorithm exchangeable would have introduces complexity in a software that is already complex, and also less efficient (one of the reasons git was created was speed for large project like the Linux kernel) for no real purpose. If you want to change the algorithm, given that you will break compatibility with all existing repositories, tools, and clients, you make a fork of git because you are changing too much. I don't see why migrating to SHA-256. The collisions are still very unlikely to generate accidentally, and sure, if you want to do damage to a repository you can create one on purpose, as you can as well commit some hooks that contains malware, or alter the git history in any way you want, so what's the point?

madars 5y ago
Collisions definitely do matter for git security: many people pin explicit git hashes for their dependancies, and thus they can be tricked in running malicious forks. This requires placing a chosen commit in the git repo (so unlike second preimage break it does not mean that you could attack repos you have no control over) but that's not an unrealistic threat model overall.
- y4mi 5y ago
  What is the thread model though? I don't think it's possible to create a collision that's also executeable code which adds a security hole or anything. So what exactly would they achieve with the collision? And how do they push these gigantic files that have the hash collisions to a server? The upload time would be significant.
  - madars 5y ago
    The files don't need to be gigantic. You could, for example, have a binary config file which in one colliding version encodes a potentially dangerous debugging setting, e.g., "allow_unauthenticated_rpc = false" but in other has it to "true".
  - noway421 5y ago
    The possible attack is to prepare 2 versions of a commit, both resulting in the same commit id. Then later on, after the project is successful/etc, swap out the commit with the second version, while keeping the other commits intact. Granted, the file that the commit touches would need to be not touched in other commits. That's not out of question in a typical software project - maybe a file in the utils folder which is only written once and never changed? > I don't think it's possible to create a collision that's also executeable code You can include an unreadable binary blob in the commit. Tweak the blob to find the collision while keeping the code the way attack requires.
    - deleted 5y ago
      [deleted]

elfchief 5y ago

Git was not intended (AFAIK) to be cryptographically secure. Being unsuitable for crypto != being unsuitable for other uses.

wcoenen 5y ago
Surely signed tags and signed commits in git are supposed to be cryptographically secure? [0] Doesn't the security of those signatures depend on the security of the SHA-1 hashes that are being signed? [0] https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work
- danachow 5y ago
  No - after all, the most common use of digital signatures is to sign documents that can be easily tampered. All the security is in the signature, not the content being signed.
  - lvh 5y ago
    This makes no sense. Sure: you can generate an X.509 certificate that says whatever you want, but the point is that you can validate the signature and see that it's a forgery. In the case of a hash-addressed system like git, the problem is that the signature is over a collision, so it no longer certifies the thing its supposed to certify. Git uses the hash as a shorthand for a revision, including its entire history--so yes, it is using the hash that way. By that logic, would MD5 be fine? MD4? CRC32?
  - wongarsu 5y ago
    A digital signature usually signs (~encrypts) a hash of the content. So asking what exactly signing a commit or tag entails is a very valid question. I would expect that signing a tag is only as strong as SHA-1, since a tag is essentially a label for a commit hash. For signing commits I have no clue, but would be quite interested as well.

formerly_proven 5y ago

Git is designed around content-addressed storage and while you can cater for hash migrations, it gets pretty messy design-wise. Gits core data structures were also designed really quickly to patch a very urgent need of the kernel hackers. I doubt it has anything to do with being in love passing 20 byte strings on the stack. The fine git folks have produced a rather detailed document about the hash migration (https://git-scm.com/docs/hash-function-transition/ https://git-scm.com/docs/hash-function-transition/) and it is not particularly simple to do this.

umvi 5y ago

"We note that classical collisions and chosen-prefix collisions do not threaten all usages of SHA-1."

ori_b 5y ago

For someone to be able to break your repo using sha1 collisions, they need to be able to commit to it. If you don't trust someone, don't let them commit to your repo. > Were they so in love with the performance of C/C++ being able to pass around 20 byte hashes on the stack without worrying about a more complicated structure (eg a collection of variable length hashes)? The hashes show up everywhere. They're how every object in git is identified. They make it onto disk. They're not just commit ids. Changing hashes changes the disk format.

lamontcg 5y ago
Yeah I don't think any of these proposed attacks work without write access to the repo, at which point the game is already pretty much over already.
- seoaeu 5y ago
  There's plenty of workflows where untrusted people have their proposed changes audited and then written to a git repo
  - lamontcg 5y ago
    How do they get through the auditing? If I'm looking at a PR which contains a hash collision with code that does a `sudo rm -rf --no-preserve-root /` and I don't accept the malicious code, the hash collision becomes irrelevant.
kragen 5y ago
They might not need to commit to your repo to break it using SHA-1 collisions. They might, for example, compute a collision, commit the malicious version to a private GitHub repository, and then send you a patch with the non-malicious version via email, or by sending you a merge request on GitLab. When you accept and merge in their changes and then push to GitHub, perhaps GitHub will recognize the hash of one of your blobs as a hash that it already knows the blob for (from the private commit), and so it will ignore whatever data you send it for that blob. Maybe that particular scenario won't work — I haven't tried it — but there are hundreds of possible angles of attack. Fundamentally the security of Git is based on the idea that hashes don't collide.

dec0dedab0de 5y ago

Linus posted about it on google plus in 2017. I haven't re-read it yet, but I remember one of the ideas floating around hn at the time was to just have two hashes per commit. That is, two insecure hashes may be secure together for git's purposes. Even though we can generate collisions for md5, and sha1, it would be much more difficult to have a file generate an arbitrary collision for both at the same time. Here is a link to Linus's post on the internet archive https://web.archive.org/web/20170717192607/https://plus.google.com/+LinusTorvalds/posts/7tp2gYWQugL https://web.archive.org/web/20170717192607/https://plus.goog... And here are some hn posts around the same time https://news.ycombinator.com/item?id=13733481 https://news.ycombinator.com/item?id=13733481 https://news.ycombinator.com/item?id=13719368 https://news.ycombinator.com/item?id=13719368

bawolff 5y ago
> That is, two insecure hashes may be secure together for git's purposes. Generally that's not true for Merkle–Damgård (e.g. sha1, sha2) hashes - afaik the difficulty of finding a collision in the more difficult hash is the same (up to big-oh) as finding it in both hashes.
zzo38computer 5y ago
In git I don't know, but in fossil, it already does that if the file that the collision is generated for is not the manifest file. The manifest file and all other files are identified by the SHA-1 hash (in newer versions SHA3-256 is also supported), although the manifest file also has a R card which is the MD5 hash of the all of the other files. This means that if there is a collision, it will be detected and the file would be rejected.

deleted 5y ago

[deleted]

ploxiln 5y ago

> I'm honestly still shocked that updating the hashing algorithm wasn't built into Git from day one. I really wonder why. The first basically functional version of Git was created in less than a month by Linus Torvalds, immediately after Bitkeeper revoked the license/permission for linux kernel developers to use that proprietary source control tool for free. Linus took a look at Monotone and Mercurial IIRC, but they were not nearly fast enough at that time, so he created Git to just do what the linux kernel development community needed. This isn't the first time Linus did something that academics said was a terrible mistake, and then his creation took over the freaking world, because it works so freaking well. (And it still does today, 16 years later. Maybe tomorrow it won't, I dunno. But it probably will.)

ezst 5y ago
> Linus took a look at Monotone and Mercurial IIRC, but they were not nearly fast enough at that time That's incorrect for Mercurial, it was started by another kernel developer (Matt Mackall) at the same time, for the same reasons (but they ended-up with different trade-offs, which makes mercurial still very interesting and relevant in a post-github world).

throwaway984393 5y ago

> I'm honestly still shocked that updating the hashing algorithm wasn't built into Git from day one. The name literally means stupid person.

oshiar53-0 5y ago

FYI signed git commits and tags weren't a thing back then.