4 ms·

There were some plans to migrate to SHA256, but somehow it still hasn't happened. The practical upshot is a git commit hash is not enough l to know you are dis

by Terr_ 2y ago

There were some plans to migrate to SHA256, but somehow it still hasn't happened. The practical upshot is a git commit hash is not enough l to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger. This is particularly important for tools that rely on it for dependency management, local caches, etc.

shakna 2y ago
Compatibility between remotes using one or the other hasn't arrived yet, and git doesn't want to break compatibility. But you can create SHA256 one's today. [0] [0] https://lwn.net/Articles/898522/ https://lwn.net/Articles/898522/
onedognight 2y ago
The is not a full git commit hash collision. It has to do with a git note which only needs to matche a 12 character prefix of the git commit.
- usr1106 2y ago
  While you corrected one mistake you added a new one:) Those are git trailers, see git-interpret-trailers(1). git-notes(1) is something completely different and not used by the kernel.
theamk 2y ago
people don't really care, because current collision methods are mitigated. Git does not actually use "sha1", despite what all the docs say, it uses "sha1dc", which is just like sha1 except for inputs which can cause collisions, in which case it either fails with clear error message or returns completely different value. https://news.ycombinator.com/item?id=17825441 https://news.ycombinator.com/item?id=17825441 so don't worry, git hashes _are_ enough to know you are distributing/executing the legitimate code. (not to mention you need a preimage attack to replace known commit, and this is not yet possible with sha1)
- akoboldfrying 2y ago
  From your link: >In this case "hash" will be the same as SHA1(input) in all cases, except those where the input is detected to be malicious (as in the SHAttered attack) I don't see how this can be more than a fundamentally forward-incompatible sticking plaster over the problem. The problem isn't merely that "detecting maliciousness" seems fraught in itself (how does one infer intent reliably?) -- it's that today's SHA1DC() implementation can only detect and optionally correct today's known attacks, so each new attack necessitates a new, incompatible version of SHA1DC().
  - Dylan16807 2y ago
    Sure it's plaster, but plaster can last a good while. Sufficiently new attacks don't come around all that often, and every hash has a risk of new attacks showing up.
    - hcs 2y ago
      fwiw, I wasn't too familiar with this usage, looked it up and sticking plaster is in the sense of bandaid here https://en.wikipedia.org/wiki/Adhesive_bandage https://en.wikipedia.org/wiki/Adhesive_bandage But agreed that it's a rather tougher patch than I'd originally thought, reading elsewhere in this thread.
  - theamk 2y ago
    Each new _unrelated_ SHA1 attack will need an update in SHA1DC. But it has to be truly unrelated, as the collision detection method is fairly robust. I recommend reading the original "Counter-cryptanalysis" paper [0] for details on how attacks work and how they can be mitigated (there is certain internal state in SHA1 that is used in all known attacks). BTW, this paper has an interesting anecdote: apparently Flame malware had exploited MD5 collisions using novel unpublished attack method... and yet it was detected by collision detector (section 3.2). Another example is that SHA-mbles attack, published 3 years after "Counter-cryptanalysis", was detected as well, with no required code changes. No, there is nothing "fundamentally incompatible" in the new SHA1DC method. After all, git came out in 2005, 11 years before SHA1 attacks were known, so it used regular SHA1. The collision detector was added in 2017 and nothing broke, because false positive chance is 2^-90 [1]. I have not heard of any new SHA1 collision results, but if they are based on no-difference differential paths, git has nothing to worry about. And if they are not, it may be possible to extend DC detector to seamlessly detect and prevent those attacks, and then only upgrade git clients, keeping backward and forward compatibility for data. Of course there is always a chance that someone will come out with all-new SHA1 preimage attack that cannot be detected without high rate of false positive, so it's prudent to switch git to sha256. There is a lot of work being done: git's sha256 mode went out of beta in 2.42 (2023), but neither github nor gitlab support it. But since the current state of git's sha is that there is nothing broken, and git git commit hash _is_ "enough to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger", there is no real pressure. [0] https://marc-stevens.nl/research/papers/C13-S.pdf https://marc-stevens.nl/research/papers/C13-S.pdf [1] https://github.com/cr-marcstevens/sha1collisiondetection https://github.com/cr-marcstevens/sha1collisiondetection
bandrami 2y ago
The hash space is atoms-in-the-universe range; this is a collision in a much, much smaller subset of that space
TheDong 2y ago
> The practical upshot is a git commit hash is not enough l to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger Really now? Mind if I challenge you? I have on my machine a git repo with commit '75eb4e3b1369706a4dcd61cc80e49660ac341ea4'. If you can give me a second git repo with such a commit containing different contents, I'll happily send you $10k USD, or donate it to a charity of your choice.
- sgjohnson 2y ago
  > If you can give me a second git repo with such a commit containing different contents, I'll happily send you $10k USD, or donate it to a charity of your choice. Calculating that SHA1 collision is going to be a bit more expensive than $10k, by a couple of orders of magnitude. Finding it in the wild is improbable, but calculating it is definitely possible, and has been done before. http://shattered.io/ http://shattered.io/
  - chippiewill 2y ago
    Shattered didn't produce a collision for an arbitrary hash, it produced two documents with the same hash (which is a slightly easier problem, about 100,000x faster). SHA1 is certainly insecure at this point, but not even close to trivially so.
    - codeflo 2y ago
      That is enough to distribute malicious code though, at least in certain scenarios. Someone might create a setup where reviewers check/sign one version of the source code, and what gets distributed is another version with the same hash.
      - ExoticPearTree 2y ago
        Can you create a proof of concept and show it here?
        2y ago
oefrha 2y ago
Switching to SHA-256 and switching to longer substrings of hashes for identification are basically orthogonal problems. The former is hardly going to help with the latter, except in the we already broke everything so why not take the chance to break some more sense.
usr1106 2y ago
TFA has nothing to do with SHA-1 or SHA-1 collisions. It's about abbreviated hash values introduced for readability by humans. Now these values are used by auxiliary scripts. Which again has little to do with git proper. It's just what the kernel community writes into commit messages and what scripts they use to parse those messages.
dec0dedab0de 2y ago
I think multiple hashes is the way to go to avoid collisions. it can even be something simple like md5. the chances of finding a collision that matches two or more algorithms is near impossible. Obviously that doesn’t work for passwords, but for verifying that data hasn’t been tampered with, it works.

by Terr_ 2y ago

shakna 2y ago

Compatibility between remotes using one or the other hasn't arrived yet, and git doesn't want to break compatibility. But you can create SHA256 one's today. [0] [0] https://lwn.net/Articles/898522/ https://lwn.net/Articles/898522/

onedognight 2y ago

The is not a full git commit hash collision. It has to do with a git note which only needs to matche a 12 character prefix of the git commit.

usr1106 2y ago
While you corrected one mistake you added a new one:) Those are git trailers, see git-interpret-trailers(1). git-notes(1) is something completely different and not used by the kernel.

theamk 2y ago

people don't really care, because current collision methods are mitigated. Git does not actually use "sha1", despite what all the docs say, it uses "sha1dc", which is just like sha1 except for inputs which can cause collisions, in which case it either fails with clear error message or returns completely different value. https://news.ycombinator.com/item?id=17825441 https://news.ycombinator.com/item?id=17825441 so don't worry, git hashes _are_ enough to know you are distributing/executing the legitimate code. (not to mention you need a preimage attack to replace known commit, and this is not yet possible with sha1)

akoboldfrying 2y ago
From your link: >In this case "hash" will be the same as SHA1(input) in all cases, except those where the input is detected to be malicious (as in the SHAttered attack) I don't see how this can be more than a fundamentally forward-incompatible sticking plaster over the problem. The problem isn't merely that "detecting maliciousness" seems fraught in itself (how does one infer intent reliably?) -- it's that today's SHA1DC() implementation can only detect and optionally correct today's known attacks, so each new attack necessitates a new, incompatible version of SHA1DC().
- Dylan16807 2y ago
  Sure it's plaster, but plaster can last a good while. Sufficiently new attacks don't come around all that often, and every hash has a risk of new attacks showing up.
  - hcs 2y ago
    fwiw, I wasn't too familiar with this usage, looked it up and sticking plaster is in the sense of bandaid here https://en.wikipedia.org/wiki/Adhesive_bandage https://en.wikipedia.org/wiki/Adhesive_bandage But agreed that it's a rather tougher patch than I'd originally thought, reading elsewhere in this thread.
- theamk 2y ago
  Each new _unrelated_ SHA1 attack will need an update in SHA1DC. But it has to be truly unrelated, as the collision detection method is fairly robust. I recommend reading the original "Counter-cryptanalysis" paper [0] for details on how attacks work and how they can be mitigated (there is certain internal state in SHA1 that is used in all known attacks). BTW, this paper has an interesting anecdote: apparently Flame malware had exploited MD5 collisions using novel unpublished attack method... and yet it was detected by collision detector (section 3.2). Another example is that SHA-mbles attack, published 3 years after "Counter-cryptanalysis", was detected as well, with no required code changes. No, there is nothing "fundamentally incompatible" in the new SHA1DC method. After all, git came out in 2005, 11 years before SHA1 attacks were known, so it used regular SHA1. The collision detector was added in 2017 and nothing broke, because false positive chance is 2^-90 [1]. I have not heard of any new SHA1 collision results, but if they are based on no-difference differential paths, git has nothing to worry about. And if they are not, it may be possible to extend DC detector to seamlessly detect and prevent those attacks, and then only upgrade git clients, keeping backward and forward compatibility for data. Of course there is always a chance that someone will come out with all-new SHA1 preimage attack that cannot be detected without high rate of false positive, so it's prudent to switch git to sha256. There is a lot of work being done: git's sha256 mode went out of beta in 2.42 (2023), but neither github nor gitlab support it. But since the current state of git's sha is that there is nothing broken, and git git commit hash _is_ "enough to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger", there is no real pressure. [0] https://marc-stevens.nl/research/papers/C13-S.pdf https://marc-stevens.nl/research/papers/C13-S.pdf [1] https://github.com/cr-marcstevens/sha1collisiondetection https://github.com/cr-marcstevens/sha1collisiondetection

bandrami 2y ago

The hash space is atoms-in-the-universe range; this is a collision in a much, much smaller subset of that space

TheDong 2y ago

> The practical upshot is a git commit hash is not enough l to know you are distributing/executing the legitimate code, as opposed to a malicious doppelganger Really now? Mind if I challenge you? I have on my machine a git repo with commit '75eb4e3b1369706a4dcd61cc80e49660ac341ea4'. If you can give me a second git repo with such a commit containing different contents, I'll happily send you $10k USD, or donate it to a charity of your choice.

sgjohnson 2y ago
> If you can give me a second git repo with such a commit containing different contents, I'll happily send you $10k USD, or donate it to a charity of your choice. Calculating that SHA1 collision is going to be a bit more expensive than $10k, by a couple of orders of magnitude. Finding it in the wild is improbable, but calculating it is definitely possible, and has been done before. http://shattered.io/ http://shattered.io/
- chippiewill 2y ago
  Shattered didn't produce a collision for an arbitrary hash, it produced two documents with the same hash (which is a slightly easier problem, about 100,000x faster). SHA1 is certainly insecure at this point, but not even close to trivially so.
  - codeflo 2y ago
    That is enough to distribute malicious code though, at least in certain scenarios. Someone might create a setup where reviewers check/sign one version of the source code, and what gets distributed is another version with the same hash.
    - ExoticPearTree 2y ago
      Can you create a proof of concept and show it here?
      - 2y ago

oefrha 2y ago

Switching to SHA-256 and switching to longer substrings of hashes for identification are basically orthogonal problems. The former is hardly going to help with the latter, except in the we already broke everything so why not take the chance to break some more sense.

usr1106 2y ago

TFA has nothing to do with SHA-1 or SHA-1 collisions. It's about abbreviated hash values introduced for readability by humans. Now these values are used by auxiliary scripts. Which again has little to do with git proper. It's just what the kernel community writes into commit messages and what scripts they use to parse those messages.

dec0dedab0de 2y ago

I think multiple hashes is the way to go to avoid collisions. it can even be something simple like md5. the chances of finding a collision that matches two or more algorithms is near impossible. Obviously that doesn’t work for passwords, but for verifying that data hasn’t been tampered with, it works.