12 ms·

This is one of the reasons why Go has its own versioning system. From a project's `go.sum`: example.com/example v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdz

This is one of the reasons why Go has its own versioning system. From a project's `go.sum`: example.com/example v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo= Where "h1" is an upgradeable hash (h1 is SHA-256). If there's ever a problem with h1, the hash can be simply upgraded. Git's documentation describes how to sign a git commit: $ git commit -a -S -m 'signed commit' When signing a git commit using the built in gpg function the project is not rehashed with a secure hash function, like SHA-256 or SHA3-256. Instead gpg signs the SHA-1 commit digest directly. It's not signing the result of a secure hash algorithm. SHA-1 has been considered weak for a long time (about 17 years). Bruce Schneier warned in February 2005 that SHA-1 needed to be replaced. Git development didn't start until April 2005. Before git started development, SHA-1 was identified as needing deprecation.

er4hn 4y ago
Just to nit on your portion of signing: wouldn't you need to rehash all prior commits as well so that they used the better hash function? Otherwise someone could find a collision for a prior commit hashed with sha-1, slip that in, and the final commit being hashed with sha256 wouldn't matter. This then makes the signing code use its own form of hashing that is different from the rest of git's commmit hashing, and seems like a novel way to introduce tooling issues / bugs / etc.
- chimeracoder 4y ago
  > and the final commit being hashed with sha256 wouldn't matter. Git stores content, not diffs. So the signature verifies all content stored in that commit. It does t verify anything that came before it, unless those are specifically signed as well.
  - ElectricalUnion 4y ago
    > Git stores content, not diffs. But the "contents" is just pointers to tree roots with a trusted hash. If the hash is no longer secure, you can't garantee that any such trees are your content, or safe.
    - Arnavion 4y ago
      The assumption in this context is that all those have been rehashed to SHA-256 too. The point was about whether that rehashing needed to be extended to previous commits.
bawolff 4y ago
Versioning hashes is definitely not a new idea with go - just look at how unix stores password hashes.
- barsonme 4y ago
  The author of the comment did not imply this.
Groxx 4y ago
There's no need to explicitly version your first version of this though. Those first-version values are easy to identify: they don't contain versioning information :) E.g. say you have `5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8`. What version is that? Well. It's exactly as long as a SHA1 hash. It doesn't start with "sha256:" or "md5:" or "h1:" or "rot13:". So it's SHA1. Easy and totally unambiguous. Versioning can almost always begin with version 2.
- morelisp 4y ago
  Me, sowing: "Each record begins with a 4 octet BE value indicating the record length." Me, reaping: "Each record begins with a single byte indicating the record format version. In version 0, this is followed by a 3 octet BE value indicating the record length."
  - Groxx 4y ago
    if you're storing the raw binary rather than hex or base64: yeah. there are often no illegal values, so there's no way to safely extend it, unless you can differentiate on length. for those, you have to leave versioning room up-front. even 1 bit is enough, since a `1` can imply "following data describes the version", if a bit wastefully in the long run.
  - nh23423fefe 4y ago
    sow then reap
    - morelisp 4y ago
      Well this fucking sucks. What the fuck.
      - cdelsolar 4y ago
        What are you guys even talking about
        4y ago
kelnos 4y ago
I think the implications for Go are a bit different, though. It's a very simple matter to change the hash algorithm used for go.mod. Even if there was no hash version prefix, it's trivial to add one after the fact, though older tools would probably give a confusing error message without foreknowledge of the concept of an unrecognized hash algorithm. And adding a new hash algorithm is just a matter of writing a relatively small amount of code, and then probably waiting a few Go releases before making it the default and assuming most people will have it. Git's entire foundation relies on SHA1 hashes. Each commit is its own hash, and contains a list of the hashes of all files that are a part of it. Branches have hashes, tags have hashes. Everything has a hash. A repository that uses a different hash algorithm is a completely different repository, even if the contents and commits are otherwise identical. You can't even store your code on someone else's server (well, aside from manually copying the repository data over, though that won't be too useful) unless that server has upgraded their git version.
- samatman 4y ago
  The counterpoint: Fossil did it, it was easy, no big deal. Well, Fossil's database is much better designed, you reply. That it is!
  - jupp0r 4y ago
    I think the argument that gp is trying to make is that it's really hard for git to implement this in a backwards compatible way. You may be right (I don't know anything about Fossil, will take a look!) that Fossil allowed for this by making good design decisions in the past. This is not something that git maintainers can do right now without a time machine though. Old versions are in use out there and will need to keep working if the goal is to make the transition easier for users.
kazinator 4y ago
Whenever the word "upgrade" rears its head, beware. The intent behind it is obsolescence and phasing out, resulting in an endless make-work treadmill for the users. If there is ever a "problem with h1", and you neglect to upgrade your data right there and then, five to ten years, it will be unreadable.
- howinteresting 4y ago
  What in the world are you talking about? Generally, systems with upgradeable hashes will remain backwards-compatible with old ones forever.
- arccy 4y ago
  or you know, find the version that handles transition and run it to upgrade stepping through required versions a common operation
lewisl9029 4y ago
Also check out multihash from the IPFS folks: https://github.com/multiformats/multihash https://github.com/multiformats/multihash It's a more robust, well-specified, interoperable version of this concept. Though it's probably overkill if you control both the consumer and producer side (i.e. don't need the interoperability) and are just looking to make hash upgrades smoother, in that case a simple version prefix like Go's approach described above has lower overhead.
brasic 4y ago
> Instead gpg signs the SHA-1 commit digest directly A minor correction: when signing a commit, gpg does not sign the SHA-1 digest of that commit. This is impossible since the signature becomes part of the commit header which is one of the inputs to the hash function that produces the oid. Instead, GPG signs the serialized data (parents,headers,tree,message) which would otherwise be the input to SHA-1. Then the sig is inserted into the buffer at the end of the header and the string is digested to produce an oid. Source: https://github.com/git/git/blob/39c15e485575089eb77c769f6da02f98a55905e0/commit.c#L1587 https://github.com/git/git/blob/39c15e485575089eb77c769f6da0...
- Zamicol 4y ago
  Thank you for pointing this out, and thank you for the link the the relevant code section. Excellent comment.
  - omegalulw 4y ago
    Lmao how were you that confident in your original comment which basically claimed git signatures are only as secure as SHA-1.
    - brasic 4y ago
      Perhaps you misinterpreted my reply. I didn’t intend to dispute parent’s claim, merely to correct a mechanical detail. Git signatures are more or less only as secure as SHA-1, although the properties it relies on are not yet compromised and several factors mitigate the real-world risk. A practical preimage attack on SHA-1 would seriously undermine the security of git signatures since the string being signed includes two or more SHA-1 hashes representing a content snapshot and the commit ancestry. Arbitrary preimage attacks would make it possible to modify a repo’s contents or history without invalidating oids or signatures. In practice only collision attacks have been found, all of which have a detectable signature that git has been modified to detect. Disclosure: I work at GitHub, but am speaking for myself.

er4hn 4y ago

Just to nit on your portion of signing: wouldn't you need to rehash all prior commits as well so that they used the better hash function? Otherwise someone could find a collision for a prior commit hashed with sha-1, slip that in, and the final commit being hashed with sha256 wouldn't matter. This then makes the signing code use its own form of hashing that is different from the rest of git's commmit hashing, and seems like a novel way to introduce tooling issues / bugs / etc.

chimeracoder 4y ago
> and the final commit being hashed with sha256 wouldn't matter. Git stores content, not diffs. So the signature verifies all content stored in that commit. It does t verify anything that came before it, unless those are specifically signed as well.
- ElectricalUnion 4y ago
  > Git stores content, not diffs. But the "contents" is just pointers to tree roots with a trusted hash. If the hash is no longer secure, you can't garantee that any such trees are your content, or safe.
  - Arnavion 4y ago
    The assumption in this context is that all those have been rehashed to SHA-256 too. The point was about whether that rehashing needed to be extended to previous commits.

bawolff 4y ago

Versioning hashes is definitely not a new idea with go - just look at how unix stores password hashes.

barsonme 4y ago
The author of the comment did not imply this.

Groxx 4y ago

There's no need to explicitly version your first version of this though. Those first-version values are easy to identify: they don't contain versioning information :) E.g. say you have `5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8`. What version is that? Well. It's exactly as long as a SHA1 hash. It doesn't start with "sha256:" or "md5:" or "h1:" or "rot13:". So it's SHA1. Easy and totally unambiguous. Versioning can almost always begin with version 2.

morelisp 4y ago
Me, sowing: "Each record begins with a 4 octet BE value indicating the record length." Me, reaping: "Each record begins with a single byte indicating the record format version. In version 0, this is followed by a 3 octet BE value indicating the record length."
- Groxx 4y ago
  if you're storing the raw binary rather than hex or base64: yeah. there are often no illegal values, so there's no way to safely extend it, unless you can differentiate on length. for those, you have to leave versioning room up-front. even 1 bit is enough, since a `1` can imply "following data describes the version", if a bit wastefully in the long run.
- nh23423fefe 4y ago
  sow then reap
  - morelisp 4y ago
    Well this fucking sucks. What the fuck.
    - cdelsolar 4y ago
      What are you guys even talking about
      - 4y ago

kelnos 4y ago

I think the implications for Go are a bit different, though. It's a very simple matter to change the hash algorithm used for go.mod. Even if there was no hash version prefix, it's trivial to add one after the fact, though older tools would probably give a confusing error message without foreknowledge of the concept of an unrecognized hash algorithm. And adding a new hash algorithm is just a matter of writing a relatively small amount of code, and then probably waiting a few Go releases before making it the default and assuming most people will have it. Git's entire foundation relies on SHA1 hashes. Each commit is its own hash, and contains a list of the hashes of all files that are a part of it. Branches have hashes, tags have hashes. Everything has a hash. A repository that uses a different hash algorithm is a completely different repository, even if the contents and commits are otherwise identical. You can't even store your code on someone else's server (well, aside from manually copying the repository data over, though that won't be too useful) unless that server has upgraded their git version.

samatman 4y ago
The counterpoint: Fossil did it, it was easy, no big deal. Well, Fossil's database is much better designed, you reply. That it is!
- jupp0r 4y ago
  I think the argument that gp is trying to make is that it's really hard for git to implement this in a backwards compatible way. You may be right (I don't know anything about Fossil, will take a look!) that Fossil allowed for this by making good design decisions in the past. This is not something that git maintainers can do right now without a time machine though. Old versions are in use out there and will need to keep working if the goal is to make the transition easier for users.

kazinator 4y ago

Whenever the word "upgrade" rears its head, beware. The intent behind it is obsolescence and phasing out, resulting in an endless make-work treadmill for the users. If there is ever a "problem with h1", and you neglect to upgrade your data right there and then, five to ten years, it will be unreadable.

howinteresting 4y ago
What in the world are you talking about? Generally, systems with upgradeable hashes will remain backwards-compatible with old ones forever.
arccy 4y ago
or you know, find the version that handles transition and run it to upgrade stepping through required versions a common operation

lewisl9029 4y ago

Also check out multihash from the IPFS folks: https://github.com/multiformats/multihash https://github.com/multiformats/multihash It's a more robust, well-specified, interoperable version of this concept. Though it's probably overkill if you control both the consumer and producer side (i.e. don't need the interoperability) and are just looking to make hash upgrades smoother, in that case a simple version prefix like Go's approach described above has lower overhead.

brasic 4y ago

> Instead gpg signs the SHA-1 commit digest directly A minor correction: when signing a commit, gpg does not sign the SHA-1 digest of that commit. This is impossible since the signature becomes part of the commit header which is one of the inputs to the hash function that produces the oid. Instead, GPG signs the serialized data (parents,headers,tree,message) which would otherwise be the input to SHA-1. Then the sig is inserted into the buffer at the end of the header and the string is digested to produce an oid. Source: https://github.com/git/git/blob/39c15e485575089eb77c769f6da02f98a55905e0/commit.c#L1587 https://github.com/git/git/blob/39c15e485575089eb77c769f6da0...

Zamicol 4y ago
Thank you for pointing this out, and thank you for the link the the relevant code section. Excellent comment.
- omegalulw 4y ago
  Lmao how were you that confident in your original comment which basically claimed git signatures are only as secure as SHA-1.
  - brasic 4y ago
    Perhaps you misinterpreted my reply. I didn’t intend to dispute parent’s claim, merely to correct a mechanical detail. Git signatures are more or less only as secure as SHA-1, although the properties it relies on are not yet compromised and several factors mitigate the real-world risk. A practical preimage attack on SHA-1 would seriously undermine the security of git signatures since the string being signed includes two or more SHA-1 hashes representing a content snapshot and the commit ancestry. Arbitrary preimage attacks would make it possible to modify a repo’s contents or history without invalidating oids or signatures. In practice only collision attacks have been found, all of which have a detectable signature that git has been modified to detect. Disclosure: I work at GitHub, but am speaking for myself.