4 ms·

The forges are keenly aware of this effort, e.g. the person who's by far done most of the work on the SHA-256 transition (brian m. carlson) has I believe mostly

by avar 4y ago

The forges are keenly aware of this effort, e.g. the person who's by far done most of the work on the SHA-256 transition (brian m. carlson) has I believe mostly or entirely been done so on behalf of GitHub. I myself do some work on upstream Git on behalf of GitLab, although none of it's been on anything related to the SHA-256 transition. As to why no big forge has SHA-256 support, I think it's a bit of a chicken & egg problem (and these comments are entirely my own, and not on behalf of anyone). I think it's safe to say that all of the forges are expecting the transition, e.g. I don't think there's anyone creating CHAR(40) database tables for Git hashes anymore (or if they are, someone is planning to deal with it). Another is that for a successful transition for anything except entirely new repository networks (which already use SHA-1) you really need the "git" client to play along, see my other comment discussing hash interop plans. Some of that same code then needs to run on the server-side. That code isn't part of git yet, and it's really needed for any sort of viable migration plan. I mean, it's not really needed, at some point a lot of people reading this migrated from CVS and/or SVN to Git. But a full export/import with a lot of users is painful. We really want it to suck less for Git, to the point that it should Just Work for most or all users. And a major one is the human factor. For things to happen in free software development someone needs to submit patches, brian m. carlson has been performing a heroic amount of effort over the year on the transition over the years. As he notes in the linked ML thread he's had life reasons for why he hasn't been able to work on it as actively recently as he did in the past.

rurban 4y ago
brian moved from Texas to Canada, but mostly his employee, GitHub, is not priotizing the remaining sha256 patches. someone needs to finish up the transition patches, and forges need to double their disc space.
- cerved 4y ago
  I've seen this before but why do forges need to double space?
  - rurban 4y ago
    for the old hash, plus the new hash, plus the transition tables. (new <-> old)
    - cerved 4y ago
      but I would expect the vast majority of storage to be metadata and blobs, not hashes
  - avar 4y ago
    They may or may not, it's a time-space trade-off, and I'm just guessing that they'll be more likely to go for eating disk over eating CPU. There's some details on this in https://git-scm.com/docs/hash-function-transition/#_fetch https://git-scm.com/docs/hash-function-transition/#_fetch; part of the expected hash transition document assumes that you could have say a SHA-1 *.pack, but maintain both a SHA-1 and SHA-256 index into its contents. For blob objects you could serve them up as-is, but for the other types which refer to other objects (commits, trees and tags) you'd either need to store two copies and do something close to a sendfile(), or rewrite them on-the-fly as you stream them, using your SHA-1<->SHA-256 translation table. So I got a bit ahead of myself there, but none of this interop code exists yet, so the trade-offs the forges will have to make are unknown at this point.
    - vlovich123 4y ago
      Don’t blobs dominate the total disk space? It sounds like some data would double but the overall disk space usage shouldn’t be quite so dramatic (except for really small repos where it probably doesn’t matter?)
      - cerved 4y ago
        that was my intuition which is why the 2x space seemed odd

by avar 4y ago

rurban 4y ago

brian moved from Texas to Canada, but mostly his employee, GitHub, is not priotizing the remaining sha256 patches. someone needs to finish up the transition patches, and forges need to double their disc space.

cerved 4y ago
I've seen this before but why do forges need to double space?
- rurban 4y ago
  for the old hash, plus the new hash, plus the transition tables. (new <-> old)
  - cerved 4y ago
    but I would expect the vast majority of storage to be metadata and blobs, not hashes
- avar 4y ago
  They may or may not, it's a time-space trade-off, and I'm just guessing that they'll be more likely to go for eating disk over eating CPU. There's some details on this in https://git-scm.com/docs/hash-function-transition/#_fetch https://git-scm.com/docs/hash-function-transition/#_fetch; part of the expected hash transition document assumes that you could have say a SHA-1 *.pack, but maintain both a SHA-1 and SHA-256 index into its contents. For blob objects you could serve them up as-is, but for the other types which refer to other objects (commits, trees and tags) you'd either need to store two copies and do something close to a sendfile(), or rewrite them on-the-fly as you stream them, using your SHA-1<->SHA-256 translation table. So I got a bit ahead of myself there, but none of this interop code exists yet, so the trade-offs the forges will have to make are unknown at this point.
  - vlovich123 4y ago
    Don’t blobs dominate the total disk space? It sounds like some data would double but the overall disk space usage shouldn’t be quite so dramatic (except for really small repos where it probably doesn’t matter?)
    - cerved 4y ago
      that was my intuition which is why the 2x space seemed odd