4 ms·

This release incorporates "delta islands" which is code GitHub has been using since 2015-ish (or earlier?) to store repositories and their forks in the same obj

by avar 8y ago

This release incorporates "delta islands" which is code GitHub has been using since 2015-ish (or earlier?) to store repositories and their forks in the same object storage, with "islands" of objects as they diverge. See https://githubengineering.com/counting-objects/ https://githubengineering.com/counting-objects/ for a related blogpost of theirs. This was work sponsored by GitLab, and I know they plan to use it in production sometime soon. So it's interesting for other similar hosting sites with a "fork" feature to look into it.

reificator 8y ago
Your post as of my writing this comment makes it unclear: Is the code in this specific release written by GitHub, GitLab, or both?
- Jarwain 8y ago
  If I had to guess: Github added the feature to their platform, but it didn't go upstream. Gitlab sponsored the dev-hours to rewrite the feature and actually push it upstream
- avar 8y ago
  It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space. The feature isn't interesting except for those that run a GitHub-like setup. I.e. repositories with a fork network where clones & fetches can be expected to be served from any repository in the network. In those setups it can make a lot of difference.
  - Already__Taken 8y ago
    With gitlab-ce being ~500mb this must be a gigantic improvement for them.
  - reificator 8y ago
    > It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space. That makes it clear, thank you. My assumption was that GitLab was copying work done by GitHub. > The feature isn't interesting except for those that run a GitHub-like setup. Sure, doesn't mean I'm not interested.
beefsack 8y ago
If the last chart in that post[1] is anything to go by, then we will hopefully see some significant performance improvements from this. [1]: https://githubengineering.com/counting-objects/#the-results https://githubengineering.com/counting-objects/#the-results

by avar 8y ago

reificator 8y ago

Your post as of my writing this comment makes it unclear: Is the code in this specific release written by GitHub, GitLab, or both?

Jarwain 8y ago
If I had to guess: Github added the feature to their platform, but it didn't go upstream. Gitlab sponsored the dev-hours to rewrite the feature and actually push it upstream
avar 8y ago
It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space. The feature isn't interesting except for those that run a GitHub-like setup. I.e. repositories with a fork network where clones & fetches can be expected to be served from any repository in the network. In those setups it can make a lot of difference.
- Already__Taken 8y ago
  With gitlab-ce being ~500mb this must be a gigantic improvement for them.
- reificator 8y ago
  > It was written by GitHub, and has been used in their internal fork of git.git for a long time. The work of integrating it with upstream was sponsored by GitLab, since it's something they were interested in to save space. That makes it clear, thank you. My assumption was that GitLab was copying work done by GitHub. > The feature isn't interesting except for those that run a GitHub-like setup. Sure, doesn't mean I'm not interested.

beefsack 8y ago

If the last chart in that post[1] is anything to go by, then we will hopefully see some significant performance improvements from this. [1]: https://githubengineering.com/counting-objects/#the-results https://githubengineering.com/counting-objects/#the-results