5 ms·

There seems to be a fundamental misunderstanding with a lot of these writeups. Are they 100% sure history was not rewritten at any point? Going back in time on

There seems to be a fundamental misunderstanding with a lot of these writeups. Are they 100% sure history was not rewritten at any point? Going back in time on the repo prior to listed involvement doesn't do anything as the attacker had full control. Starting from the last signed release prior to their involvement is the only way to actually move this forward (history may be fully lost at this point), the rest is posturing.

AtNightWeCode 2y ago
People think git is immutable. It is not.
- fl7305 2y ago
  Can you elaborate? Are you thinking of intentional SHA-1 has collisions? Would that work in practice?
  - AtNightWeCode 2y ago
    The history. Every time something like this attack happens people think they can read the complete git history in the repo.
    - fl7305 2y ago
      If some commits are signed by people you trust, can the chain before that still be compromised?
- Lichtso 2y ago
  Yes and no. A local GIT repo can be changed (including its history) however you please. But once you have shared it with others you can't take that back. If you try to, then others will notice that the hashes mismatch and that their HEAD diffs uncleanly. I know the term is infamous here, but GIT is essentially a blockchain. Each commit has a hash, which is based on the hashes of previous commits, forming a linked list (+ some DAG branching).
  - craftkiller 2y ago
    Its a Merkle Tree. They were invented 3 years before blockchains: https://en.wikipedia.org/wiki/Merkle_tree https://en.wikipedia.org/wiki/Merkle_tree
    - dboreham 2y ago
      Blockchains were invented in 1982?
      - Lichtso 2y ago
        In short, yes: https://en.wikipedia.org/wiki/Blockchain#History https://en.wikipedia.org/wiki/Blockchain#History People conflate blockchains, distributed networks and cryptocurrencies.
    - Lichtso 2y ago
      It also uses a Merkle tree to compress the snapshot versions associated with commits. But the actual commit structure builds on top of that. A pure Merkle tree or forest would only give you a set of overlapping snapshots, without any directionality. So, I think it is fair to call it a blockchain as well.
  - The_Colonel 2y ago
    > If you try to, then others will notice that the hashes mismatch and that their HEAD diffs uncleanly. So it relies on a human noticing and acting upon it. People not noticing backdoors being merged into the project is kinda the source of this problem.
    - fl7305 2y ago
      You can automate checks for if a large part of the previous git history suddenly changed. You can't automate checks for malicious code.
      - The_Colonel 2y ago
- ptx 2y ago
  Well, it is and it isn't: It has mutable pointers (branches and tags) to immutable nodes in a graph (commits).
azornathogron 2y ago
While it's certainly possible to rewrite git history, it's tricky to do it without other maintainers or contributors noticing, since anyone trying to pull into an existing local repo (rather than cloning fresh) would be hit with an unexpected non-fast-forward merge. It seems likely to me that Lasse Collin would have one or more long-standing local working copies. So IMHO injecting malicious changes back in time in the git history seems unlikely to me. But not strictly impossible.
- KyleSanderson 2y ago
  Based on how this has gone (remember xz has effectively been orphaned for years, and the majority of long-standing setups were using the release archives), unless if Lasse has never run any code from Jia (unlikely) I'd consider the entire machine untrusted (keys, etc). Provided the tarballs are still signed from that date, from another immutable source, that's really the only starting point here to rebuilding.
fl7305 2y ago
> Are they 100% sure history was not rewritten at any point? With git, one way to check is if other people still have clones of the xz repository from a time when it was trusted. If you suspect the repo history has been tampered with, you can check against those copies. I believe it would be hard to introduce such a history rewrite, since people pulling from the xz repo would start getting git error messages when things don't match up? I don't know to what degree intentional SHA-1 hash collisions could be used to work around that?
- dist-epoch 2y ago
  You can create pairs of SHA-1 hash collission, but not for a particular existing SHA-1 hash (the git one)
mxmlnkn 2y ago
Even history rewrites would be visible with Github's new Activity tab, e.g., see the two force-pushes in llama.cpp https://github.com/ggerganov/llama.cpp/activity https://github.com/ggerganov/llama.cpp/activity So, while, yes, git history can be rewritten, commits pushed to Github can effectively never be deleted. Personally, I find this to be a downside. Think, personal information, etc. But, in this case, it is helpful. Of course, the repository is suspended right now, so the Activity cannot be checked.
pdw 2y ago
In any case Debian has its own archive of every xz-utils version they've used in the past.
rkta 2y ago
The attacker had access to the GH mirror of the repo. The original repo remained at https://git.tukaani.org/ https://git.tukaani.org/
smartmic 2y ago
Concerning history rewrite, it makes sense to point to Fossil and its major difference to Git: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki#history https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki#... There is also a link to "Is Fossil a Blockchain?", an interesting read because the term was mentioned elsewhere is this thread.

AtNightWeCode 2y ago

People think git is immutable. It is not.

fl7305 2y ago
Can you elaborate? Are you thinking of intentional SHA-1 has collisions? Would that work in practice?
- AtNightWeCode 2y ago
  The history. Every time something like this attack happens people think they can read the complete git history in the repo.
  - fl7305 2y ago
    If some commits are signed by people you trust, can the chain before that still be compromised?
Lichtso 2y ago
Yes and no. A local GIT repo can be changed (including its history) however you please. But once you have shared it with others you can't take that back. If you try to, then others will notice that the hashes mismatch and that their HEAD diffs uncleanly. I know the term is infamous here, but GIT is essentially a blockchain. Each commit has a hash, which is based on the hashes of previous commits, forming a linked list (+ some DAG branching).
- craftkiller 2y ago
  Its a Merkle Tree. They were invented 3 years before blockchains: https://en.wikipedia.org/wiki/Merkle_tree https://en.wikipedia.org/wiki/Merkle_tree
  - dboreham 2y ago
    Blockchains were invented in 1982?
    - Lichtso 2y ago
      In short, yes: https://en.wikipedia.org/wiki/Blockchain#History https://en.wikipedia.org/wiki/Blockchain#History People conflate blockchains, distributed networks and cryptocurrencies.
  - Lichtso 2y ago
    It also uses a Merkle tree to compress the snapshot versions associated with commits. But the actual commit structure builds on top of that. A pure Merkle tree or forest would only give you a set of overlapping snapshots, without any directionality. So, I think it is fair to call it a blockchain as well.
- The_Colonel 2y ago
  > If you try to, then others will notice that the hashes mismatch and that their HEAD diffs uncleanly. So it relies on a human noticing and acting upon it. People not noticing backdoors being merged into the project is kinda the source of this problem.
  - fl7305 2y ago
    You can automate checks for if a large part of the previous git history suddenly changed. You can't automate checks for malicious code.
    - The_Colonel 2y ago
ptx 2y ago
Well, it is and it isn't: It has mutable pointers (branches and tags) to immutable nodes in a graph (commits).

azornathogron 2y ago

While it's certainly possible to rewrite git history, it's tricky to do it without other maintainers or contributors noticing, since anyone trying to pull into an existing local repo (rather than cloning fresh) would be hit with an unexpected non-fast-forward merge. It seems likely to me that Lasse Collin would have one or more long-standing local working copies. So IMHO injecting malicious changes back in time in the git history seems unlikely to me. But not strictly impossible.

KyleSanderson 2y ago
Based on how this has gone (remember xz has effectively been orphaned for years, and the majority of long-standing setups were using the release archives), unless if Lasse has never run any code from Jia (unlikely) I'd consider the entire machine untrusted (keys, etc). Provided the tarballs are still signed from that date, from another immutable source, that's really the only starting point here to rebuilding.

fl7305 2y ago

> Are they 100% sure history was not rewritten at any point? With git, one way to check is if other people still have clones of the xz repository from a time when it was trusted. If you suspect the repo history has been tampered with, you can check against those copies. I believe it would be hard to introduce such a history rewrite, since people pulling from the xz repo would start getting git error messages when things don't match up? I don't know to what degree intentional SHA-1 hash collisions could be used to work around that?

dist-epoch 2y ago
You can create pairs of SHA-1 hash collission, but not for a particular existing SHA-1 hash (the git one)

mxmlnkn 2y ago

Even history rewrites would be visible with Github's new Activity tab, e.g., see the two force-pushes in llama.cpp https://github.com/ggerganov/llama.cpp/activity https://github.com/ggerganov/llama.cpp/activity So, while, yes, git history can be rewritten, commits pushed to Github can effectively never be deleted. Personally, I find this to be a downside. Think, personal information, etc. But, in this case, it is helpful. Of course, the repository is suspended right now, so the Activity cannot be checked.

pdw 2y ago

In any case Debian has its own archive of every xz-utils version they've used in the past.

rkta 2y ago

The attacker had access to the GH mirror of the repo. The original repo remained at https://git.tukaani.org/ https://git.tukaani.org/

smartmic 2y ago

Concerning history rewrite, it makes sense to point to Fossil and its major difference to Git: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki#history https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki#... There is also a link to "Is Fossil a Blockchain?", an interesting read because the term was mentioned elsewhere is this thread.