5 ms·

Fossil's main author is chiming in the discussion of this on Fossil's forums: (https://fossil-scm.org/forum/forumpost/50a5bea5fb https://fossil-scm.org/forum/f

Fossil's main author is chiming in the discussion of this on Fossil's forums: (https://fossil-scm.org/forum/forumpost/50a5bea5fb https://fossil-scm.org/forum/forumpost/50a5bea5fb) > That's appalling. Fossil's implementation doesn't require a conversion. “This is a key point, that I want to highlight. I'm sorry that it wasn't made more clear in the LWN posting nor in the HN discussion. “With Fossil, to begin using the new SHA3 hash algorithm, you just upgrade your fossil binary. No further actions, workflow changes, disruptions, or thought are required on the part of the user. * “Old check-ins with SHA1 hashes continue to use their SHA1 hash names.” * “New check-ins automatically get more secure SHA3 hash names.” * “No repository conversions need to occur” * “Given a hash prefix, Fossil automatically figures out whether it is dealing with a SHA1 or a SHA3 hash” * “No human brain-cycles are wasted trying to navigate through a hash-algorithm cut-over.” “Contrast this to Git, where a repository must be either all-SHA1 or all-SHA2. Hence, to cut-over a repository requires rebuilding the repository and in the process renaming all historical artifacts -- essentially rebasing the entire repository. The historical artifact renaming means that external links to historical check-ins (such as in tickets) are broken. And during the transition period, users have to be constantly aware of whether they are using SHA1 or SHA2 hash names. It is a big mess. It is no wonder, then, that few people have been eager to transition their repositories over to the newer SHA2 format.”

seniorsassycat 7y ago
The way I read the fossil's authors comments, old commits continue to use sha1 hashes. A repository will be vulnerable to sha1 collision attacks as long as there is an object in the repository that has not been hashed with the new algorithm. For example, floppy.c could be replaced in a repo with file with the same sha1 hash as long as the last commit that modifies floppy.c used a sha1 hash. Right?
- SQLite 7y ago
  Just to be clear: Every time you modify a file, the new changes get put in using SHA3. In an older repository, any given commit might have some files identified using SHA1 (assuming they have not changed in 3 years) and others identified using SHA3. For example, the manifest of the latest SQLite check-in is see at (https://www.sqlite.org/src/artifact/29a969d6b1709b80 https://www.sqlite.org/src/artifact/29a969d6b1709b80). You can see that most of the files have longer SHA3 hashes, but some of the files that have not been touched in three years still carry SHA1 hashes. An attack like what you describe is possible if you could generate an evil.c file that has the exact same SHA1 hash as the older floppy.c file. Then you could substitute the evil.c artifact in place of the floppy.c artifact, get some unsuspecting victim to clone your modified repository, and cause mischief that way. Note, however, that this is a pre-image attack, which is rather more difficult to pull off than the collision attacks against SHA1, and (to my knowledge) has never been publicly demonstrated. Furthermore, the evil.c file with the same SHA1 hash would need to be valid C code that does something evil while still yielding the same hash (good luck with that!) and Fossil (like Git) has also switched over to Hardened SHA1, making the attack even harder still. As still more defense, Fossil also maintains a MD5 hash against the entire content of the commit. So, in addition to finding evil.c that compiles, does your evil bidding, has the same hardened-SHA1 hash as floppy.c, you also have to make sure that the entire commit has the same MD5 hash after substituting the text of evil.c in place of floppy.c. So, no, it is not really practical to hack a Fossil repository as you describe.
  - wyoung2 7y ago
    > Furthermore, the evil.c file with the same SHA1 hash would need to be valid C code that does something evil while still yielding the same hash ...and also produce an innocent-looking diff! I mean, you could stuff a bunch of random bytes into a C comment to force the desired hash in the output using these documented attack techniques, but anyone inspecting the diffs between versions is likely to see such an explosion of noise and call foul. If you want an analogy, it's like someone saying they've learned to impersonate federal agent identification cards, only it requires that the person carrying the fake ID to have a thousand rainbow-dyed ducks on a leash in tow behind him. Such attacks are fine when it's dumb software systems doing the checks, but for a source code repository where people do in fact visually check the diffs occasionally? Well, let's just say that when someone manages to use SHAttered and/or SHAmbles type attacks on Git (or even Fossil) I expect that it won't take a genius detective to see that the repo's been attacked.
    - tjoff 7y ago
      Many diff tools don't highlight whitespace-only changes. Or at least not in a clear manner. Also, if something is replaced in the history how often do people go back and view diffs in old code? Hardly often enough to rely on it being spotted.
      - wyoung2 7y ago
        It only takes one person to raise the flag. Sure, many thousands of people doing blind "git clone && configure && sudo make install" could be burned by a problem like this, but someone would eventually do a diff and see the problem on any project big enough to have those thousands of trusting users in the first place. I'm not excusing these SHA-1 weaknesses, only pointing out that it won't be trivial to apply them to program source code repos no matter how cheap the attacks get. For instance, the demonstration case for SHAttered was a pair of PDFs: humans can't reasonably inspect those to find whatever noise had to be stuffed into them to achieve the result. I also understand that these SHA-1 weaknesses have been used to attack X.509 certificates, but there again you have a case very unlike a software code repo, where the one doing the checking isn't another programmer but a program.
- wyoung2 7y ago
  In addition to D. Richard Hipp's thoughts as HN user SQLite — author also of Fossil, so he oughtta know — I offer these: 1. Keep in mind that Fossil and Git are both applications of blockchain technology, which in this particular practical case means you must not only forge a single artifact's hash, you must also do it in a way that allows it to fit into the overall blockchain. 2. Fossil's sync protocol purposefully won't apply Dr. Hipp's hypothetical evil.c to an existing Fossil blockchain if presented it. Fossil will say, "I've already got that one, thanks," and move on. Only new or outdated clones could be so-fooled.
  - ajkjk 7y ago
    > applications of blockchain technology Are we saying this now? More like blockchain is an application of git technology.
    - seniorsassycat 7y ago
      https://en.wikipedia.org/wiki/Merkle_tree https://en.wikipedia.org/wiki/Merkle_tree
      - SQLite 7y ago
        "blockchain" is self-descriptive, easier to pronounce (only two syllables instead of three), and easier to spell correctly. :-)
        ajkjk 7y ago
        "blockchain technology" is a lot more syllables. Plus there's the downside of sounding like a loon.
    - _jal 7y ago
      No. We are not. If you're looking for prior art, ZFS's application of Merkle trees predates both. I think there was some other public use before that, but I can't recall right now.
- Tyr42 7y ago
  They are also using "Hardened SHA1", which detects collision attacks, and assigns a longer id to commits which seem malicious, while being backwards compatible.
dariusj18 7y ago
So if a repo has anyone commit to it using a new binary, then anyone accessing the repo will need the new binary as well?

seniorsassycat 7y ago

The way I read the fossil's authors comments, old commits continue to use sha1 hashes. A repository will be vulnerable to sha1 collision attacks as long as there is an object in the repository that has not been hashed with the new algorithm. For example, floppy.c could be replaced in a repo with file with the same sha1 hash as long as the last commit that modifies floppy.c used a sha1 hash. Right?

SQLite 7y ago
Just to be clear: Every time you modify a file, the new changes get put in using SHA3. In an older repository, any given commit might have some files identified using SHA1 (assuming they have not changed in 3 years) and others identified using SHA3. For example, the manifest of the latest SQLite check-in is see at (https://www.sqlite.org/src/artifact/29a969d6b1709b80 https://www.sqlite.org/src/artifact/29a969d6b1709b80). You can see that most of the files have longer SHA3 hashes, but some of the files that have not been touched in three years still carry SHA1 hashes. An attack like what you describe is possible if you could generate an evil.c file that has the exact same SHA1 hash as the older floppy.c file. Then you could substitute the evil.c artifact in place of the floppy.c artifact, get some unsuspecting victim to clone your modified repository, and cause mischief that way. Note, however, that this is a pre-image attack, which is rather more difficult to pull off than the collision attacks against SHA1, and (to my knowledge) has never been publicly demonstrated. Furthermore, the evil.c file with the same SHA1 hash would need to be valid C code that does something evil while still yielding the same hash (good luck with that!) and Fossil (like Git) has also switched over to Hardened SHA1, making the attack even harder still. As still more defense, Fossil also maintains a MD5 hash against the entire content of the commit. So, in addition to finding evil.c that compiles, does your evil bidding, has the same hardened-SHA1 hash as floppy.c, you also have to make sure that the entire commit has the same MD5 hash after substituting the text of evil.c in place of floppy.c. So, no, it is not really practical to hack a Fossil repository as you describe.
- wyoung2 7y ago
  > Furthermore, the evil.c file with the same SHA1 hash would need to be valid C code that does something evil while still yielding the same hash ...and also produce an innocent-looking diff! I mean, you could stuff a bunch of random bytes into a C comment to force the desired hash in the output using these documented attack techniques, but anyone inspecting the diffs between versions is likely to see such an explosion of noise and call foul. If you want an analogy, it's like someone saying they've learned to impersonate federal agent identification cards, only it requires that the person carrying the fake ID to have a thousand rainbow-dyed ducks on a leash in tow behind him. Such attacks are fine when it's dumb software systems doing the checks, but for a source code repository where people do in fact visually check the diffs occasionally? Well, let's just say that when someone manages to use SHAttered and/or SHAmbles type attacks on Git (or even Fossil) I expect that it won't take a genius detective to see that the repo's been attacked.
  - tjoff 7y ago
    Many diff tools don't highlight whitespace-only changes. Or at least not in a clear manner. Also, if something is replaced in the history how often do people go back and view diffs in old code? Hardly often enough to rely on it being spotted.
    - wyoung2 7y ago
      It only takes one person to raise the flag. Sure, many thousands of people doing blind "git clone && configure && sudo make install" could be burned by a problem like this, but someone would eventually do a diff and see the problem on any project big enough to have those thousands of trusting users in the first place. I'm not excusing these SHA-1 weaknesses, only pointing out that it won't be trivial to apply them to program source code repos no matter how cheap the attacks get. For instance, the demonstration case for SHAttered was a pair of PDFs: humans can't reasonably inspect those to find whatever noise had to be stuffed into them to achieve the result. I also understand that these SHA-1 weaknesses have been used to attack X.509 certificates, but there again you have a case very unlike a software code repo, where the one doing the checking isn't another programmer but a program.
wyoung2 7y ago
In addition to D. Richard Hipp's thoughts as HN user SQLite — author also of Fossil, so he oughtta know — I offer these: 1. Keep in mind that Fossil and Git are both applications of blockchain technology, which in this particular practical case means you must not only forge a single artifact's hash, you must also do it in a way that allows it to fit into the overall blockchain. 2. Fossil's sync protocol purposefully won't apply Dr. Hipp's hypothetical evil.c to an existing Fossil blockchain if presented it. Fossil will say, "I've already got that one, thanks," and move on. Only new or outdated clones could be so-fooled.
- ajkjk 7y ago
  > applications of blockchain technology Are we saying this now? More like blockchain is an application of git technology.
  - seniorsassycat 7y ago
    https://en.wikipedia.org/wiki/Merkle_tree https://en.wikipedia.org/wiki/Merkle_tree
    - SQLite 7y ago
      "blockchain" is self-descriptive, easier to pronounce (only two syllables instead of three), and easier to spell correctly. :-)
      - ajkjk 7y ago
        "blockchain technology" is a lot more syllables. Plus there's the downside of sounding like a loon.
  - _jal 7y ago
    No. We are not. If you're looking for prior art, ZFS's application of Merkle trees predates both. I think there was some other public use before that, but I can't recall right now.
Tyr42 7y ago
They are also using "Hardened SHA1", which detects collision attacks, and assigns a longer id to commits which seem malicious, while being backwards compatible.

dariusj18 7y ago

So if a repo has anyone commit to it using a new binary, then anyone accessing the repo will need the new binary as well?