8 ms·

This is a great first step but we're not done yet. It proves the binaries are built from the published code, but only when the published code has been thoroughl

by wai1234 13y ago

This is a great first step but we're not done yet. It proves the binaries are built from the published code, but only when the published code has been thoroughly vetted can we conclude there is no backdoor.

edwintorok 13y ago
Also the build is not deterministic yet, even on his own machine he got some differences in truecrypt.sys between successive builds. By definition the build is not (yet) fully deterministic then: "Using the same source and same project directory results in the same pattern of difference in the block starting at 0002CBAC, as the pattern shown between my build from the correct project directory and the original file. This means that this difference is a normal result of the compilation process, and can be considered harmless from our point of view"
- tripzilch 13y ago
  Indeed, that part itches me a little as well. I share the author's belief it's probably benign, and on the assumption that the source code contains no crazy obfuscated magic, it'd be pretty hard to actually hide malicious code or behaviour there. Still if they want to be thorough (and they should), I'd have liked a bit more explanation than waving it away with a "well these bytes seem to change all the time, so that's okay then". That's not really a justification, it's just an explanation. And it's not okay.
  - acqq 13y ago
    One possible explanation is that the compiler and linker don't produce 100.00% same output because they have somewhere uninitialized space which gets written to the disk. That's why disassembly can still match 100.00%: the bytes that differ don't contribute to any observable functionality. If you know C, you can imagine how it happens: You have somewhere char buff[ 32 ] which you don't set all to zeros, then you do strcpy( buff, "something" ) then you write all 32 bytes of buff to the file. What's behind "something"? It can be different in every run. Now look at the picture: https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binaries-analysis/img/tcsys64-un_hex.png https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binarie... See the "RSDS"? It's some initial value, just like c:\truec... that follows some bytes later. And what's behind? Some small sequence of random bytes. I've just checked and confirmed that it appears in the ".rdata" (read only data) section of the executable. Now the fact that this doesn't happen on much more places also means that somebody in Microsoft obviously from time to time does fully clean the code, that is, somebody worries about such effects too, but it seems that from time to time some "late" fix then slips through, misbehaving. Another explanation would be that there is something in the building code or script that produces some different values in the initialization area. That can be then observed by source inspection. The third explanation would be some kind of "unique id" generated by the compiler or linker. Then this effect should be observed in all binaries even when the source is fully different (e.g. compile some program which generates Vogon poetry, observe the same effect). This hypothesis doesn't match the observations, according to the pictures presented. So I believe the highest possibility is the first assumption being true.
    - moyix 13y ago
      RSDS is part of the debug information section. The structure looks like (using the construct library's syntax): CV_RSDS_HEADER = Struct("CV_RSDS", Const(Bytes("Signature", 4), "RSDS"), GUID("GUID"), ULInt32("Age"), CString("Filename"), ) So it's a GUID, and one property of GUIDs is that they're "Globally unique", hence regenerated each time. Mystery solved. Edit: Here's a source [1] so you don't have to take my word for it. [1] http://www.godevtool.com/Other/pdb.htm http://www.godevtool.com/Other/pdb.htm
- 0x0 13y ago
  The disassembly for both versions of the file was a 100% match, though, which is a pretty good indicator that the binary difference must be something unimportant and not related to the actual code.
  - edwintorok 13y ago
    Yes that is a good step towards fully deterministic build. There are ways to manipulate the PE file though to achieve slightly different behaviour. If you want to be thorough and make sure that your binary was not infected by malware for example then comparing the binaries makes sense.
  - ynik 13y ago
    It's possible that the difference is in the actual code: x86 machine code sometimes has multiple different encodings for the same assembler instruction. These would show up as single-byte differences in the binary. See http://www.strchr.com/machine_code_redundancy http://www.strchr.com/machine_code_redundancy for some examples. Compilers might use this to stenographically hide additional data in the binary. Printer manufacturers already do something similar (https://en.wikipedia.org/wiki/Printer_steganography https://en.wikipedia.org/wiki/Printer_steganography). I wouldn't be surprised if MS compilers embedded a hidden data in binaries -- it could be useful to track down malware authors; or identify software created with pirated MS tooling.
    - oakwhiz 13y ago
      Perhaps this could even be used by an organization to differentiate official binaries of closed-source software from leaked binaries.
    - 0x0 13y ago
      If you read the whole article, you'd see that every single bit difference is accounted for; none of the bit differences occur in the actual code.
- lallysingh 13y ago
  It seems like if you can have the tools alter the embedded timestamp (even as a post-processing step), you can match the binaries pretty closely. The last bit is the signature, which you can't duplicate, but you can also just take out, as it's not code. But, if you're paranoid about that, zero-out or remove the signature after verifying it.
nilved 13y ago
Why are people so adamant about auditing TrueCrypt instead of just migrating to an alternative?
- josephlord 13y ago
  TrueCrypt is highly trusted, they are verifying that trust is well placed. What alternative were you proposing?
  - nilved 13y ago
    LUKS, GPG as examples only. I feel like time would be better spent switching to something less suspicious than trying to prove the suspicions false.
    - jlgreco 13y ago
      Why do you think that a binary distribution of LUKS deserves less suspicion than binary distributions of Truecrypt? If everyone were using LUKS instead, one would hope that people would attempt to verify that trust in binary distributions LUKS was well-placed as well.
    - rmk2 13y ago
      GPG doesn't do disk encryption (full or any), and LUKS is linux-only. That's what keeps people from "just migrating away"... It doesn't matter what you or I personally prefer, if you want what at least seems to be trustworthy and secure[1], cross-platform cryptography, Truecrypt is what you want. [1]: Obviously, that is the part that is slowly being evaluated and tested. We'll see what'll happen.
      - nilved 13y ago
        What's the point of implementing cryptography on a closed-source OS? We'll audit TrueCrypt and then have people say "there can still be backdoors in Windows or Mac OS." When it comes to security and cryptography, Linux (and similar) are the only things that matter.

by wai1234 13y ago

edwintorok 13y ago

Also the build is not deterministic yet, even on his own machine he got some differences in truecrypt.sys between successive builds. By definition the build is not (yet) fully deterministic then: "Using the same source and same project directory results in the same pattern of difference in the block starting at 0002CBAC, as the pattern shown between my build from the correct project directory and the original file. This means that this difference is a normal result of the compilation process, and can be considered harmless from our point of view"

tripzilch 13y ago
Indeed, that part itches me a little as well. I share the author's belief it's probably benign, and on the assumption that the source code contains no crazy obfuscated magic, it'd be pretty hard to actually hide malicious code or behaviour there. Still if they want to be thorough (and they should), I'd have liked a bit more explanation than waving it away with a "well these bytes seem to change all the time, so that's okay then". That's not really a justification, it's just an explanation. And it's not okay.
- acqq 13y ago
  One possible explanation is that the compiler and linker don't produce 100.00% same output because they have somewhere uninitialized space which gets written to the disk. That's why disassembly can still match 100.00%: the bytes that differ don't contribute to any observable functionality. If you know C, you can imagine how it happens: You have somewhere char buff[ 32 ] which you don't set all to zeros, then you do strcpy( buff, "something" ) then you write all 32 bytes of buff to the file. What's behind "something"? It can be different in every run. Now look at the picture: https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binaries-analysis/img/tcsys64-un_hex.png https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binarie... See the "RSDS"? It's some initial value, just like c:\truec... that follows some bytes later. And what's behind? Some small sequence of random bytes. I've just checked and confirmed that it appears in the ".rdata" (read only data) section of the executable. Now the fact that this doesn't happen on much more places also means that somebody in Microsoft obviously from time to time does fully clean the code, that is, somebody worries about such effects too, but it seems that from time to time some "late" fix then slips through, misbehaving. Another explanation would be that there is something in the building code or script that produces some different values in the initialization area. That can be then observed by source inspection. The third explanation would be some kind of "unique id" generated by the compiler or linker. Then this effect should be observed in all binaries even when the source is fully different (e.g. compile some program which generates Vogon poetry, observe the same effect). This hypothesis doesn't match the observations, according to the pictures presented. So I believe the highest possibility is the first assumption being true.
  - moyix 13y ago
    RSDS is part of the debug information section. The structure looks like (using the construct library's syntax): CV_RSDS_HEADER = Struct("CV_RSDS", Const(Bytes("Signature", 4), "RSDS"), GUID("GUID"), ULInt32("Age"), CString("Filename"), ) So it's a GUID, and one property of GUIDs is that they're "Globally unique", hence regenerated each time. Mystery solved. Edit: Here's a source [1] so you don't have to take my word for it. [1] http://www.godevtool.com/Other/pdb.htm http://www.godevtool.com/Other/pdb.htm
0x0 13y ago
The disassembly for both versions of the file was a 100% match, though, which is a pretty good indicator that the binary difference must be something unimportant and not related to the actual code.
- edwintorok 13y ago
  Yes that is a good step towards fully deterministic build. There are ways to manipulate the PE file though to achieve slightly different behaviour. If you want to be thorough and make sure that your binary was not infected by malware for example then comparing the binaries makes sense.
- ynik 13y ago
  It's possible that the difference is in the actual code: x86 machine code sometimes has multiple different encodings for the same assembler instruction. These would show up as single-byte differences in the binary. See http://www.strchr.com/machine_code_redundancy http://www.strchr.com/machine_code_redundancy for some examples. Compilers might use this to stenographically hide additional data in the binary. Printer manufacturers already do something similar (https://en.wikipedia.org/wiki/Printer_steganography https://en.wikipedia.org/wiki/Printer_steganography). I wouldn't be surprised if MS compilers embedded a hidden data in binaries -- it could be useful to track down malware authors; or identify software created with pirated MS tooling.
  - oakwhiz 13y ago
    Perhaps this could even be used by an organization to differentiate official binaries of closed-source software from leaked binaries.
  - 0x0 13y ago
    If you read the whole article, you'd see that every single bit difference is accounted for; none of the bit differences occur in the actual code.
lallysingh 13y ago
It seems like if you can have the tools alter the embedded timestamp (even as a post-processing step), you can match the binaries pretty closely. The last bit is the signature, which you can't duplicate, but you can also just take out, as it's not code. But, if you're paranoid about that, zero-out or remove the signature after verifying it.

nilved 13y ago

Why are people so adamant about auditing TrueCrypt instead of just migrating to an alternative?

josephlord 13y ago
TrueCrypt is highly trusted, they are verifying that trust is well placed. What alternative were you proposing?
- nilved 13y ago
  LUKS, GPG as examples only. I feel like time would be better spent switching to something less suspicious than trying to prove the suspicions false.
  - jlgreco 13y ago
    Why do you think that a binary distribution of LUKS deserves less suspicion than binary distributions of Truecrypt? If everyone were using LUKS instead, one would hope that people would attempt to verify that trust in binary distributions LUKS was well-placed as well.
  - rmk2 13y ago
    GPG doesn't do disk encryption (full or any), and LUKS is linux-only. That's what keeps people from "just migrating away"... It doesn't matter what you or I personally prefer, if you want what at least seems to be trustworthy and secure[1], cross-platform cryptography, Truecrypt is what you want. [1]: Obviously, that is the part that is slowly being evaluated and tested. We'll see what'll happen.
    - nilved 13y ago
      What's the point of implementing cryptography on a closed-source OS? We'll audit TrueCrypt and then have people say "there can still be backdoors in Windows or Mac OS." When it comes to security and cryptography, Linux (and similar) are the only things that matter.