9 ms·
How I compiled TrueCrypt 7.1a for Win32 and matched the official binaries
- generalpf 13y agoThat's amazing work. Well done to the author.
- junto 13y agoCame here to say this too. It is this kind of in depth investigative work, that has genuinely taken time and smarts to achieve, is what I come to HN to enjoy reading. All credit to the author.
- yeukhon 13y ago"TrueCrypt is a project that doesn't provide deterministic builds." Why? What is the benefit of doing so when everyone wants a deterministic build?
- dublinben 13y agoThey also use a unique license that is incompatible with most other open source projects. They seem to have strong reasons for doing things the way they do, and aren't fond of explaining them.
- im3w1l 13y ago>and aren't fond of explaining them. Who tried to poke them how, and what happened?
- aroch 13y agoDeterministic builds are hard...really hard. The combined power of the Debian community has trouble getting deterministic builds, as does the Tor project.
- yeukhon 13y agoOh! I thought that was the norm. Author made it sounded like TrueCrypt doesn't and others do the way he said it.
- sliverstorm 13y agoWhen I think of deterministic builds, I think of the ROMs on the old GameBoy. Apparently some of the ROMs had whatever empty space was left in the fixed-size image padded with random data pulled straight out of the build host's memory. As if consistent tool versions weren't already enough of a challenge!
- x3c 13y agoJust in case someone (like me) is not familiar with the concept of deterministic builds, this is a good read: https://blog.torproject.org/blog/deterministic-builds-part-one-cyberwar-and-global-compromise https://blog.torproject.org/blog/deterministic-builds-part-o... Synopsis: "deterministic builds" -- packages which are byte-for-byte identical no matter who actually builds them, or what hardware they use.
- antocv 13y agoIsnt nix and guix a kind of solution that the deterministic build problem? They produce builds with hashes of the source and all dependencies and tools used, or so I believe after skimming their manuals.
- edwintorok 13y agoThey're not really a solution if the official binaries are compiled using VC, and you want to compare to that.
- deleted 13y ago[deleted]
- gngeal 13y agoDeterministic builds are hard...really hard. I'd have thought that deterministic builds are really simple unless your toolkit ecosystem is FUBAR. After all, a compiler is a simple function from input to output (unless the FUBAR ecosystem syndrome arises, as I said).
- Lagged2Death 13y agoI wish the term "reproducible builds" were the more popular one. I think that's more accurate.
- 3825 13y agoFor what it is worth, that is what Debian[0] seems to call it as well. [0] https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds
- eterm 13y agoTldr: Binaries didn't match, here's some handwaving at the differences.
- e40 13y agoI think that's inaccurate. The disassembly of them matched perfectly.
- PeterisP 13y agoWell then, what and why are the differences? I mean, if there's an arbitrary data block somewhere, then the "matching disassembly" can have wildly different behavior by simply copying & executing parts of that block.
- voltagex_ 13y agoIt's explained in the article. Timestamps, file paths, certificates and oddities of the PE format.
- hamburglar 13y agoI can't wait until the source audit uncovers a funny little subroutine that loads the certificate from the .EXE, decodes the public key into RAM, and then starts executing it. :) edit: not that this seems like a realistic method of injecting malicious code. If you could get away with that in an open source project, you could probably get away with just hiding the malicious code in the app directly.
- PeterisP 13y agoI got an impression from the article that disassembly was what he did to explain the binary differences that remained AFTER he corrected for timestamps/certificates/etc.
- Dylan16807 13y agoYour impression was wrong. He showed all of the differences in the screenshots. That's how few there were. Not a single bit of the code portions was different. Only a handful of metadata bytes, plus appended certificate. The disassembly was only there to be cute and emphasize the point.
- zokier 13y agoJust a slightly off-topic question, but WTF does TC require VC 1.52 for?
- sliverstorm 13y agoConsidering they document their build process very carefully, I can only imagine they wound up standardizing on that library a long time ago and keep using it to this day.
- acqq 13y agoYes, even if there is any recent open source alternative to build 16-bit code, selecting the 20 years old compiler binaries (http://en.wikipedia.org/wiki/Visual_C++ http://en.wikipedia.org/wiki/Visual_C++) for which it can be easiest to be sure they haven't changed (there's still enough CD's around produced at that time, I believe I still have one too) is a damn good decision.
- twistedpair 13y agoHmm... 20 year older binaries? Anyone ever read "Reflections on Trusting Trust"? Perhaps they don't trust newer code from MS? Think about it, MS can't backdoor an old binary.
- bri3d 13y agoIt's the easiest way to build C or C++ targeting 16-bit real mode with a Windows host/toolchain, and they use it to compile a boot loader.
- CUViper 13y ago> TrueCrypt is not backdoored in a way that is not visible from the sources ... as long as you also trust the compiler not to introduce any backdoor... (cf. Reflections on Trusting Trust)
- dmix 13y agoOh god, not this conversation again.
- CUViper 13y agoWhy not? If you're being so paranoid about the origin of a binary, you have to at least acknowledge the fact that you're trusting the compiler in making this comparison. So let me throw out an idea that might help justify this trust too. Compile the same TrueCrypt sources with a totally different compiler, then use both binaries in a deterministic way and compare the raw encrypted result. (I'm assuming here that the same encryption keys and data will give the same result, but don't know for sure if that's true.)
- jlgreco 13y agoCreate a piss-poor barebones compiler (compiler A) by hand (literally, if possible. punchcards would allow you to hand-verify the contents of the program in a way that is not susceptible to Thompson's attack (there is no risk that your eyes were built with a compromised compiler)) that suffices to compile a compiler that you want to verify (compiler B). Compiler A should to run on hardware that you can trust (need not be x86). (= ((a-binary b-source) b-source) b-binary) If you trust a-binary, and you trust b-source, then if that returns true you should be able to trust anything created with b-binary. (a-binary b-source) will not equal b-binary, but ((a-binary b-source) b-source) should. If the hardware is not executing binaries as described, then all bets are off. Compiler A could, in theory, be build and run on a homebrew CPU (http://www.homebrewcpu.com/ http://www.homebrewcpu.com/), but if you are putting that much effort into this, this better be your hobby...
- 13y ago
- wai1234 13y agoThis is a great first step but we're not done yet. It proves the binaries are built from the published code, but only when the published code has been thoroughly vetted can we conclude there is no backdoor.
- edwintorok 13y agoAlso the build is not deterministic yet, even on his own machine he got some differences in truecrypt.sys between successive builds. By definition the build is not (yet) fully deterministic then: "Using the same source and same project directory results in the same pattern of difference in the block starting at 0002CBAC, as the pattern shown between my build from the correct project directory and the original file. This means that this difference is a normal result of the compilation process, and can be considered harmless from our point of view"
- tripzilch 13y agoIndeed, that part itches me a little as well. I share the author's belief it's probably benign, and on the assumption that the source code contains no crazy obfuscated magic, it'd be pretty hard to actually hide malicious code or behaviour there. Still if they want to be thorough (and they should), I'd have liked a bit more explanation than waving it away with a "well these bytes seem to change all the time, so that's okay then". That's not really a justification, it's just an explanation. And it's not okay.
- acqq 13y agoOne possible explanation is that the compiler and linker don't produce 100.00% same output because they have somewhere uninitialized space which gets written to the disk. That's why disassembly can still match 100.00%: the bytes that differ don't contribute to any observable functionality. If you know C, you can imagine how it happens: You have somewhere char buff[ 32 ] which you don't set all to zeros, then you do strcpy( buff, "something" ) then you write all 32 bytes of buff to the file. What's behind "something"? It can be different in every run. Now look at the picture: https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binaries-analysis/img/tcsys64-un_hex.png https://madiba.encs.concordia.ca/~x_decarn/truecrypt-binarie... See the "RSDS"? It's some initial value, just like c:\truec... that follows some bytes later. And what's behind? Some small sequence of random bytes. I've just checked and confirmed that it appears in the ".rdata" (read only data) section of the executable. Now the fact that this doesn't happen on much more places also means that somebody in Microsoft obviously from time to time does fully clean the code, that is, somebody worries about such effects too, but it seems that from time to time some "late" fix then slips through, misbehaving. Another explanation would be that there is something in the building code or script that produces some different values in the initialization area. That can be then observed by source inspection. The third explanation would be some kind of "unique id" generated by the compiler or linker. Then this effect should be observed in all binaries even when the source is fully different (e.g. compile some program which generates Vogon poetry, observe the same effect). This hypothesis doesn't match the observations, according to the pictures presented. So I believe the highest possibility is the first assumption being true.
- bliker 13y agoI am just shooting into darkness, but would not it be easier to compile it twice and diff outcomes to find found out what parts are being changed so those can be ruled out?
- pionar 13y agoHe did that further down in the article, when he noticed that file path was encoded in the TrueCrypt.sys file.
- pamparosendo 13y agoI entered just to say it's an incredible work done by this guy... it's been years since I analized a file on hex mode (from Norton Commander, jeje).
- proctor 13y agoit seems to me that the relaxed gpg key verification that the author uses doesn't give us any more assurances regarding the authenticity of the source than a simple hash offered on the website would. i think in this situation, if the author did not intend to attempt more rigorous verification of the truecrypt pgp key, at least cross-checking that the key offered on the site matches the key offered on a public key server pgp.mit.edu for example would be prudent before signing the truecrypt key with your own. Import the .asc file in the keyring (File > Import certificates). Now you should mark the key as trusted: right click on the TrueCrypt Foundation public key in the list under Imported Certificate tab > Change Owner Trust, and set it as I believe checks are casual. You should also generate your own key pair to sign this key in order to show you really trust it and get a nice confirmation when verifying the binary.
- acqq 13y agoYou can do yours checks and compare with the author's! It's a very fast thing to do for the checks you mention. The more people repeat it, the better.
- proctor 13y agoi think there are two concerns here. one is that the source is not tainted by a third party during or before the download. the second (arguably much more important in this case) concern is that the compiled binary matches the source. the second concern is addressed well by the author as far as i can tell, but i think that there is room for improvement in their concerns about the former. i assume they have thought about this and do have at least some concerns because it is mentioned that The PGP signature of the binary can be downloaded through the button PGP Signature, which makes you download TrueCrypt Setup 7.1a.exe.sig over HTTPS (*although with the NSA in the middle, it might not mean much*). [emphasis mine] cross-referencing the pgp signature with at least one other (public) source would go a long way toward allaying those concerns (that the HTTPS might not mean much). this criticism is in no way meant to detract from the rest of the work, and i mean only to refer to pgp sig verification best practices here.
- smegel 13y agoKudos for effort.
- xbeta 13y agoCoolest post I've read today! Good work!
- TheRealWatson 13y agoPlease God, don't let the author be working for the NSA. These days I get suspicious at every "it's all good" piece of news.
- pilif 13y agoThe good thing about his analysis: It has all the information you need to reproduce it and form your own conclusion. So even if he was working for the NSA, by following his steps, you will either come to the same result or not. That's how science works.
- pointernil 13y agoI get the point reg. verifying the Windows-Compiling-Build, but wouldn't the same verification on an open source platform allow for even easier (maybe even automatic) verification? How about an vmware/vbox image setup explicitly for that purpose? Not feasible for windows due to licencing issues, i guess. Also, huge kudos for the effort going into this work. Thanks!