6 ms·

A tool to make Windows builds reproducible

80 points by BuuQu9hu 10y ago

Seylerius 10y ago
On the one hand, my knee-jerk reaction to this involves a meaningful amount of Windows-hate. In short: there is a tight limit to the amount of trust I can give to any build on a platform that I can't build. On the other hand, reproducible builds are /important/. The better we can trust our tools and verify that they're exactly what we think they are, the less we need to worry about the security of our tools, and the more we can get on with /using them/. In the end, despite enabling the continued use of a deeply proprietary and anti-freedom platform, this is a good thing.
- davexunit 10y ago
  Agreed. The GNU Guix project (of which I am one of the developers) sees reproducibility as a means to an end: empowering users with verifiable, trustable software that they can control. Thus, reproducible builds on a proprietary system, while better than non-reproducible builds on a proprietary system, don't accomplish very much for user freedom. https://www.gnu.org/software/guix/news/reproducible-builds-a-means-to-an-end.html https://www.gnu.org/software/guix/news/reproducible-builds-a...
metaphor 10y ago
Are modern compilers truly that deterministic when processing arbitrarily complex code, shoot-yourself-in-the-foot example aside? In the reconfigurable world, place and route engines run metaheuristics[1][2] to find good-enough solutions in an intractable search space. I'm imagining if there's a compiler stage which implements even a single optimization heuristic, any prospect of hash-wise reproducibility goes out the window? [1] https://en.wikipedia.org/wiki/Simulated_annealing https://en.wikipedia.org/wiki/Simulated_annealing [2] https://en.wikipedia.org/wiki/Genetic_algorithm https://en.wikipedia.org/wiki/Genetic_algorithm
- qwename 10y ago
  Last year I reversed engineered a (36KiB) Windows PE binary[1] to obtain C++ source code that compiled to almost the exact same bytes. What stumped me was an undocumented section that contained metadata, which I gave up on figuring out. Assuming you compile with the same options and in the same environment, the machine instructions should be the same. EDIT: The compiler in question though is whatever Visual Basic 6.0 uses, so I wouldn't call it modern today. [1] https://github.com/qwename/quirky-protector https://github.com/qwename/quirky-protector
  - throwaway7767 10y ago
    One of the longstanding issues the Tor project had with their reproducible builds was that the windows builds, built with Visual Studio, would always have a small piece of data in the same place that differed for each build. It took months to figure it out, but it turned out to be a bug in the MS compiler where they were including some uninitialized memory from the compiler into the resulting binary. IIRC it was fixed after being reported to Microsoft. Could be the same or similar issue you are facing, given the age of the compiler. I don't have a reference to link to though, Roger Dingledine told me this in person at a conference when discussing issues related to reproducible builds. But there should be an issue in the tor bug tracker.
- BuuQu9hu 10y ago
  If you call GCC modern, then yes: https://tests.reproducible-builds.org/debian/ https://tests.reproducible-builds.org/debian/
- johntb86 10y ago
  If your random numbers are generated with a fixed seed and the algorithm runs for a deterministic number of steps (e.g. not using a wall-clock timeout) then the output should be deterministic. One potential for nondeterminism is if the compiler or linker is multithreaded, and multiple threads could update a shared data structure in no fixed order.
  - metaphor 10y ago
civodul 10y ago
It may be a helpful tool from an engineering viewpoint, but I think it misses the big picture of _why_ reproducible builds matter: https://gnu.org/software/guix/news/reproducible-builds-a-means-to-an-end.html https://gnu.org/software/guix/news/reproducible-builds-a-mea...
jheriko 10y ago
> Git is used to get the current commit hash. that seems a bit of a problem for every product not using git (which is still a lot, and whoever said git was the final solution anyway?)
milesf 10y ago
I remember being new to Information Technology in 1998 when Windows 95 & 98 were the norm. I had more than one client who would take delivery of 10 - 12 new, identical workstations and it was very common to have half with one problem and half without. I'm not familiar with the concepts in this article. Would this explain the sort of issues I had?
- Retr0spectrum 10y ago
  Probably not, no.
ommunist 10y ago
Very nice. But the title imposes that Windows builds are unique without a tool.
ddfs 10y ago
Windows compilers aren't deterministic in some areas. Roslyn already has a switch to do this, /deterministic. There is a spec for this though, https://www.microsoft.com/en-us/download/confirmation.aspx?id=19509 https://www.microsoft.com/en-us/download/confirmation.aspx?i... > A stamp that is used for different purposes in several places in a PE or COFF file. In most cases, the format of each stamp is the same as that used by the time functions in the C run-time library. For exceptions, see the descripton of IMAGE_DEBUG_TYPE_REPRO (section 6.1.2). If the stamp value is 0 or 0xFFFFFFFF, it does not represent a real or meaningful date/time stamp. and > The presence of an entry of type IMAGE_DEBUG_TYPE_REPRO indicates the PE file is built in a way to achieve determinism or reproducibility. If the input does not change, the output PE file is guaranteed to be bit-for-bit identical no matter when or where the PE is produced. Various date/time stamp fields in the PE file are filled with part or all the bits from a calculated hash value that uses PE file content as input, and therefore no longer represent the actual date and time when a PE file or related specific data within the PE is produced. The raw data of this debug entry may be empty, or may contain a calculated hash value preceded by a four-byte value that represents the hash value length.
- fowl2 10y ago
  unless the key to reproducible builds is hidden somewhere inside "How to Implement Compatible IDs in Printing Devices", that link may be wrong
  - ddfs 10y ago
    hit download and it pops up a window where you can select the correct doc. It's PECoff.docx

80 points by BuuQu9hu 10y ago

On the one hand, my knee-jerk reaction to this involves a meaningful amount of Windows-hate. In short: there is a tight limit to the amount of trust I can give to any build on a platform that I can't build. On the other hand, reproducible builds are /important/. The better we can trust our tools and verify that they're exactly what we think they are, the less we need to worry about the security of our tools, and the more we can get on with /using them/. In the end, despite enabling the continued use of a deeply proprietary and anti-freedom platform, this is a good thing.

davexunit 10y ago
Agreed. The GNU Guix project (of which I am one of the developers) sees reproducibility as a means to an end: empowering users with verifiable, trustable software that they can control. Thus, reproducible builds on a proprietary system, while better than non-reproducible builds on a proprietary system, don't accomplish very much for user freedom. https://www.gnu.org/software/guix/news/reproducible-builds-a-means-to-an-end.html https://www.gnu.org/software/guix/news/reproducible-builds-a...

metaphor 10y ago

Are modern compilers truly that deterministic when processing arbitrarily complex code, shoot-yourself-in-the-foot example aside? In the reconfigurable world, place and route engines run metaheuristics[1][2] to find good-enough solutions in an intractable search space. I'm imagining if there's a compiler stage which implements even a single optimization heuristic, any prospect of hash-wise reproducibility goes out the window? [1] https://en.wikipedia.org/wiki/Simulated_annealing https://en.wikipedia.org/wiki/Simulated_annealing [2] https://en.wikipedia.org/wiki/Genetic_algorithm https://en.wikipedia.org/wiki/Genetic_algorithm

qwename 10y ago
Last year I reversed engineered a (36KiB) Windows PE binary[1] to obtain C++ source code that compiled to almost the exact same bytes. What stumped me was an undocumented section that contained metadata, which I gave up on figuring out. Assuming you compile with the same options and in the same environment, the machine instructions should be the same. EDIT: The compiler in question though is whatever Visual Basic 6.0 uses, so I wouldn't call it modern today. [1] https://github.com/qwename/quirky-protector https://github.com/qwename/quirky-protector
- throwaway7767 10y ago
  One of the longstanding issues the Tor project had with their reproducible builds was that the windows builds, built with Visual Studio, would always have a small piece of data in the same place that differed for each build. It took months to figure it out, but it turned out to be a bug in the MS compiler where they were including some uninitialized memory from the compiler into the resulting binary. IIRC it was fixed after being reported to Microsoft. Could be the same or similar issue you are facing, given the age of the compiler. I don't have a reference to link to though, Roger Dingledine told me this in person at a conference when discussing issues related to reproducible builds. But there should be an issue in the tor bug tracker.
BuuQu9hu 10y ago
If you call GCC modern, then yes: https://tests.reproducible-builds.org/debian/ https://tests.reproducible-builds.org/debian/
johntb86 10y ago
If your random numbers are generated with a fixed seed and the algorithm runs for a deterministic number of steps (e.g. not using a wall-clock timeout) then the output should be deterministic. One potential for nondeterminism is if the compiler or linker is multithreaded, and multiple threads could update a shared data structure in no fixed order.
- metaphor 10y ago

civodul 10y ago

It may be a helpful tool from an engineering viewpoint, but I think it misses the big picture of _why_ reproducible builds matter: https://gnu.org/software/guix/news/reproducible-builds-a-means-to-an-end.html https://gnu.org/software/guix/news/reproducible-builds-a-mea...

jheriko 10y ago

> Git is used to get the current commit hash. that seems a bit of a problem for every product not using git (which is still a lot, and whoever said git was the final solution anyway?)

milesf 10y ago

I remember being new to Information Technology in 1998 when Windows 95 & 98 were the norm. I had more than one client who would take delivery of 10 - 12 new, identical workstations and it was very common to have half with one problem and half without. I'm not familiar with the concepts in this article. Would this explain the sort of issues I had?

Retr0spectrum 10y ago
Probably not, no.

ommunist 10y ago

Very nice. But the title imposes that Windows builds are unique without a tool.

ddfs 10y ago

Windows compilers aren't deterministic in some areas. Roslyn already has a switch to do this, /deterministic. There is a spec for this though, https://www.microsoft.com/en-us/download/confirmation.aspx?id=19509 https://www.microsoft.com/en-us/download/confirmation.aspx?i... > A stamp that is used for different purposes in several places in a PE or COFF file. In most cases, the format of each stamp is the same as that used by the time functions in the C run-time library. For exceptions, see the descripton of IMAGE_DEBUG_TYPE_REPRO (section 6.1.2). If the stamp value is 0 or 0xFFFFFFFF, it does not represent a real or meaningful date/time stamp. and > The presence of an entry of type IMAGE_DEBUG_TYPE_REPRO indicates the PE file is built in a way to achieve determinism or reproducibility. If the input does not change, the output PE file is guaranteed to be bit-for-bit identical no matter when or where the PE is produced. Various date/time stamp fields in the PE file are filled with part or all the bits from a calculated hash value that uses PE file content as input, and therefore no longer represent the actual date and time when a PE file or related specific data within the PE is produced. The raw data of this debug entry may be empty, or may contain a calculated hash value preceded by a four-byte value that represents the hash value length.

fowl2 10y ago
unless the key to reproducible builds is hidden somewhere inside "How to Implement Compatible IDs in Printing Devices", that link may be wrong
- ddfs 10y ago
  hit download and it pops up a window where you can select the correct doc. It's PECoff.docx