5 ms·

> However, this means that the most commonly used crate is shipping an arbitrary binary for production builds without any way to opt-out. The nokogiri rubygem

> However, this means that the most commonly used crate is shipping an arbitrary binary for production builds without any way to opt-out. The nokogiri rubygem has been shipping a prebuilt windows dll for probably more than a decade. The sun has not imploded into a black hole at any point in the meantime. > If we're going to be trusting some random guy's binaries, I think we are in the right to demand that it is byte-for-byte reproducible on commodity hardware without having to reverse-engineer the build process and figure out which nightly version of the compiler is being used to compile this binary blob that will be run everywhere. I have some really, really fucking bad news about the O/S distros most of you all use...

yw3410 3y ago
The author uses NixOS.
- chungy 3y ago
  Debian and Fedora have likewise both been pushing reproducible builds.
  - lamontcg 3y ago
    I used to use a FreeBSD desktop at work back in like 2002. And that is a reasonable thing for those projects to try to achieve. Still we've been downloading binaries for decades and the world hasn't ended, this issue never warranted people treating it like nuclear war was imminent and hurling abuse into Dtolnay's github issues like he was a child pornographer. There's a way to have a discussion about this being the wrong direction without doing that. And I'd encourage everyone to take an honest audit of how many precompiled binaries are in their lives, even if they've switched to NixOS and how much stuff they trust downloading from the internet. Very few people achieve an RMS-level of purity in their technological lives.
    - jacquesm 3y ago
      This is all true, but even if you don't care about reproducible builds that doesn't mean that nobody cares about reproducible builds. And the people that do care about such things care about them very much. To the point that if you don't offer it to them that they'll go somewhere else.
      - lamontcg 3y ago
        I didn't say not to try to get to reproducible builds, though, but not to treat Dtolnay like he's a criminal. There is a very real problem that he was trying to solve, but the right way to do it will require core language support. And his heart is certainly in the right place, since "slow build times" is probably the #1 complaint about Rust. If you actually polled most Rust users they probably care about their build times more than they care about this issue, particularly if you designed the survey impartially and just asked them to stack rank priorities. They probably all would prefer "both" being the right answer though. Where the criticism really needs to be leveled is on the core language design (although I appreciate that their job is also extremely tough). [And the one good thing to come out of this shitshow will probably be getting people focused on solving this issue the right way]
        jacquesm 3y ago
        It's funny how opposed the 'Go' and 'Rust' eco systems are, the 'Go' people optimized for compile speed from day 1 but have trouble making things safe, the 'Rust' people optimized for safety from day 1 but have trouble making things fast. Maybe they could learn a trick or two from each other?
        _gabe_ 3y ago
        I don’t program in Go, but this is the first I’ve heard of it being unsafe. I thought Go was a GC language implemented via the Go runtime and properly handles memory safety (memory corruption and OOB access). Are you using a different definition of safety here or is there something else that I’m not aware of?
- lamontcg 3y ago
  "Most of you all", addressing the HN audience, not the author.
  - whatisyour 3y ago
    most distros are moving towards reproducible builds.
ptx 3y ago
With a distro we're trusting a single trustworthy source for all the software, whereas with package managers for most programming languages we're trusting hundreds of individual developers, any one of which can suddenly decide to sell us out.
- hsbauauvhabzb 3y ago
  Distro maintainers also apply some level of audit prior to merging packages. I doubt it’s a huge amount but it’s infinitely more than directly trusting a package maintainer directly.
  - est31 3y ago
    The distro maintainers definitely had been in contact with the maintainer about the binary change, and I'm very thankful for them to have done that.
    - hsbauauvhabzb 3y ago
      Sorry to clarify I’m talking generically me installing something from cargo, pip or npm, rather than any specific package. I personally don’t hold much value in ‘reproducible builds’ as being some silver bullet - back doors can almost as easily be hidden in source code, particularly if nobody gives it a once-over read, same goes for compiled code only it normally requires a sandbox and some R.E.
      - est31 3y ago
        I disagree, it is easier to hide back doors in binaries than in source code. "some RE" is way harder to do on scale than looking at diffs of source code. But yes, it's also possible to put back doors into source code as well. Still, I think in the open source world, the primary distribution method, at least the one emanating from the original developer, should be in forms of source code, to ensure that the original developer can't include special closed source features. Also, for end users it's easier when the source code is available, then they can rebuild things using standardized methodology. all the distro packages have a bunch of commands you can run and then you can rebuild the distro package.
vbezhenar 3y ago
One of the most used platforms nowadays is Java. Java libraries are distributed as binary bytecode blobs. They can also include native libraries, which usually are extracted to /tmp and dynamically loaded into the process. The security of this approach is questionable, if you ask me, however that's the way things were for the long time and that's the way things will be for the long time. I think that striving for better security is a noble goal, but if that prevents usability, security should step aside, when the world around is not as secure anyway.
- richbell 3y ago
  > Java libraries are distributed as binary bytecode blobs. I don't think it's fair to describe JARs this way. While it may be technically true, a JAR file is far less opaque than an actual binary executable.
  - connicpu 3y ago
    In the sense that it's easier to decompile sure, but plenty of non-OSS java libraries run an obfuscator on their internals and in that case it's only marginally less opaque than machine code
- pgwhalen 3y ago
  > that's the way things will be for the long time. There are some changes coming where application developers will have to explicitly acknowledge that libraries they're using are using native code: https://openjdk.org/jeps/8307341 https://openjdk.org/jeps/8307341 (the new, soon-to-be-stable FFM API already has this behavior). It doesn't change the distribution mechanism of native code, but it's some sort of step towards better security.

yw3410 3y ago

The author uses NixOS.

chungy 3y ago
Debian and Fedora have likewise both been pushing reproducible builds.
- lamontcg 3y ago
  I used to use a FreeBSD desktop at work back in like 2002. And that is a reasonable thing for those projects to try to achieve. Still we've been downloading binaries for decades and the world hasn't ended, this issue never warranted people treating it like nuclear war was imminent and hurling abuse into Dtolnay's github issues like he was a child pornographer. There's a way to have a discussion about this being the wrong direction without doing that. And I'd encourage everyone to take an honest audit of how many precompiled binaries are in their lives, even if they've switched to NixOS and how much stuff they trust downloading from the internet. Very few people achieve an RMS-level of purity in their technological lives.
  - jacquesm 3y ago
    This is all true, but even if you don't care about reproducible builds that doesn't mean that nobody cares about reproducible builds. And the people that do care about such things care about them very much. To the point that if you don't offer it to them that they'll go somewhere else.
    - lamontcg 3y ago
      I didn't say not to try to get to reproducible builds, though, but not to treat Dtolnay like he's a criminal. There is a very real problem that he was trying to solve, but the right way to do it will require core language support. And his heart is certainly in the right place, since "slow build times" is probably the #1 complaint about Rust. If you actually polled most Rust users they probably care about their build times more than they care about this issue, particularly if you designed the survey impartially and just asked them to stack rank priorities. They probably all would prefer "both" being the right answer though. Where the criticism really needs to be leveled is on the core language design (although I appreciate that their job is also extremely tough). [And the one good thing to come out of this shitshow will probably be getting people focused on solving this issue the right way]
      - jacquesm 3y ago
        It's funny how opposed the 'Go' and 'Rust' eco systems are, the 'Go' people optimized for compile speed from day 1 but have trouble making things safe, the 'Rust' people optimized for safety from day 1 but have trouble making things fast. Maybe they could learn a trick or two from each other?
        _gabe_ 3y ago
        I don’t program in Go, but this is the first I’ve heard of it being unsafe. I thought Go was a GC language implemented via the Go runtime and properly handles memory safety (memory corruption and OOB access). Are you using a different definition of safety here or is there something else that I’m not aware of?
lamontcg 3y ago
"Most of you all", addressing the HN audience, not the author.
- whatisyour 3y ago
  most distros are moving towards reproducible builds.

ptx 3y ago

With a distro we're trusting a single trustworthy source for all the software, whereas with package managers for most programming languages we're trusting hundreds of individual developers, any one of which can suddenly decide to sell us out.

hsbauauvhabzb 3y ago
Distro maintainers also apply some level of audit prior to merging packages. I doubt it’s a huge amount but it’s infinitely more than directly trusting a package maintainer directly.
- est31 3y ago
  The distro maintainers definitely had been in contact with the maintainer about the binary change, and I'm very thankful for them to have done that.
  - hsbauauvhabzb 3y ago
    Sorry to clarify I’m talking generically me installing something from cargo, pip or npm, rather than any specific package. I personally don’t hold much value in ‘reproducible builds’ as being some silver bullet - back doors can almost as easily be hidden in source code, particularly if nobody gives it a once-over read, same goes for compiled code only it normally requires a sandbox and some R.E.
    - est31 3y ago
      I disagree, it is easier to hide back doors in binaries than in source code. "some RE" is way harder to do on scale than looking at diffs of source code. But yes, it's also possible to put back doors into source code as well. Still, I think in the open source world, the primary distribution method, at least the one emanating from the original developer, should be in forms of source code, to ensure that the original developer can't include special closed source features. Also, for end users it's easier when the source code is available, then they can rebuild things using standardized methodology. all the distro packages have a bunch of commands you can run and then you can rebuild the distro package.

vbezhenar 3y ago

One of the most used platforms nowadays is Java. Java libraries are distributed as binary bytecode blobs. They can also include native libraries, which usually are extracted to /tmp and dynamically loaded into the process. The security of this approach is questionable, if you ask me, however that's the way things were for the long time and that's the way things will be for the long time. I think that striving for better security is a noble goal, but if that prevents usability, security should step aside, when the world around is not as secure anyway.

richbell 3y ago
> Java libraries are distributed as binary bytecode blobs. I don't think it's fair to describe JARs this way. While it may be technically true, a JAR file is far less opaque than an actual binary executable.
- connicpu 3y ago
  In the sense that it's easier to decompile sure, but plenty of non-OSS java libraries run an obfuscator on their internals and in that case it's only marginally less opaque than machine code
pgwhalen 3y ago
> that's the way things will be for the long time. There are some changes coming where application developers will have to explicitly acknowledge that libraries they're using are using native code: https://openjdk.org/jeps/8307341 https://openjdk.org/jeps/8307341 (the new, soon-to-be-stable FFM API already has this behavior). It doesn't change the distribution mechanism of native code, but it's some sort of step towards better security.