5 ms·

Having to recompile all applications and libraries to remove the potential crypto bug is a pretty scary bug :-/

by 0x0 11y ago

Armand_Grillet 11y ago
Fortunately, the bug's frequency is really low: "On 64-bit systems, the frequency of the bug is [...] less than one in 2^50".
JoachimSchipper 11y ago
Linking everything statically is not always an advantage...
- coldtea 11y ago
  In this case it actually is. Imagine the nightmare of having to hunt tens of shared libraries versions, and even have to re-build them and patch them with the correct version of the language/apis so that they continue to work.
  - eropple 11y ago
    How does that square? I mean, I have one copy of a shared library on any of my machines. I don't even necessarily know off the top of my head what of the Go tools my developers found bloggable enough to want in the stack that might be impacted by this bug. (Or rather, I do, but that's because it's my job, and I'm actually good at my job. I don't have the same high hopes with regards to most other infrastructure folks I've worked with.)
    - JamesMcMinn 11y ago
      You don't need to know what Go tools your "developers found bloggable enough to want in the stack". You simply need to rebuild whatever binaries you run, this will re-compile everything (libraries and all) and prevent the exploit. What libraries you or your developers use is irrelevant. As an aside, do you have any idea how pretentious you come across?
      - nindalf 11y ago
        I can't understand why someone making a statement like that ("bloggable..") which is untrue can be upvoted while you're downvoted. Worse, he even bragged about being an excellent engineer who knows everything there is to know when he clearly doesn't in this case.
      - elithrar 11y ago
        `go build -a` rebuilds all packages (i.e. those already cached under $GOPATH/pkg/ or /vendor). Alternatively you can just rm -rf your `$GOPATH/pkg` directory.
      - eropple 11y ago
        > You simply need to rebuild whatever binaries you run "Stack", not "app". That includes, say, Docker. Or nsq. Which are packaged by the operating system (because, as noted in a reply to `tptacek, paying me to build out what they get from the operating system is generally not something a client is going to want to do). And which will not be updated as conscientiously as I can expect a core library on my system to be updated. > As an aside, do you have any idea how pretentious you come across? Pretension, frustration, whichever. My low opinion of Go aside, it's my job to deal with the details of my clients' stacks, make a coherent experience for their developers, and deal with the system-level security concerns they don't think about. If getting annoyed that Go and its community make my life suck a little bit more in ways it need not suck is pretension, I'm really gonna be okay with that. I mean, jeez, it's 2016. "Rebuild the entire world when a mouse farts" is fragile, dangerous garbage that puts the lie to the idea of "engineering" in this profession. We should be better than this. (And Go's first iteration, lovingly called Java 1.4, was. We're going backwards.)
      - dang 11y ago
        > As an aside, do you have any idea how pretentious you come across? No personal attacks on HN, please. This comment would be fine without that last sentence.
    - coldtea 11y ago
      >How does that square? I mean, I have one copy of a shared library on any of my machines. That's all well and good, but that's just you. Real systems often get messed with multiple copies/versions of shared libraries. Also known as dll-hell in Windows. In Java land you could easily end up with 20 versions of some jars just as easily, all installed locally by maven and needed by this or that lib/framework/etc. >Or rather, I do, but that's because it's my job, and I'm actually good at my job. Citation needed.
      - eropple 11y ago
        I haven't seen that on a decently-maintained Linux system--certainly not any I'd call a "real system"--in a really long time, unless they're side-by-side packages from the OS.
        coldtea 11y ago
        What "Linux system"? We're not talking about your distro userland packages here. We're talking about businesses deploying multiple server apps and such. Neither Java nor .NET shops for example rely on "OS packages" for their server dependencies when it comes to Jars, dlls, etc.
        eropple 11y ago
        Yes, actually, I was talking about distro userland packages, but whatever. If you're deploying a Java application, everything should be coming out of Maven (and thus avoid any sort of shared library hell) in the first place, no?
        coldtea 11y ago
        >Yes, actually, I was talking about distro userland packages, but whatever. Then you were off topic, since we were discussing Golang packages for deployment, which aren't userland packages either. But whatever. >If you're deploying a Java application, everything should be coming out of Maven (and thus avoid any sort of shared library hell) in the first place, no? You still get multiple copies of jars. And you still need to replace them. And you still might have older projects that need particular versions of a jar that you need to update somehow in case such a long-reaching issue is discovered (and upstream might not do that at all).
- ori_b 11y ago
  You're right. Let's put it in a docker image instead, and ship shared libraries that way.
tacticus 11y ago
Thankfully it's rather trivial to do.
- 0x0 11y ago
  It's trivial to do when you know which binary to replace (and you have the source code available). But it's probably not so trivial to hunt down every single old binary on every system. There's also a continuous risk that you may receive old binaries from third parties in the future. For example, gitlab has started to ship a component written in Go in recent versions. Things like these are easy to forget about.
  - karaziox 11y ago
    If you can't track the binaries you are running you have a bigger problem than recompiling some of them. Security issues aren't the one in a decade thing, if you can't easily locate which code has to be updated, you are in trouble already...
  - donatj 11y ago
    I'd say as a general overzealous rule that if at least two people in the company don't know the code, it shouldn't be publicly accessible if it being compromised could cause you financial harm.
ominous_prime 11y ago
Not really. Go doesn't use shared libraries (yet), so you're only really recompiling the end binaries. If you're using Go, you should already have the infrastructure to do this. Everything we have gets rebuilt for each go release anyway, so this is no different, just a little more immediate.
- eropple 11y ago
  > If you're using Go, you should already have the infrastructure to do this. Unless you're using packages in an APT or Yum repo. You instead get to wait for packages to come downstream, almost certainly not in anything approaching synchronicity. Awesome. We got away from statically linked monoliths for a reason.
  - bliti 11y ago
    Feeling your pain right now. I have a service running in go that runs in multiple machines and this will keep me busy for the night. Worst is that I can't have a sysadmin do the whole process. Now QA has to sign off on it... It's no wonder things stay broken for a long time. It is a lotnif work. Sorry for the rant. :)
    - helper 11y ago
      Isn't that what automation tools are for?
      - bliti 11y ago
        You can't blindly trust automated tools. They also don't replace protocol.
        ownagefool 11y ago
        That's why you write tests. Not that always works either, but it's not like a qa team of point and clixkers running over a manual set of those same tests will always get it right.

Having to recompile all applications and libraries to remove the potential crypto bug is a pretty scary bug :-/

by 0x0 11y ago

Having to recompile all applications and libraries to remove the potential crypto bug is a pretty scary bug :-/

Fortunately, the bug's frequency is really low: "On 64-bit systems, the frequency of the bug is [...] less than one in 2^50".

JoachimSchipper 11y ago

Linking everything statically is not always an advantage...

coldtea 11y ago
In this case it actually is. Imagine the nightmare of having to hunt tens of shared libraries versions, and even have to re-build them and patch them with the correct version of the language/apis so that they continue to work.
- eropple 11y ago
  How does that square? I mean, I have one copy of a shared library on any of my machines. I don't even necessarily know off the top of my head what of the Go tools my developers found bloggable enough to want in the stack that might be impacted by this bug. (Or rather, I do, but that's because it's my job, and I'm actually good at my job. I don't have the same high hopes with regards to most other infrastructure folks I've worked with.)
  - JamesMcMinn 11y ago
    You don't need to know what Go tools your "developers found bloggable enough to want in the stack". You simply need to rebuild whatever binaries you run, this will re-compile everything (libraries and all) and prevent the exploit. What libraries you or your developers use is irrelevant. As an aside, do you have any idea how pretentious you come across?
    - nindalf 11y ago
      I can't understand why someone making a statement like that ("bloggable..") which is untrue can be upvoted while you're downvoted. Worse, he even bragged about being an excellent engineer who knows everything there is to know when he clearly doesn't in this case.
    - elithrar 11y ago
      `go build -a` rebuilds all packages (i.e. those already cached under $GOPATH/pkg/ or /vendor). Alternatively you can just rm -rf your `$GOPATH/pkg` directory.
    - eropple 11y ago
      > You simply need to rebuild whatever binaries you run "Stack", not "app". That includes, say, Docker. Or nsq. Which are packaged by the operating system (because, as noted in a reply to `tptacek, paying me to build out what they get from the operating system is generally not something a client is going to want to do). And which will not be updated as conscientiously as I can expect a core library on my system to be updated. > As an aside, do you have any idea how pretentious you come across? Pretension, frustration, whichever. My low opinion of Go aside, it's my job to deal with the details of my clients' stacks, make a coherent experience for their developers, and deal with the system-level security concerns they don't think about. If getting annoyed that Go and its community make my life suck a little bit more in ways it need not suck is pretension, I'm really gonna be okay with that. I mean, jeez, it's 2016. "Rebuild the entire world when a mouse farts" is fragile, dangerous garbage that puts the lie to the idea of "engineering" in this profession. We should be better than this. (And Go's first iteration, lovingly called Java 1.4, was. We're going backwards.)
    - dang 11y ago
      > As an aside, do you have any idea how pretentious you come across? No personal attacks on HN, please. This comment would be fine without that last sentence.
  - coldtea 11y ago
    >How does that square? I mean, I have one copy of a shared library on any of my machines. That's all well and good, but that's just you. Real systems often get messed with multiple copies/versions of shared libraries. Also known as dll-hell in Windows. In Java land you could easily end up with 20 versions of some jars just as easily, all installed locally by maven and needed by this or that lib/framework/etc. >Or rather, I do, but that's because it's my job, and I'm actually good at my job. Citation needed.
    - eropple 11y ago
      I haven't seen that on a decently-maintained Linux system--certainly not any I'd call a "real system"--in a really long time, unless they're side-by-side packages from the OS.
      - coldtea 11y ago
        What "Linux system"? We're not talking about your distro userland packages here. We're talking about businesses deploying multiple server apps and such. Neither Java nor .NET shops for example rely on "OS packages" for their server dependencies when it comes to Jars, dlls, etc.
        eropple 11y ago
        Yes, actually, I was talking about distro userland packages, but whatever. If you're deploying a Java application, everything should be coming out of Maven (and thus avoid any sort of shared library hell) in the first place, no?
        coldtea 11y ago
        >Yes, actually, I was talking about distro userland packages, but whatever. Then you were off topic, since we were discussing Golang packages for deployment, which aren't userland packages either. But whatever. >If you're deploying a Java application, everything should be coming out of Maven (and thus avoid any sort of shared library hell) in the first place, no? You still get multiple copies of jars. And you still need to replace them. And you still might have older projects that need particular versions of a jar that you need to update somehow in case such a long-reaching issue is discovered (and upstream might not do that at all).
ori_b 11y ago
You're right. Let's put it in a docker image instead, and ship shared libraries that way.

tacticus 11y ago

Thankfully it's rather trivial to do.

0x0 11y ago
It's trivial to do when you know which binary to replace (and you have the source code available). But it's probably not so trivial to hunt down every single old binary on every system. There's also a continuous risk that you may receive old binaries from third parties in the future. For example, gitlab has started to ship a component written in Go in recent versions. Things like these are easy to forget about.
- karaziox 11y ago
  If you can't track the binaries you are running you have a bigger problem than recompiling some of them. Security issues aren't the one in a decade thing, if you can't easily locate which code has to be updated, you are in trouble already...
- donatj 11y ago
  I'd say as a general overzealous rule that if at least two people in the company don't know the code, it shouldn't be publicly accessible if it being compromised could cause you financial harm.

ominous_prime 11y ago

Not really. Go doesn't use shared libraries (yet), so you're only really recompiling the end binaries. If you're using Go, you should already have the infrastructure to do this. Everything we have gets rebuilt for each go release anyway, so this is no different, just a little more immediate.

eropple 11y ago
> If you're using Go, you should already have the infrastructure to do this. Unless you're using packages in an APT or Yum repo. You instead get to wait for packages to come downstream, almost certainly not in anything approaching synchronicity. Awesome. We got away from statically linked monoliths for a reason.
- bliti 11y ago
  Feeling your pain right now. I have a service running in go that runs in multiple machines and this will keep me busy for the night. Worst is that I can't have a sysadmin do the whole process. Now QA has to sign off on it... It's no wonder things stay broken for a long time. It is a lotnif work. Sorry for the rant. :)
  - helper 11y ago
    Isn't that what automation tools are for?
    - bliti 11y ago
      You can't blindly trust automated tools. They also don't replace protocol.
      - ownagefool 11y ago
        That's why you write tests. Not that always works either, but it's not like a qa team of point and clixkers running over a manual set of those same tests will always get it right.