6 ms·

Note that if you do this then you will opt out of any security updates not just for jq but also for its regular expression parsing dependency onigurama. For exa

by rlpb 2y ago

Note that if you do this then you will opt out of any security updates not just for jq but also for its regular expression parsing dependency onigurama. For example, there was a security update for onigurama previously; if this sort of thing happens again, you'd be vulnerable, and jq is often used to parse untrusted JSON. > * SECURITY UPDATE: Fix multiple invalid pointer dereference, out-of-bounds write memory corruption and stack buffer overflow. (that one was for CVE-2017-9224, CVE-2017-9226, CVE-2017-9227, CVE-2017-9228 and CVE-2017-9229)

topspin 2y ago
This is certainly true. Also, by replacing the allocator and changing compiler flags, you're possibly immunizing yourself from attacks that rely on some specific memory layout.
- smarx007 2y ago
  There are also many flags that should be enabled by default for non-debug builds like ubsan, stack protection, see https://news.ycombinator.com/item?id=35758898 https://news.ycombinator.com/item?id=35758898
  - ryao 2y ago
    UBSAN is usually a debug build only thing. You can run it in production for some added safety, but it comes at a performance cost and theoretically, if you test all execution paths on a debug build and fix all complaints, there should be no benefit to running it in production.
    - smarx007 2y ago
      I think it's time for the C/C++ communities to consider a mindset shift and pivot to having almost all protectors, canaries, sanitizers, assertions (e.g. via _GLIBCXX_ASSERTIONS) on by default and recommended for use in release builds in production. The opposite (i.e, the current state of affairs) should be discouraged and begrudginly accepted in select few cases. https://www.youtube.com/watch?v=gG4BJ23BFBE https://www.youtube.com/watch?v=gG4BJ23BFBE is a presentation that best represents my view on the kind of mindset that's long overdue to become the new norm in our industry.
      - ryao 2y ago
        I do not think things like the time command need to be compiled with such things. It is pointless, but your suggestion here is to do it anyway. Why bother? Assertions in release builds are a bad idea since they can be fairly expensive. It is a better idea to have a different variety of assertion like the verify statements that OpenZFS uses, which are assertions that run even in release builds. They are used in situations where it is extremely important for an assertion to be done at runtime, without the performance overhead of the less important assertions that are in performance critical paths.
        smarx007 2y ago
        Why would I want potentially undefined behaviour in 'time'? I expect it to crash anytime it's about to enter UB. Sure, you may want to minimize such statements between the start/stop of the timer, but I expect any processing of stdout/stderr of the child process to be UB-proofed as much as possible. I think it's a philosophical difference of opinions and it's one of the things that drive Rust, Go, C# etc. ahead - not merely language ergonomics (I hope Zig ends up as the language that replaces C). The society at large is mostly happy to take a 1-3% perf hit to get rid of buffer overflows and other UB-inducing errors. But I agree with you on not having "expensive" asserts in releases.
- internetter 2y ago
  That is, if you are a believer in security via obscurity
  - frontfor 2y ago
    Are you arguing that ASLR is “security via obscurity”?
    - topspin 2y ago
      I would, and there is no shame in it, as far as I'm concerned. I don't need to outrun the bear. I just need to outrun you.
      - msm_ 2y ago
        ASLR is not security through obscurity though. It forces attacker to get a pointer leak before doing almost anything (even arbitrary read and arbitrary write primitives are useless without a leak with ASLR). As someone with a bit of experience in exploit dev, it makes a world of a difference and is one of the most influential hardenings, next to maybe stack cookies and W^X.
        ryao 2y ago
        ASLR obscures the memory layout. That is security by obscurity by definition. People thought this was okay if the entropy was high enough, but then the ASLR⊕Cache attack was published and now its usefulness is questionable.
        usefulcat 2y ago
        I'm genuinely curious what was so undesirable about this sibling comment that it was removed: "ASLR obscures the memory layout. That is security by obscurity by definition. People thought this was okay if the entropy was high enough, but then the ASLR⊕Cache attack was published and now its usefulness is questionable." Usually when a comment is removed, it's pretty obvious why, but in this case I'm really not seeing it at all. I read up (briefly) on the mentioned attack and can confirm that the claims made in the above comment are at the very least plausible sounding. I checked other comments from that user and don't see any other recent ones that were removed, so it doesn't seem to be a user-specific thing. I realize this is completely off-topic, but I'd really like to understand why it was removed. Perhaps it was removed by mistake?
- estebarb 2y ago
  By hardwiring the allocator you may end up with binaries that load two different allocators. It is too fun to debug a program that is using jemalloc free to release memory allocated by glibc. Unless you know what you are doing, it is better to leave it as is.
ryao 2y ago
A userland package manager like Gentoo Prefix could be used to install a custom build of this and still get security updates.
- rlpb 2y ago
  Indeed, there are many methods to have a custom build and still get security updates, including at least one method that is native to Ubuntu and doesn’t need any external tooling. However my warning refers to the method presented in the article, where this isn’t the case.
  - alextingle 2y ago
    > including at least one method that is native to Ubuntu and doesn’t need any external tooling. Can you explain a little more? Search has failed me on this one.
Onavo 2y ago
What about PGO?
- ryao 2y ago
  https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization
jeffbee 2y ago
This is generally true but specifically false. The builds described in the gist are still linking onigurama dynamically. It is in another package, libonig5, that would be updated normally.
- rlpb 2y ago
  The gist uses the orig tarball only, so skips the distro patch that selects the distro packaged libonig in favor of the "vendored" one. At least that's how it appears to me. I only skimmed the situation. Or do you see something deeper that ensures that the distro libonig is actually the one that gets used?
jmward01 2y ago
But isn't there still the kernel of an idea here for a package management system that intelligently decides to build based on platform? Seems like a lot of performance to leave on the table.
- stkdump 2y ago
  Rebuilding from scratch also takes longer than installing a prebuilt package. So while it might be worth it for a heavily used application, in general I doubt it. Also I think in earlier days the argument to build was so you can optimize the application for the specific capabilities of your system like the supported SIMD instruction set or similar. I think nowadays that is much less of a factor. Instead it would probably be better to do things like that on a package or distribution level (i.e. have one binary distribution package prebuilt by the distribution for different CPU capabilities).
taeric 2y ago
I'm curious how applicable these are, in general? Feels like pointing out that using interior doors in your house misses out on the security afforded from a vault door. Not wrong, but there is also a reason every door in a bank is not a vault door. That is, I don't want to devalue the CVE system; but it is also undeniable that there are major differences in impact between findings?
- skrebbel 2y ago
  > I don't want to devalue the CVE system You could, though. It's 99.9% stuff like this!
- lpapez 2y ago
  In my experience, most CVEs are reports about ice cream trucks lacking nuclear-proof bank vault doors.
- rlpb 2y ago
  Sure, but jq is very much a "front door" in your analogy. You'd have to look at each individual CVE to assess the risk for your specific case, but for jq, claimed security vulnerabilities are worth paying attention to.
guappa 2y ago
The normal way is to use dpkg to rebuild and patch, and use dch to increase the patch version with a .1 or something similar, so that the OS version always takes precedence, and then rebuild.

by rlpb 2y ago

topspin 2y ago

This is certainly true. Also, by replacing the allocator and changing compiler flags, you're possibly immunizing yourself from attacks that rely on some specific memory layout.

smarx007 2y ago
There are also many flags that should be enabled by default for non-debug builds like ubsan, stack protection, see https://news.ycombinator.com/item?id=35758898 https://news.ycombinator.com/item?id=35758898
- ryao 2y ago
  UBSAN is usually a debug build only thing. You can run it in production for some added safety, but it comes at a performance cost and theoretically, if you test all execution paths on a debug build and fix all complaints, there should be no benefit to running it in production.
  - smarx007 2y ago
    I think it's time for the C/C++ communities to consider a mindset shift and pivot to having almost all protectors, canaries, sanitizers, assertions (e.g. via _GLIBCXX_ASSERTIONS) on by default and recommended for use in release builds in production. The opposite (i.e, the current state of affairs) should be discouraged and begrudginly accepted in select few cases. https://www.youtube.com/watch?v=gG4BJ23BFBE https://www.youtube.com/watch?v=gG4BJ23BFBE is a presentation that best represents my view on the kind of mindset that's long overdue to become the new norm in our industry.
    - ryao 2y ago
      I do not think things like the time command need to be compiled with such things. It is pointless, but your suggestion here is to do it anyway. Why bother? Assertions in release builds are a bad idea since they can be fairly expensive. It is a better idea to have a different variety of assertion like the verify statements that OpenZFS uses, which are assertions that run even in release builds. They are used in situations where it is extremely important for an assertion to be done at runtime, without the performance overhead of the less important assertions that are in performance critical paths.
      - smarx007 2y ago
        Why would I want potentially undefined behaviour in 'time'? I expect it to crash anytime it's about to enter UB. Sure, you may want to minimize such statements between the start/stop of the timer, but I expect any processing of stdout/stderr of the child process to be UB-proofed as much as possible. I think it's a philosophical difference of opinions and it's one of the things that drive Rust, Go, C# etc. ahead - not merely language ergonomics (I hope Zig ends up as the language that replaces C). The society at large is mostly happy to take a 1-3% perf hit to get rid of buffer overflows and other UB-inducing errors. But I agree with you on not having "expensive" asserts in releases.
internetter 2y ago
That is, if you are a believer in security via obscurity
- frontfor 2y ago
  Are you arguing that ASLR is “security via obscurity”?
  - topspin 2y ago
    I would, and there is no shame in it, as far as I'm concerned. I don't need to outrun the bear. I just need to outrun you.
    - msm_ 2y ago
      ASLR is not security through obscurity though. It forces attacker to get a pointer leak before doing almost anything (even arbitrary read and arbitrary write primitives are useless without a leak with ASLR). As someone with a bit of experience in exploit dev, it makes a world of a difference and is one of the most influential hardenings, next to maybe stack cookies and W^X.
      - ryao 2y ago
        ASLR obscures the memory layout. That is security by obscurity by definition. People thought this was okay if the entropy was high enough, but then the ASLR⊕Cache attack was published and now its usefulness is questionable.
      - usefulcat 2y ago
        I'm genuinely curious what was so undesirable about this sibling comment that it was removed: "ASLR obscures the memory layout. That is security by obscurity by definition. People thought this was okay if the entropy was high enough, but then the ASLR⊕Cache attack was published and now its usefulness is questionable." Usually when a comment is removed, it's pretty obvious why, but in this case I'm really not seeing it at all. I read up (briefly) on the mentioned attack and can confirm that the claims made in the above comment are at the very least plausible sounding. I checked other comments from that user and don't see any other recent ones that were removed, so it doesn't seem to be a user-specific thing. I realize this is completely off-topic, but I'd really like to understand why it was removed. Perhaps it was removed by mistake?
estebarb 2y ago
By hardwiring the allocator you may end up with binaries that load two different allocators. It is too fun to debug a program that is using jemalloc free to release memory allocated by glibc. Unless you know what you are doing, it is better to leave it as is.

ryao 2y ago

A userland package manager like Gentoo Prefix could be used to install a custom build of this and still get security updates.

rlpb 2y ago
Indeed, there are many methods to have a custom build and still get security updates, including at least one method that is native to Ubuntu and doesn’t need any external tooling. However my warning refers to the method presented in the article, where this isn’t the case.
- alextingle 2y ago
  > including at least one method that is native to Ubuntu and doesn’t need any external tooling. Can you explain a little more? Search has failed me on this one.

Onavo 2y ago

What about PGO?

ryao 2y ago
https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization

jeffbee 2y ago

This is generally true but specifically false. The builds described in the gist are still linking onigurama dynamically. It is in another package, libonig5, that would be updated normally.

rlpb 2y ago
The gist uses the orig tarball only, so skips the distro patch that selects the distro packaged libonig in favor of the "vendored" one. At least that's how it appears to me. I only skimmed the situation. Or do you see something deeper that ensures that the distro libonig is actually the one that gets used?

jmward01 2y ago

But isn't there still the kernel of an idea here for a package management system that intelligently decides to build based on platform? Seems like a lot of performance to leave on the table.

stkdump 2y ago
Rebuilding from scratch also takes longer than installing a prebuilt package. So while it might be worth it for a heavily used application, in general I doubt it. Also I think in earlier days the argument to build was so you can optimize the application for the specific capabilities of your system like the supported SIMD instruction set or similar. I think nowadays that is much less of a factor. Instead it would probably be better to do things like that on a package or distribution level (i.e. have one binary distribution package prebuilt by the distribution for different CPU capabilities).

taeric 2y ago

I'm curious how applicable these are, in general? Feels like pointing out that using interior doors in your house misses out on the security afforded from a vault door. Not wrong, but there is also a reason every door in a bank is not a vault door. That is, I don't want to devalue the CVE system; but it is also undeniable that there are major differences in impact between findings?

skrebbel 2y ago
> I don't want to devalue the CVE system You could, though. It's 99.9% stuff like this!
lpapez 2y ago
In my experience, most CVEs are reports about ice cream trucks lacking nuclear-proof bank vault doors.
rlpb 2y ago
Sure, but jq is very much a "front door" in your analogy. You'd have to look at each individual CVE to assess the risk for your specific case, but for jq, claimed security vulnerabilities are worth paying attention to.

guappa 2y ago

The normal way is to use dpkg to rebuild and patch, and use dch to increase the patch version with a .1 or something similar, so that the OS version always takes precedence, and then rebuild.