4 ms·

Is this implemented with any sort of thought to security? I don't see anything that implies as such, all FAQ and comparisons to other message queues say nothin

by tetrep 3y ago

Is this implemented with any sort of thought to security? I don't see anything that implies as such, all FAQ and comparisons to other message queues say nothing of security (I skimmed and ctrl+f so I could have missed it). This doesn't bode well for a network-based application written in a memory unsafe language. edit: I'm surprised so many people are interpeting this as a weird cargo cult thing. Application security includes a lot of mundane and critical things like: Authentication, authorization, message signing / authentication (e.g. HMAC), encryption, secret/key management, how the system handles updates, etc. You can read a bit more about it here: https://en.m.wikipedia.org/wiki/Application_security https://en.m.wikipedia.org/wiki/Application_security

mrits 3y ago
Would you prefer it written in Rust with everything wrapped in unsafe block?
- tetrep 3y ago
  I care more about methodology than specific methods. I would like applications that handle I/O, especially network I/O, to be designed and built with security in mind. I asked my question because that doesn't appear to be the case here, and I think that is dissapointing and concerning from a security perspective.
- coder543 3y ago
  That is an obviously false dichotomy. The author of the comment never mentioned Rust, and wasn’t asking about Rust. Such a false dichotomy feels like flamebait, which is against HN guidelines. Your false dichotomy also implies that “unsafe” blocks are as unsafe as C++, which is not true. “Unsafe” in Rust turns off very few checks[0], most of them are still active. No one would write a serious Rust program entirely in unsafe anyways. Regardless, asking about security considerations is a valid thing to do, even if it were written in Rust. Security is not just about memory safety. [0]: https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abi...
  - touisteur 3y ago
    Furthermore, choice of language can have an effect as to the actual security of the networking layer. Having parsers (and serializers) proved absent of runtime errors (e.g. with something like SPARK) is a form of guarantee I wish I'd see as the default in any 'serious' network-facing library or component. It's not even that hard to achieve, compared to the learning curve of the borrow checker. Once rust gets plugged into Why3 and gets some industrial-grade proof capacity, the question of 'is it written in rust?' will be automatic (as in 'why would you do it any other way?').
  - mrits 3y ago
    You probably want to spend a bit of time in the Rust community before your speak on it.
- mikepurvis 3y ago
  Rust's guarantees are all about what's happening inside your own process with memory ownership. "Security" in the context of a middleware like this is almost certainly more about external validation/auth— what prevents some random node from injecting messages into the queue, or adding itself as a subscriber to the output? Also, it's a super bad-faith argument to talk about an entire program being in an unsafe block. Rust's unsafe is about specifically and intentionally opting out of certain guardrails in a limited scope of code where it should be fairly easy for a human reviewer to understand and validate the more complex invariants that aren't enforceable by the compiler itself.
  - mrits 3y ago
    This is absolutely not what Rust unsafe block is about. It is about getting around the limitations of the compiler to gain efficiency. And it happens a lot.
    - mikepurvis 3y ago
      > Unsafe Rust exists because, by nature, static analysis is conservative. When the compiler tries to determine whether or not code upholds the guarantees, it’s better for it to reject some valid programs than to accept some invalid programs. Although the code might be okay, if the Rust compiler doesn’t have enough information to be confident, it will reject the code. In these cases, you can use unsafe code to tell the compiler, “Trust me, I know what I’m doing.” https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html
      - mrits 3y ago
        You might want to read the page you linked "you can give up guaranteed safety in exchange for greater performance "
rewmie 3y ago
> Is this implemented with any sort of thought to security? There's a hefty load of cargo cult mentality revolving the topic of security. Security is not a product, but a process. This sort of talk is getting tiresome.
ljm 3y ago
Security purely in terms of the language used, or security in terms of cluster access, authentication and so on? I mean, I don't often see documentation for networked services that goes in-depth on how exactly they write memory safe code in C or C++, as if that would mean the application is therefore secure; it's always much higher level.
- tetrep 3y ago
  > Security purely in terms of the language used, or security in terms of cluster access, authentication and so on? I'm personally interested in both, but from a docs perspective I'd mostly expect the latter, more "security for users of this system" than anything else. I'm a little embarassed I didn't remember it until now, but I think the general term is "application security." I'm concerned that I don't see any hints of either in the project. I'm not at a computer with access to my Github account so I can't easily search the source code for hints of obvious signs of care or neglect with respect to security.
  - ljm 3y ago
    https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293d38244a9ada6dc8cbafe5711/docs/docs/introduction/roadmap.md?plain=1#L48-L59 https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293... Not sure where they are on the roadmap but authentication in particular is 'mid-term' but lacks detail. I expect that if you want to deploy this then you're doing it on an internal network; its deployment examples are based on Docker so I expect it's relying on what you can do with k8s.
    - tetrep 3y ago
      Thanks! That is the sort of stuff I was asking about. It looks like it's an after thought but at least on their mind now, which is very fair with respect to Bloomberg's wants/needs. It'd be nice if they had a bit of a warning about using this until it has some basic auth(n) and TLS since they're releasing it to the public. I think it is, relativley speaking, rude to release insecure networked software without giving users a notice as to what sorts of insecurity is at least known/expected.
cogman10 3y ago
I perused the code and it was written in a C++03 style. Pointer chasing and manual memory management in the few files I poked through. (IE, no unique_ptr, just new/delete). This may be perfectly safe... however, I'd give it a few years of battle testing before touching it. Here's an example: https://github.com/bloomberg/blazingmq/blob/main/src/groups/bmq/bmqimp/bmqimp_messagedumper.cpp https://github.com/bloomberg/blazingmq/blob/main/src/groups/...
- d4damager 3y ago
  I don’t see any examples of new/delete there? Also they do make heavy use of managed pointers in the codebase.
  - MathMonkeyMan 3y ago
    Life is harder without move semantics, but allocators are easier.
- HWR_14 3y ago
  Isn't this opensourced after 8 years of being battletested in a very time/error sensitive environment (feeding bloomberg terminals)?
  - cogman10 3y ago
    Lol. I'm sorry but my work has me directly interfacing with Bloomberg and other financial institutions. If anything, I trust the code LESS because of who wrote it. Just because software is heavily used and deals with finance, doesn't mean it's secure or well written. Consider the Equifax breach of 2017, everyone's credit info leaked because Equifax was using a crusty old version of struts with known critical CVEs.
    - HWR_14 3y ago
      I thought everyone's info leaked because an Equifax office in South America had open access to the internet with default passwords.

by tetrep 3y ago

mrits 3y ago

Would you prefer it written in Rust with everything wrapped in unsafe block?

tetrep 3y ago
I care more about methodology than specific methods. I would like applications that handle I/O, especially network I/O, to be designed and built with security in mind. I asked my question because that doesn't appear to be the case here, and I think that is dissapointing and concerning from a security perspective.
coder543 3y ago
That is an obviously false dichotomy. The author of the comment never mentioned Rust, and wasn’t asking about Rust. Such a false dichotomy feels like flamebait, which is against HN guidelines. Your false dichotomy also implies that “unsafe” blocks are as unsafe as C++, which is not true. “Unsafe” in Rust turns off very few checks[0], most of them are still active. No one would write a serious Rust program entirely in unsafe anyways. Regardless, asking about security considerations is a valid thing to do, even if it were written in Rust. Security is not just about memory safety. [0]: https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abilities https://doc.rust-lang.org/std/keyword.unsafe.html#unsafe-abi...
- touisteur 3y ago
  Furthermore, choice of language can have an effect as to the actual security of the networking layer. Having parsers (and serializers) proved absent of runtime errors (e.g. with something like SPARK) is a form of guarantee I wish I'd see as the default in any 'serious' network-facing library or component. It's not even that hard to achieve, compared to the learning curve of the borrow checker. Once rust gets plugged into Why3 and gets some industrial-grade proof capacity, the question of 'is it written in rust?' will be automatic (as in 'why would you do it any other way?').
- mrits 3y ago
  You probably want to spend a bit of time in the Rust community before your speak on it.
mikepurvis 3y ago
Rust's guarantees are all about what's happening inside your own process with memory ownership. "Security" in the context of a middleware like this is almost certainly more about external validation/auth— what prevents some random node from injecting messages into the queue, or adding itself as a subscriber to the output? Also, it's a super bad-faith argument to talk about an entire program being in an unsafe block. Rust's unsafe is about specifically and intentionally opting out of certain guardrails in a limited scope of code where it should be fairly easy for a human reviewer to understand and validate the more complex invariants that aren't enforceable by the compiler itself.
- mrits 3y ago
  This is absolutely not what Rust unsafe block is about. It is about getting around the limitations of the compiler to gain efficiency. And it happens a lot.
  - mikepurvis 3y ago
    > Unsafe Rust exists because, by nature, static analysis is conservative. When the compiler tries to determine whether or not code upholds the guarantees, it’s better for it to reject some valid programs than to accept some invalid programs. Although the code might be okay, if the Rust compiler doesn’t have enough information to be confident, it will reject the code. In these cases, you can use unsafe code to tell the compiler, “Trust me, I know what I’m doing.” https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html
    - mrits 3y ago
      You might want to read the page you linked "you can give up guaranteed safety in exchange for greater performance "

rewmie 3y ago

> Is this implemented with any sort of thought to security? There's a hefty load of cargo cult mentality revolving the topic of security. Security is not a product, but a process. This sort of talk is getting tiresome.

ljm 3y ago

Security purely in terms of the language used, or security in terms of cluster access, authentication and so on? I mean, I don't often see documentation for networked services that goes in-depth on how exactly they write memory safe code in C or C++, as if that would mean the application is therefore secure; it's always much higher level.

tetrep 3y ago
> Security purely in terms of the language used, or security in terms of cluster access, authentication and so on? I'm personally interested in both, but from a docs perspective I'd mostly expect the latter, more "security for users of this system" than anything else. I'm a little embarassed I didn't remember it until now, but I think the general term is "application security." I'm concerned that I don't see any hints of either in the project. I'm not at a computer with access to my Github account so I can't easily search the source code for hints of obvious signs of care or neglect with respect to security.
- ljm 3y ago
  https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293d38244a9ada6dc8cbafe5711/docs/docs/introduction/roadmap.md?plain=1#L48-L59 https://github.com/bloomberg/blazingmq/blob/93e6426d474a4293... Not sure where they are on the roadmap but authentication in particular is 'mid-term' but lacks detail. I expect that if you want to deploy this then you're doing it on an internal network; its deployment examples are based on Docker so I expect it's relying on what you can do with k8s.
  - tetrep 3y ago
    Thanks! That is the sort of stuff I was asking about. It looks like it's an after thought but at least on their mind now, which is very fair with respect to Bloomberg's wants/needs. It'd be nice if they had a bit of a warning about using this until it has some basic auth(n) and TLS since they're releasing it to the public. I think it is, relativley speaking, rude to release insecure networked software without giving users a notice as to what sorts of insecurity is at least known/expected.

cogman10 3y ago

I perused the code and it was written in a C++03 style. Pointer chasing and manual memory management in the few files I poked through. (IE, no unique_ptr, just new/delete). This may be perfectly safe... however, I'd give it a few years of battle testing before touching it. Here's an example: https://github.com/bloomberg/blazingmq/blob/main/src/groups/bmq/bmqimp/bmqimp_messagedumper.cpp https://github.com/bloomberg/blazingmq/blob/main/src/groups/...

d4damager 3y ago
I don’t see any examples of new/delete there? Also they do make heavy use of managed pointers in the codebase.
- MathMonkeyMan 3y ago
  Life is harder without move semantics, but allocators are easier.
HWR_14 3y ago
Isn't this opensourced after 8 years of being battletested in a very time/error sensitive environment (feeding bloomberg terminals)?
- cogman10 3y ago
  Lol. I'm sorry but my work has me directly interfacing with Bloomberg and other financial institutions. If anything, I trust the code LESS because of who wrote it. Just because software is heavily used and deals with finance, doesn't mean it's secure or well written. Consider the Equifax breach of 2017, everyone's credit info leaked because Equifax was using a crusty old version of struts with known critical CVEs.
  - HWR_14 3y ago
    I thought everyone's info leaked because an Equifax office in South America had open access to the internet with default passwords.