7 ms·

Show HN: Iceoryx2 – Fast IPC Library for Rust, C++, and C

122 points by elfenpiff 2y ago

Hello everyone, Today we released iceoryx2 v0.4! iceoryx2 is a service-based inter-process communication (IPC) library designed to make communication between processes as fast as possible - like Unix domain sockets or message queues, but orders of magnitude faster and easier to use. It also comes with advanced features such as circular buffers, history, event notifications, publish-subscribe messaging, and a decentralized architecture with no need for a broker. For example, if you're working in robotics and need to process frames from a camera across multiple processes, iceoryx2 makes it simple to set that up. Need to retain only the latest three camera images? No problem - circular buffers prevent your memory from overflowing, even if a process is lagging. The history feature ensures you get the last three images immediately after connecting to the camera service, as long as they’re still available. Another great use case is for GUI applications, such as window managers or editors. If you want to support plugins in multiple languages, iceoryx2 allows you to connect processes - perhaps to remotely control your editor or window manager. Best of all, thanks to zero-copy communication, you can transfer gigabytes of data with incredibly low latency. Speaking of latency, on some systems, we've achieved latency below 100ns when sending data between processes - and we haven't even begun serious performance optimizations yet. So, there’s still room for improvement! If you’re in high-frequency trading or any other use case where ultra-low latency matters, iceoryx2 might be just what you need. If you’re curious to learn more about the new features and what’s coming next, check out the full iceoryx2 v0.4 release announcement. Elfenpiff Links: * GitHub: https://github.com/eclipse-iceoryx/iceoryx2 https://github.com/eclipse-iceoryx/iceoryx2 * iceoryx2 v0.4 release announcement: https://ekxide.io/blog/iceoryx2-0-4-release/ https://ekxide.io/blog/iceoryx2-0-4-release/ * crates.io: https://crates.io/crates/iceoryx2 https://crates.io/crates/iceoryx2 * docs.rs: https://docs.rs/iceoryx2/0.4.0/iceoryx2/ https://docs.rs/iceoryx2/0.4.0/iceoryx2/

deleted 2y ago
[deleted]
npalli 2y ago
Congrats on the release. What's the difference between iceoryx and iceoryx2? I don't want to use Rust and want to stick to C++ if possible.
- tbillington 2y ago
  > Language bindings for C and C++ with CMake and Bazel support right out of the box. Python and other languages are coming soon.
- elBoberido 2y ago
  Besides being written in Rust, the big difference is the decentralized approach. With iceoryx1 a central daemon is required but with iceoryx2 this in not the case anymore. Furthermore, more fine grained control over the resources like memory and endpoints like publisher. Overall the architecture is more modular and it should be easier to port iceoryx2 to even more platforms and customize it with 3rd party extension. With this release we have initial support for C and C++. Not all features of the Rust version are supported yet, but the plan is to finish the bindings with the next release. Furthermore, with an upcoming release we will make it trivial to communicate between Rust, C and C++ applications and all the other language bindings we are going to provide, with Python being probably the next one.
  - sebastos 2y ago
    I've been looking around for some kind of design documents that explain how you were able to ditch the central broker, but I haven't found much. Do you have breadcrumbs?
    - simfoo 2y ago
      Same here. Shared memory is one of those things where the kernel could really help some more with reliable cleanup (1). Until then you're mostly doomed to have a rock solid cleanup daemon or are limited to eventual cleanup by restarting processes. I have my doubts that it isn't possible to get into a situation where segments are being exhausted and you're forced to intervene (1) I'm referring to automatic refcounting of shm segments using posix shm (not sys v!) when the last process dies or unmaps
    - elfenpiff 2y ago
      This is a longer story, but I'll try to provide the essence. * All IPC resources are represented in the file system and have a global naming scheme. So if you would like to perform a service discovery, you take a look at the `/tmp/iceoryx2/services` list all service toml files that you are allowed to access and handle them. * Connecting to a service means, under the hood, opening a specific shared memory identified via a naming scheme, adding yourself to the participant list, and receiving/sending data. * Crashing/resource cleanup is done decentrally by every process that has the permissions to perform them. * In a central/broker architecture you would have the central broker that checks this in a loop. * In a decentralized architecture, we defined certain sync points where this is checked. These points are placed so that you check the misbehavior before it would affect you. For instance, when a sender shall send you a message every second but you do not receive it, you would actively check if it is still alive. Other sync points are, when an iceoryx2 node is created or you connect or disconnect to a service. The main point is that the API is decentralized but you can always use it in a central daemon if you like - but you don't have to. It is optional.
emmanueloga_ 2y ago
Looks great! From a quick glance it seems like it is a cross platform shared memory library. Maybe similar to this? [1]. Suggestion: would be cool to have a quick description of the system calls involved for each supported platform [2]. I'm guessing mmap on linux/osx and CreateFileMapping on Windows? -- 1: https://github.com/LiveAsynchronousVisualizedArchitecture/simdb https://github.com/LiveAsynchronousVisualizedArchitecture/si... 2: https://github.com/eclipse-iceoryx/iceoryx2?tab=readme-ov-file#supported-platforms https://github.com/eclipse-iceoryx/iceoryx2?tab=readme-ov-fi...
- elfenpiff 2y ago
  You guessed right. We have a layered architecture that abstracts this away for every platform. With this, we can support every OS as long as it has a way of sharing memory between processes (or tasks as some RTOSes are calling it) and you have a way of sending notifications.
westurner 2y ago
How does this compare to and/or integrate with OTOH Apache Arrow which had "arrow plasma IPC" and is supported by pandas with dtype_backend="pyarrow", lancedb/lancedb, and Serde.rs? https://serde.rs/#data-formats https://serde.rs/#data-formats
- dumah 2y ago
  https://github.com/apache/arrow/issues/34738 https://github.com/apache/arrow/issues/34738 https://lists.apache.org/thread/lk277x3b9gjol42sjg27bst2ggm5s0j2 https://lists.apache.org/thread/lk277x3b9gjol42sjg27bst2ggm5...
  - zxexz 2y ago
    Not downvoting, but those two links don't really describe much - though they are part of the story. For other readers, here's a general takeaway - - Arrow Plasma was deprecated, and is now no longer even present in the Arrow project. - The maintainers behind the Plasma store in Arrow forked it, into Ray. Plasma is still alive and well in Ray - and it still uses the Arrow IPC format, among other things. In addition to the links from the above poster, read the original blog post on Plasma [0], and the section on Ray on [1]. I use Ray quite a bit. For more lightweight stuff, or for more low-level control, I use Arrow Flight [2] (and [3] for Python examples) [0] https://ray-project.github.io/2017/08/08/plasma-in-memory-object-store.html https://ray-project.github.io/2017/08/08/plasma-in-memory-ob... [1] https://arrow.apache.org/powered_by/ https://arrow.apache.org/powered_by/ [2] https://arrow.apache.org/docs/python/flight.html https://arrow.apache.org/docs/python/flight.html [3] https://arrow.apache.org/cookbook/py/flight.html https://arrow.apache.org/cookbook/py/flight.html
- zxexz 2y ago
  The other commenter answering you is, I think, trying to point out that the Arrow plasma store is deprecated (and no longer present in the arrow project). I think it's worth being a little more clear here - Arrow IPC is _not_ deprecated, and has massive momentum - so much so that it's more or less already become the default IPC format for many libraries. To me it remains unclear what the benefits of Iceoryx2 over the Arrow ecosystem is, and what the level of interoperability is, and what the tradeoffs of either are relative to eachother. Within a single machine, you can mmap the IPC file. You can use Arrow Flight for inter-node or inter-process communication. You can use Arrow with Ray, which is where Plasma went. I love anything new in this space though, if/when I have time I'll check this out - would love it if somebody could actually eloborate on the differences though.
hardwaresofton 2y ago
Been doing some IPC experiments recently following the 3tilley post[0], because there just isn't enough definitive information (even if it's a snapshot in time) out there. Shared memory is crazy fast, and I'm surprised that there aren't more things that take advantage of it. Super odd that gRPC doesn't do shared memory, and basically never plans to?[1]. All that said, the constructive criticism I can offer for this post is that in mass-consumption announcements like this one for your project, you should: - RPC throughput (with the usual caveats/disclaimers) - Comparison (ideally graphed) to an alternative approach (ex. domain sockets) - Your best/most concise & expressive usage snippet 100ns is great to know, but I would really like to know how much RPC/s this translates to without doing the math, or seeing it with realistic de-serialization on the other end. [0]: https://3tilley.github.io/posts/simple-ipc-ping-pong/ https://3tilley.github.io/posts/simple-ipc-ping-pong/ [1]: https://github.com/grpc/grpc/issues/19959 https://github.com/grpc/grpc/issues/19959
- abhirag 2y ago
  At $work we are evaluating different IPC strategies in Rust. My colleague expanded upon 3tilley's work, they have updated benchmarks with iceoryx2 included here[0]. I suppose the current release should perform even better. [0]: https://pranitha.rs/posts/rust-ipc-ping-pong/ https://pranitha.rs/posts/rust-ipc-ping-pong/
  - nh2 2y ago
    Interesting that on Linux Unix Domain Sockets are not faster than TCP. People often say that the TCP stack overhead is high but this benchmark does not confirm that.
    - jcelerier 2y ago
      I'm curious about the benchmark. In my own for another network IPC library (https://GitHub.com/ossia/libossia https://GitHub.com/ossia/libossia) Unix sockets were consistently faster than the alternatives when sending the same payloads.
    - 2y ago
forrestthewoods 2y ago
Why is Windows target support tier 2 and not tier 1?
- elfenpiff 2y ago
  Tier 1 also means all security/safety features. Windows is not used in mission-critical systems like cars or plans, so we do not need to add those to Windows. We aim to support Windows so iceoryx2 can be used safely and securely in a desktop environment.
boolit2 2y ago
I'm in robotics education and we mostly work with Python, to make life easier for students. I'd love to push for more Rust, but so far there's no point to it. Multiprocess communication is something that we found lacking in Python (we want everything to be easily pip installable) and we ended up using shared memory primitives, which is a lot of code to maintain. What is the main roadblock for iceoryx2 Python bindings? Is it something you are looking for contributors for?
- orecham 2y ago
  The only real roadblock for this is time. We'd welcome any contributor, especially with such an impactful contribution as this. Making iceoryx2 an option for Python would be amazing. If you or someone you know wants to take it on, we would support as much as possible. So far for this topic, we have only done some brief research on the options available to us, such us going over the C API or using something like PyO3.
fefe23 2y ago
This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on. Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in the framework before handing it off). Their web page mentions zero copy, so it's probably not the second one. Also, benchmarks are misleading. It's easy to get good latency if your throughput is so high that you can do polling or spin locks, like for example in benchmarks. But that's probably not a good assumption for general usage because it will be very inefficient and waste power and require more cooling as well.
- zbentley 2y ago
  > Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith) There are all sorts of domains where mutually-trusted parties need IPC. Off the top of my head and in no particular order: - Applications that pass validated data to/from captive subprocesses. Not everything is available as a natively-linked library. Not every language's natively-linked libraries are as convenient to reliably install as external binaries. - Parallelism/server systems farming work out to forked (but not exec'd) subprocesses. Not everything needs setuid. Somtimes you just want to parallelize number crunching without the headache of threads (or are on a platform like Python which limits threads' usefulness). - Replatforming/language transitions in data-intensive applications. Running the new runtime/platform in the same address space as the legacy platform can bring some hairy complexity, which is sidestepped (especially given the temporary-ness of the transitional state) with careful use of shared memory. And aren't systems like Postgres counterpoints to your claim? My memory isn't the greatest, but IIRC postgres's server-side connections are subprocesses which interact with the postmaster via shared memory, no?
  - fefe23 2y ago
    If you use shared memory with a captive process, that process can probably hack you if it gets taken over by an attacker. I agree with your parallelism counter-argument in principle. However even there it would probably make sense to not trust each other, to limit the blast radius of successful attacks. In your next point the "careful" illustrates exactly my point. Using shared memory for IPC is like using C or C++ and saying "well I'll be careful then". It can work but it will be very dangerous and most likely there will be security issues. You are much better off not doing it. Postgres is a beautiful argument in that respect. Yes you can write a database in C or C++ and have it use shared memory. It's just not recommended because you need professionals of the caliber of the Postgres people to pull it off. I understand many organizations think they have those. I don't think they actually do though.
MuffinFlavored 2y ago
Pretty cool. I see publish + subscribe example on GitHub but no request/response. Am I missing something?
- elBoberido 2y ago
  Request-response is on our todo list and will be introduced in an upcoming release :) What are you needing request-response for?

122 points by elfenpiff 2y ago

deleted 2y ago

[deleted]

npalli 2y ago

Congrats on the release. What's the difference between iceoryx and iceoryx2? I don't want to use Rust and want to stick to C++ if possible.

tbillington 2y ago
> Language bindings for C and C++ with CMake and Bazel support right out of the box. Python and other languages are coming soon.
elBoberido 2y ago
Besides being written in Rust, the big difference is the decentralized approach. With iceoryx1 a central daemon is required but with iceoryx2 this in not the case anymore. Furthermore, more fine grained control over the resources like memory and endpoints like publisher. Overall the architecture is more modular and it should be easier to port iceoryx2 to even more platforms and customize it with 3rd party extension. With this release we have initial support for C and C++. Not all features of the Rust version are supported yet, but the plan is to finish the bindings with the next release. Furthermore, with an upcoming release we will make it trivial to communicate between Rust, C and C++ applications and all the other language bindings we are going to provide, with Python being probably the next one.
- sebastos 2y ago
  I've been looking around for some kind of design documents that explain how you were able to ditch the central broker, but I haven't found much. Do you have breadcrumbs?
  - simfoo 2y ago
    Same here. Shared memory is one of those things where the kernel could really help some more with reliable cleanup (1). Until then you're mostly doomed to have a rock solid cleanup daemon or are limited to eventual cleanup by restarting processes. I have my doubts that it isn't possible to get into a situation where segments are being exhausted and you're forced to intervene (1) I'm referring to automatic refcounting of shm segments using posix shm (not sys v!) when the last process dies or unmaps
  - elfenpiff 2y ago
    This is a longer story, but I'll try to provide the essence. * All IPC resources are represented in the file system and have a global naming scheme. So if you would like to perform a service discovery, you take a look at the `/tmp/iceoryx2/services` list all service toml files that you are allowed to access and handle them. * Connecting to a service means, under the hood, opening a specific shared memory identified via a naming scheme, adding yourself to the participant list, and receiving/sending data. * Crashing/resource cleanup is done decentrally by every process that has the permissions to perform them. * In a central/broker architecture you would have the central broker that checks this in a loop. * In a decentralized architecture, we defined certain sync points where this is checked. These points are placed so that you check the misbehavior before it would affect you. For instance, when a sender shall send you a message every second but you do not receive it, you would actively check if it is still alive. Other sync points are, when an iceoryx2 node is created or you connect or disconnect to a service. The main point is that the API is decentralized but you can always use it in a central daemon if you like - but you don't have to. It is optional.

emmanueloga_ 2y ago

Looks great! From a quick glance it seems like it is a cross platform shared memory library. Maybe similar to this? [1]. Suggestion: would be cool to have a quick description of the system calls involved for each supported platform [2]. I'm guessing mmap on linux/osx and CreateFileMapping on Windows? -- 1: https://github.com/LiveAsynchronousVisualizedArchitecture/simdb https://github.com/LiveAsynchronousVisualizedArchitecture/si... 2: https://github.com/eclipse-iceoryx/iceoryx2?tab=readme-ov-file#supported-platforms https://github.com/eclipse-iceoryx/iceoryx2?tab=readme-ov-fi...

elfenpiff 2y ago
You guessed right. We have a layered architecture that abstracts this away for every platform. With this, we can support every OS as long as it has a way of sharing memory between processes (or tasks as some RTOSes are calling it) and you have a way of sending notifications.

westurner 2y ago

How does this compare to and/or integrate with OTOH Apache Arrow which had "arrow plasma IPC" and is supported by pandas with dtype_backend="pyarrow", lancedb/lancedb, and Serde.rs? https://serde.rs/#data-formats https://serde.rs/#data-formats

dumah 2y ago
https://github.com/apache/arrow/issues/34738 https://github.com/apache/arrow/issues/34738 https://lists.apache.org/thread/lk277x3b9gjol42sjg27bst2ggm5s0j2 https://lists.apache.org/thread/lk277x3b9gjol42sjg27bst2ggm5...
- zxexz 2y ago
  Not downvoting, but those two links don't really describe much - though they are part of the story. For other readers, here's a general takeaway - - Arrow Plasma was deprecated, and is now no longer even present in the Arrow project. - The maintainers behind the Plasma store in Arrow forked it, into Ray. Plasma is still alive and well in Ray - and it still uses the Arrow IPC format, among other things. In addition to the links from the above poster, read the original blog post on Plasma [0], and the section on Ray on [1]. I use Ray quite a bit. For more lightweight stuff, or for more low-level control, I use Arrow Flight [2] (and [3] for Python examples) [0] https://ray-project.github.io/2017/08/08/plasma-in-memory-object-store.html https://ray-project.github.io/2017/08/08/plasma-in-memory-ob... [1] https://arrow.apache.org/powered_by/ https://arrow.apache.org/powered_by/ [2] https://arrow.apache.org/docs/python/flight.html https://arrow.apache.org/docs/python/flight.html [3] https://arrow.apache.org/cookbook/py/flight.html https://arrow.apache.org/cookbook/py/flight.html
zxexz 2y ago
The other commenter answering you is, I think, trying to point out that the Arrow plasma store is deprecated (and no longer present in the arrow project). I think it's worth being a little more clear here - Arrow IPC is _not_ deprecated, and has massive momentum - so much so that it's more or less already become the default IPC format for many libraries. To me it remains unclear what the benefits of Iceoryx2 over the Arrow ecosystem is, and what the level of interoperability is, and what the tradeoffs of either are relative to eachother. Within a single machine, you can mmap the IPC file. You can use Arrow Flight for inter-node or inter-process communication. You can use Arrow with Ray, which is where Plasma went. I love anything new in this space though, if/when I have time I'll check this out - would love it if somebody could actually eloborate on the differences though.

hardwaresofton 2y ago

Been doing some IPC experiments recently following the 3tilley post[0], because there just isn't enough definitive information (even if it's a snapshot in time) out there. Shared memory is crazy fast, and I'm surprised that there aren't more things that take advantage of it. Super odd that gRPC doesn't do shared memory, and basically never plans to?[1]. All that said, the constructive criticism I can offer for this post is that in mass-consumption announcements like this one for your project, you should: - RPC throughput (with the usual caveats/disclaimers) - Comparison (ideally graphed) to an alternative approach (ex. domain sockets) - Your best/most concise & expressive usage snippet 100ns is great to know, but I would really like to know how much RPC/s this translates to without doing the math, or seeing it with realistic de-serialization on the other end. [0]: https://3tilley.github.io/posts/simple-ipc-ping-pong/ https://3tilley.github.io/posts/simple-ipc-ping-pong/ [1]: https://github.com/grpc/grpc/issues/19959 https://github.com/grpc/grpc/issues/19959

abhirag 2y ago
At $work we are evaluating different IPC strategies in Rust. My colleague expanded upon 3tilley's work, they have updated benchmarks with iceoryx2 included here[0]. I suppose the current release should perform even better. [0]: https://pranitha.rs/posts/rust-ipc-ping-pong/ https://pranitha.rs/posts/rust-ipc-ping-pong/
- nh2 2y ago
  Interesting that on Linux Unix Domain Sockets are not faster than TCP. People often say that the TCP stack overhead is high but this benchmark does not confirm that.
  - jcelerier 2y ago
    I'm curious about the benchmark. In my own for another network IPC library (https://GitHub.com/ossia/libossia https://GitHub.com/ossia/libossia) Unix sockets were consistently faster than the alternatives when sending the same payloads.
  - 2y ago

forrestthewoods 2y ago

Why is Windows target support tier 2 and not tier 1?

elfenpiff 2y ago
Tier 1 also means all security/safety features. Windows is not used in mission-critical systems like cars or plans, so we do not need to add those to Windows. We aim to support Windows so iceoryx2 can be used safely and securely in a desktop environment.

boolit2 2y ago

I'm in robotics education and we mostly work with Python, to make life easier for students. I'd love to push for more Rust, but so far there's no point to it. Multiprocess communication is something that we found lacking in Python (we want everything to be easily pip installable) and we ended up using shared memory primitives, which is a lot of code to maintain. What is the main roadblock for iceoryx2 Python bindings? Is it something you are looking for contributors for?

orecham 2y ago
The only real roadblock for this is time. We'd welcome any contributor, especially with such an impactful contribution as this. Making iceoryx2 an option for Python would be amazing. If you or someone you know wants to take it on, we would support as much as possible. So far for this topic, we have only done some brief research on the options available to us, such us going over the C API or using something like PyO3.

fefe23 2y ago

This smells like they are using shared memory, which is almost certainly a security nightmare. The way they are selling it makes me fear they aren't aware of what a time bomb they are sitting on. Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith), or you do hardcore capturing (make a copy of each message in the framework before handing it off). Their web page mentions zero copy, so it's probably not the second one. Also, benchmarks are misleading. It's easy to get good latency if your throughput is so high that you can do polling or spin locks, like for example in benchmarks. But that's probably not a good assumption for general usage because it will be very inefficient and waste power and require more cooling as well.

zbentley 2y ago
> Shared memory works as a transport if you either assume that all parties are trusted (in which case why do IPC in the first place? Just put them in a monolith) There are all sorts of domains where mutually-trusted parties need IPC. Off the top of my head and in no particular order: - Applications that pass validated data to/from captive subprocesses. Not everything is available as a natively-linked library. Not every language's natively-linked libraries are as convenient to reliably install as external binaries. - Parallelism/server systems farming work out to forked (but not exec'd) subprocesses. Not everything needs setuid. Somtimes you just want to parallelize number crunching without the headache of threads (or are on a platform like Python which limits threads' usefulness). - Replatforming/language transitions in data-intensive applications. Running the new runtime/platform in the same address space as the legacy platform can bring some hairy complexity, which is sidestepped (especially given the temporary-ness of the transitional state) with careful use of shared memory. And aren't systems like Postgres counterpoints to your claim? My memory isn't the greatest, but IIRC postgres's server-side connections are subprocesses which interact with the postmaster via shared memory, no?
- fefe23 2y ago
  If you use shared memory with a captive process, that process can probably hack you if it gets taken over by an attacker. I agree with your parallelism counter-argument in principle. However even there it would probably make sense to not trust each other, to limit the blast radius of successful attacks. In your next point the "careful" illustrates exactly my point. Using shared memory for IPC is like using C or C++ and saying "well I'll be careful then". It can work but it will be very dangerous and most likely there will be security issues. You are much better off not doing it. Postgres is a beautiful argument in that respect. Yes you can write a database in C or C++ and have it use shared memory. It's just not recommended because you need professionals of the caliber of the Postgres people to pull it off. I understand many organizations think they have those. I don't think they actually do though.

MuffinFlavored 2y ago

Pretty cool. I see publish + subscribe example on GitHub but no request/response. Am I missing something?

elBoberido 2y ago
Request-response is on our todo list and will be introduced in an upcoming release :) What are you needing request-response for?