5 ms·

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People mana

by lmilcin 12y ago

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zlib), parses it, analyzes it, sends to multiple algorithms which decide if current market situation matches certain rules, decides and fills market order, the order gets inspected by independent mechanism to stop the algorithm if it malfunctions, and only then it gets sent to market, over TCP, which is another form of IPC. All in the span of fricking 5us which is 40 times less than the benchmark suggests for this simple task. Granted, the algorithmic trading world goes to great lengths to avoid overhead including kernel overhead any kind of task switiching, branch prediction fails, etc. But still, come on, guys...

the_mitsuhiko 12y ago
> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.
- huhtenberg 12y ago
  Nothing obvious about it. If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. You certainly don't mean to imply that both are the only way to implement a "security policy driven message bus" software, do you?
  - the_mitsuhiko 12y ago
    > If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. Far from it. One of the core designs of kbus (which dbus cannot do because it's not in the kernel) is that you can seal the payload buffer from the sender so the receiver and use it safely concurrently without having to clone anything. There is obviously a lot of general inefficiency in the userland libraries but not in the design of it.
    - vezzy-fnord 12y ago
      You're talking about memfd, I think? That has nothing to do with kdbus in particular. It's an independent syscall that replaces many use cases for splice/vmsplice, even though it was introduced as part of the kdbus project - it's still a separate thing.
      - the_mitsuhiko 12y ago
        > You're talking about memfd, I think? That has nothing to do with kdbus in particular. It's an independent syscall that replaces many use cases for splice/vmsplice, even though it was introduced as part of the kdbus project - it's still a separate thing. That was created for KDBUS but has more use than than being used for KDBUS exclusively. It still was written for KDBUS.
        vezzy-fnord 12y ago
        And so it was accepted and merged. That something good came out of kdbus is excellent, but that's no justification to merge in the whole package. A ton of proposed kernel additions end up like this - a few good ideas refined and accepted, and the rest thrown out.
  - deleted 12y ago
    [deleted]
  - hp 12y ago
    the trace isn't dbus itself, it's a particular client program using the gdbus binding. The gdbus binding uses a lot of malloc and threads, and this particular client program is all blocking round trips.
- deleted 12y ago
  [deleted]
- coldtea 12y ago
  >You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive. The real problem is he doesn't give any data for volume/bandwidth processed in his case. I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc).
  - the_mitsuhiko 12y ago
    > I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc). It was in comparison with an HTTP server. I did not read much about the mail thread but from what I can tell it essentially benchmarks the uninteresting part of DBUS which is the part that runs in userspace. That's old and definitely needs improvements, but not the design of it. In fact, the design of KDBUS or the DBUS dispatching core is from what I can tell quite intelligent and makes sense from a performance point of view. Without a doubt there are a lot of old bells and whistles attached to DBUS which need rewriting.
  - deleted 12y ago
    [deleted]
latitude 12y ago
> But still, come on, guys... libnetsnmp used to make close to a million calloc() calls when responding to a query for a number of CPU sockets on the machine. That was kind of amazing in its sheer brazenness.
thuffy 12y ago
Moores Law works to make it eventually more than fast enough. Moores Law does not help it get more secure. How about we work on things that mater in this post-Snowden world, like increasing security -- and let Moore do the performance improvements! Seems a more reasonable and efficient approach, no? Adding more attackable surface area to the Kernel is not a good idea at all. It is an NSA wet dream.
- vezzy-fnord 12y ago
  Moore's law makes it faster, but Wirth's law compensates for that dearly. Let's also not forget that pesky Amdahl making our bottlenecks stay bottlenecks, as some things are just fundamentally wrong for performance.
  - thuffy 12y ago
    Kinda good points; however: Wirth's law is more true for commercial software where profit is the motive. Amdahl's law only really makes sense in parallel computing. All that is beside the real point; which is that in this post-Snowden world, we should not be sacrificing security for performance.
    - vezzy-fnord 12y ago
      Wirth's law is more true for commercial software where profit is the motive. How I wish. Sure, for proprietary and commercial consumer-facing apps, it might be the case. There's a ton of proprietary/commercial development tools and infrastructure (hyper-optimized JVMs, k/kdb...) that are inverses of Wirth. FOSS has its opuses as well, but it's just as full of crap. Amdahl's law only really makes sense in parallel computing. Sure, but that doesn't mean one can just magic away blatant bottlenecks, and even if some speedup is possible, it does not retroactively make these decisions correct.
    - random28345 12y ago
      > Amdahl's law only really makes sense in parallel computing. Contemporary computing is parallel computing. Even phones have multiple cores.

by lmilcin 12y ago

the_mitsuhiko 12y ago

> It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that. You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

huhtenberg 12y ago
Nothing obvious about it. If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. You certainly don't mean to imply that both are the only way to implement a "security policy driven message bus" software, do you?
- the_mitsuhiko 12y ago
  > If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. Far from it. One of the core designs of kbus (which dbus cannot do because it's not in the kernel) is that you can seal the payload buffer from the sender so the receiver and use it safely concurrently without having to clone anything. There is obviously a lot of general inefficiency in the userland libraries but not in the design of it.
  - vezzy-fnord 12y ago
    You're talking about memfd, I think? That has nothing to do with kdbus in particular. It's an independent syscall that replaces many use cases for splice/vmsplice, even though it was introduced as part of the kdbus project - it's still a separate thing.
    - the_mitsuhiko 12y ago
      > You're talking about memfd, I think? That has nothing to do with kdbus in particular. It's an independent syscall that replaces many use cases for splice/vmsplice, even though it was introduced as part of the kdbus project - it's still a separate thing. That was created for KDBUS but has more use than than being used for KDBUS exclusively. It still was written for KDBUS.
      - vezzy-fnord 12y ago
        And so it was accepted and merged. That something good came out of kdbus is excellent, but that's no justification to merge in the whole package. A ton of proposed kernel additions end up like this - a few good ideas refined and accepted, and the rest thrown out.
- deleted 12y ago
  [deleted]
- hp 12y ago
  the trace isn't dbus itself, it's a particular client program using the gdbus binding. The gdbus binding uses a lot of malloc and threads, and this particular client program is all blocking round trips.
deleted 12y ago
[deleted]
coldtea 12y ago
>You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive. The real problem is he doesn't give any data for volume/bandwidth processed in his case. I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc).
- the_mitsuhiko 12y ago
  > I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc). It was in comparison with an HTTP server. I did not read much about the mail thread but from what I can tell it essentially benchmarks the uninteresting part of DBUS which is the part that runs in userspace. That's old and definitely needs improvements, but not the design of it. In fact, the design of KDBUS or the DBUS dispatching core is from what I can tell quite intelligent and makes sense from a performance point of view. Without a doubt there are a lot of old bells and whistles attached to DBUS which need rewriting.
- deleted 12y ago
  [deleted]

latitude 12y ago

> But still, come on, guys... libnetsnmp used to make close to a million calloc() calls when responding to a query for a number of CPU sockets on the machine. That was kind of amazing in its sheer brazenness.

thuffy 12y ago

Moores Law works to make it eventually more than fast enough. Moores Law does not help it get more secure. How about we work on things that mater in this post-Snowden world, like increasing security -- and let Moore do the performance improvements! Seems a more reasonable and efficient approach, no? Adding more attackable surface area to the Kernel is not a good idea at all. It is an NSA wet dream.

vezzy-fnord 12y ago
Moore's law makes it faster, but Wirth's law compensates for that dearly. Let's also not forget that pesky Amdahl making our bottlenecks stay bottlenecks, as some things are just fundamentally wrong for performance.
- thuffy 12y ago
  Kinda good points; however: Wirth's law is more true for commercial software where profit is the motive. Amdahl's law only really makes sense in parallel computing. All that is beside the real point; which is that in this post-Snowden world, we should not be sacrificing security for performance.
  - vezzy-fnord 12y ago
    Wirth's law is more true for commercial software where profit is the motive. How I wish. Sure, for proprietary and commercial consumer-facing apps, it might be the case. There's a ton of proprietary/commercial development tools and infrastructure (hyper-optimized JVMs, k/kdb...) that are inverses of Wirth. FOSS has its opuses as well, but it's just as full of crap. Amdahl's law only really makes sense in parallel computing. Sure, but that doesn't mean one can just magic away blatant bottlenecks, and even if some speedup is possible, it does not retroactively make these decisions correct.
  - random28345 12y ago
    > Amdahl's law only really makes sense in parallel computing. Contemporary computing is parallel computing. Even phones have multiple cores.