5 ms·

You seem to want pipes to bend over backwards to solve each and every one of your applications' problems. > The don't solve race conditions in peers trying to

by jude- 12y ago

You seem to want pipes to bend over backwards to solve each and every one of your applications' problems. > The don't solve race conditions in peers trying to locate each other (surprisingly difficult). Not even sure what you mean here. Are you talking about peer discovery? Because DBus won't help you there either--peers have to be aware of each other's DBus object paths before they can rendezvous. Similarly, two peers need to know where the common pipe is to rendezvous. > They don't solve a standardized marshaling format. Nor should they. There are a ton of ways to skin this cat in userspace, depending on what your application needs. Protobufs come to mind, for example, but there are others. Why do you want the pipe to enforce a particular marshaling format? Does the pipe know what's best for every single application that will ever use it? > They don't come with an implementation to integrate with main loops for event polling. It's not the kernel's responsibility to implement the application's main loop. That's what libevent and friends are for today, if you need them. > They have an inherent vulnerability in FD passing where you can cause the peer to lock up. Last I checked, you pass file descriptors via UNIX sockets, not pipes. > They don't handle authentication (well, sort of). Depends on your application's threat model. The kernel provides some basic primitives that can be used to address common security-related problems (capabilities, permission bits, users, groups, and ACLs). If they're not enough, you're free to perform whatever authentication you need in userspace to secure your application against your threat model's adversaries. It is unreasonable to expect the pipe to be aware of every single threat model an applications expects, especially since they change over time. > You can get into deadlock situations in your messaging code if you aren't really careful about message sizes and when you order poll in/poll out. It's not the pipe's fault if you don't use it correctly. > They aren't introspect-able to see what the peer supports. Peer A could use the pipe to ask peer B what it can do for peer A. Why do you want the pipe to do peer B's job? > They make it super easy to not maintain ABI. Nor does DBus. Nothing stops an application from willy-nilly changing the data it serves back.

cbsmith 12y ago
> Not even sure what you mean here. Are you talking about peer discovery? Because DBus won't help you there either--peers have to be aware of each other's DBus object paths before they can rendezvous. Similarly, two peers need to know where the common pipe is to rendezvous. I suspect he was referring to socket activation and how that simplifies these kinds of messes. > Nor should they. There are a ton of ways to skin this cat in userspace, depending on what your application needs. Protobufs come to mind, for example, but there are others. Right... so that's exactly what systemd did. It used Dbus, which provides that standard serialization format. Not my favourite format, but very well established and tested and focused on systemd's problem domain. The point is, in order to have loose coupling between components, something like unix pipes is just a starting point. > Does the pipe know what's best for every single application that will ever use it? Ah, now I understand the problem with systemd. I never realized it was trying to take over every application's communications protocol! ; -) Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of... > Last I checked, you pass file descriptors via UNIX sockets, not pipes. Correct. People have a tendency to mess up their semantics though. If the original poster wasn't referring to unix domain sockets, than it is an even sillier question. > It is unreasonable to expect the pipe to be aware of every single threat model an applications expects, especially since they change over time. Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components. > Nor does DBus. Nothing stops an application from willy-nilly changing the data it serves back. ? D-Bus will drop you like a hot potato the moment you fire off invalid messages. You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up.
- jude- 12y ago
  > The point is, in order to have loose coupling between components, something like unix pipes is just a starting point. It's also an ending point. If each application gets to define its own IPC primitives, then there become as many app-to-app communication protocols as there are app-to-app pairs. This does not make for a loosely-coupled ecosystem. > Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of... This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol. > Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components. Um, data within a pipe is visible to only the endpoint processes, the root user (via procfs and /dev/kmem), and the kernel. If you don't trust an endpoint, you should stop communicating with it. If you don't trust the root user or the kernel, you can't really do anything securely at all in the first place. My point is, data within a pipe is about as secure as it's going to get. > You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up. I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it. DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using.
  - cbsmith 12y ago
    > t's also an ending point. If each application gets to define its own IPC primitives, then there become as many app-to-app communication protocols as there are app-to-app pairs. This does not make for a loosely-coupled ecosystem. That's exactly why you need a more systemic approach to the IPC mechanism... You can pretend that "oh this is just a stream so there isn't tight coupling", but the information that is communicated is the same. If you haven't imposed some structure and consistency to it, that's exactly how you end up with a ball of mud. > This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol. You misunderstood my point. I'm not justifying it on the basis that systemd is using it. I'm saying the fact that all the other systems have arrived at a similar, and in many cases the exact same mechanism, is pretty strong evidence that it is a reasonable design choice. Basically all the Linux systems out there are already using udev & dbus. Most of the non-Linux systems do as well. Everyone's done it and made it work. That systemd is adopting arguably the most entrenched one in the Linux sphere is hardly as controversial as people seem to think it is. > My point is, data within a pipe is about as secure as it's going to get. I wasn't trying to suggest it wasn't a secure point-to-point communication mechanism (it has issues, but fine enough). The issue is that you need more integration in with the security model to avoid having a rats nest of security logic on top of it. > I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it. There's a very long and glorious history of malformed and misleading IPC causing problems. Not that it is the only thing, but life becomes a lot easier when that problem is off the table. > DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using. Yes you will. However, DBus ensures that you don't have to write a ton of redundant code just verifying you are getting validly structured data and dealing with the nasty ways someone might try to exploit that. Imagine having to write a secure REST service where the only thing you had to worry about in the entire network protocol stack was the validity of the data expressed in the payload.

by jude- 12y ago

cbsmith 12y ago

> Not even sure what you mean here. Are you talking about peer discovery? Because DBus won't help you there either--peers have to be aware of each other's DBus object paths before they can rendezvous. Similarly, two peers need to know where the common pipe is to rendezvous. I suspect he was referring to socket activation and how that simplifies these kinds of messes. > Nor should they. There are a ton of ways to skin this cat in userspace, depending on what your application needs. Protobufs come to mind, for example, but there are others. Right... so that's exactly what systemd did. It used Dbus, which provides that standard serialization format. Not my favourite format, but very well established and tested and focused on systemd's problem domain. The point is, in order to have loose coupling between components, something like unix pipes is just a starting point. > Does the pipe know what's best for every single application that will ever use it? Ah, now I understand the problem with systemd. I never realized it was trying to take over every application's communications protocol! ; -) Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of... > Last I checked, you pass file descriptors via UNIX sockets, not pipes. Correct. People have a tendency to mess up their semantics though. If the original poster wasn't referring to unix domain sockets, than it is an even sillier question. > It is unreasonable to expect the pipe to be aware of every single threat model an applications expects, especially since they change over time. Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components. > Nor does DBus. Nothing stops an application from willy-nilly changing the data it serves back. ? D-Bus will drop you like a hot potato the moment you fire off invalid messages. You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up.

jude- 12y ago
> The point is, in order to have loose coupling between components, something like unix pipes is just a starting point. It's also an ending point. If each application gets to define its own IPC primitives, then there become as many app-to-app communication protocols as there are app-to-app pairs. This does not make for a loosely-coupled ecosystem. > Seriously, I think it is perfectly reasonable (and necessary) to define a standard protocol for system even notifications... I say this because it has already been done... by the standard compoents like udev & dbus that systemd is building on top of... This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol. > Yes, but you do need something more sophisticated than a pipe to manage secure communications between your systems components. Um, data within a pipe is visible to only the endpoint processes, the root user (via procfs and /dev/kmem), and the kernel. If you don't trust an endpoint, you should stop communicating with it. If you don't trust the root user or the kernel, you can't really do anything securely at all in the first place. My point is, data within a pipe is about as secure as it's going to get. > You could send valid messages with fraudlent/misleading data payloads I guess, but at least a whole host of problems are addressed by tightening that up. I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it. DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using.
- cbsmith 12y ago
  > t's also an ending point. If each application gets to define its own IPC primitives, then there become as many app-to-app communication protocols as there are app-to-app pairs. This does not make for a loosely-coupled ecosystem. That's exactly why you need a more systemic approach to the IPC mechanism... You can pretend that "oh this is just a stream so there isn't tight coupling", but the information that is communicated is the same. If you haven't imposed some structure and consistency to it, that's exactly how you end up with a ball of mud. > This is circular reasoning. You're saying "we should use systemd's notification protocol, because systemd uses it." This says nothing about the technical merits of its protocol. You misunderstood my point. I'm not justifying it on the basis that systemd is using it. I'm saying the fact that all the other systems have arrived at a similar, and in many cases the exact same mechanism, is pretty strong evidence that it is a reasonable design choice. Basically all the Linux systems out there are already using udev & dbus. Most of the non-Linux systems do as well. Everyone's done it and made it work. That systemd is adopting arguably the most entrenched one in the Linux sphere is hardly as controversial as people seem to think it is. > My point is, data within a pipe is about as secure as it's going to get. I wasn't trying to suggest it wasn't a secure point-to-point communication mechanism (it has issues, but fine enough). The issue is that you need more integration in with the security model to avoid having a rats nest of security logic on top of it. > I think you will find that the bulk of IPC problems will come from dealing with data you didn't expect--that is, processes sending "fraudlent/misleading" data and other processes acting on it. There's a very long and glorious history of malformed and misleading IPC causing problems. Not that it is the only thing, but life becomes a lot easier when that problem is off the table. > DBus won't help you there--you always always ALWAYS have to validate data you receive from untrusted parties, no matter what transport or wire format you're using. Yes you will. However, DBus ensures that you don't have to write a ton of redundant code just verifying you are getting validly structured data and dealing with the nasty ways someone might try to exploit that. Imagine having to write a secure REST service where the only thing you had to worry about in the entire network protocol stack was the validity of the data expressed in the payload.