4 ms·
I disagree they're so cleanly split, that io-uring is so different in that particular split, or that it matters so much. 1. "Control plane" is how an applicati
by test_epsilon 5y ago
I disagree they're so cleanly split, that io-uring is so different in that particular split, or that it matters so much.
1. "Control plane" is how an application specifies what action they want, and "data plane" is (more or less) the result. A read(2) system call is perfectly cleanly separated in those concerns (input arguments and return value for control, returned data for data). Can't get cleaner than that.
2. Performance critical. Control is highly, highly performance critical. Some applications can generate a large amount of independent operations or very large chunks, but latency is very often the performance limiter. And that tends to be increasingly true as parallelism increases, bandwidth increases, but latencies tendto improve at a much lower rate and sometimes stand still or go backwards (whether you're looking at DRAM, NAND, disk, network, or communication across cores in a single node).
3. On the data side of performance. Well it's really fuzzy because control is data, and data often controls control, you have metadata etc. But quite often data is easier to deal with, if it's parallelizable, prefetchable, predicatble, linear. In CPUs for example, i$ misses are often a worse problem to have than d$ because running out of instructions means the whole pipeline empties out and shuts down, but stores can be buffered and a stalled load often still leaves independent useful work to do. It's common that CPUs will prefer instruction lines in their unified cache levels for this reason.
4. Data is absolutely safety critical. I'm not sure why it couldn't be if control is. Sure you could say you have redundancy in your data, but you can also have checks in your data to help ensure the control was correct (for simple example a block of data can store its own location as well, so if control logic goes wrong and reads the wrong block, it could try to recover). That said data tends to be easier to recover from than arbitrary complex logic, but that doesn't mean the data is less critical.
5. Where does metadata sit in here? In io_uring, you could have ops that open files, read directories, etc. This is no more a clean split than traditional unix APIs IMO.
io-uring is nice because of its submission and completion model minimizes overhead and it allows asynchronous and parallel and out of order operations. No surprise, it's modeled after high performance IO device command and completion queues.
- jstimpfle 5y agoMaybe one could say that ioring's asynchronous completion model is a better split because there's no control dependency (blocking syscalls, context switches)?
- test_epsilon 5y agoLinux has non blocking / asynchronous IO APIs for many years already, so as a _general_ statement that is not one of the new interesting things of io uring. But that is something it does well when you get into specifics (both in improving overheads of currently supported operations and expanding the types of operations that can be submitted this way).
- jorangreef 5y ago> Performance critical. Control is highly, highly performance critical. I am using the term "control plane" in the typical systems sense, where the control plane is not the performance critical data pipeline. The control plane by definition is outside the critical request path. Relative to the performance profile of the data plane, the control plane is not performance critical. The control plane is usually several orders of magnitude less demanding in terms of resources (CPU, memory, network, storage) than the data plane it controls. One very obvious example of this is something like ZooKeeper, where a tiny metadata system can be responsible for switching gigantic data plane clusters. Another basic example, the control plane handles "configuration" such as the routing table in a routing protocol, and changes to this configuration synced out of band, whereas the data plane does the rapid switching based on this table, or the control plane decides whether to switch the data plane on/off. These decisions are relatively cheap but can have serious consequences. Finally, a good example is also Amazon, where their distributed systems are famous for having control planes that always do constant work regardless of the data plane to avoid bimodal behavior. If your system has the control plane and the data plane showing similar performance profiles then these concepts may have been conflated or not exploited to their potential. > Data is absolutely safety critical. Again, I think you're missing the safety that a clear definition of a control plane gives to the data pipeline it manages. As long as you treat both planes as "effectively the same" or of no consequence, you won't see how to exploit these different concepts to achieve both performance AND safety.
- test_epsilon 5y ago> I am using the term "control plane" in the typical systems sense, where the control plane is not the performance critical data pipeline. The control plane by definition is outside the critical request path. Relative to the performance profile of the data plane, the control plane is not performance critical. That doesn't make sense because you're talking about io-uring itself having separation between control and data, however it is is exclusively involved with the request path. It seems like you made this confusion with your first post, I'm just replying to what you wrote.