3 ms·

Two answers: 1) Modern hardware is getting much better at multiplexing resources (aka "making the things play nice with each other"). See for example sr-iov.

by shykes 10y ago

Two answers: 1) Modern hardware is getting much better at multiplexing resources (aka "making the things play nice with each other"). See for example sr-iov. 2) More and more applications are distributed across many machines, with the resources of each machine entirely consumed by that one application. In that case there is only one "thing" running so the problem of multiplexing goes away. This is the premise of technologies like unikernel.

signa11 10y ago
> 1) Modern hardware is getting much better at multiplexing resources (aka "making the things play nice with each other"). See for example sr-iov. sorry, but with sr-iov, you still have one single 'control' application muxing resources for applications sitting above it. for example, with dpdk, you more or less take over the complete network card. the 'control' might be even human in some cases :) which/who carefully lays out the resource mapping. another example, intel's cmt/cat techniques (https://github.com/01org/intel-cmt-cat/wiki https://github.com/01org/intel-cmt-cat/wiki) map nic's rx/tx rings to processors l3 cache etc. this is ofcourse assuming that userland applications have complete control over pci-e lanes etc. now, if you have two such control applications e.g. one doing packet-io and the other doing disk-io, how do you ensure that these control applications don't starve each other out ? in canonical settings, kernel would be democratizing (is that even a word ?) access to the underlying h/w. but since that is bypassed, we are in a strange new world...
- benlwalker 10y ago
  Sharing CPU resources between user space drivers is certainly a challenge. The best way to view this is that the kernel provides a general purpose solution for sharing resources with some associated overhead. Tools like DPDK and SPDK let you opt out of that, but now you are responsible for intelligently sharing the hardware. You, as the application developer, have a distinct advantage though - you only need to solve the problem for your application, and using that knowledge can often lead to more efficient solutions. This may mean dedicating cores to the network or disk, or it may mean working in fixed sized batches, etc.
  - signa11 10y ago
    > ... you only need to solve the problem for your application, and using that knowledge can often lead to more efficient solutions... this ! _exactly_ this :) imho, the fundamental re-architecture of I/O subsystem for x86 machines has kind of relegated this playing field now to mostly solving _only_ s/w problem, rather than a combination of h/w and s/w. for example, earlier if you wanted to write a very high performance node in, say the epc-core e.g. SGW/PGW/MME etc. you would assemble a bunch of folks with very diverse set of expertise. right from h/w i/o subsystem designers who could do npu's, switch-fabrics etc. to driver dudes, to 'infrastructure' folks to application programmers etc. etc. in the current incarnation, a vanilla off the shelf x86 machine is more than sufficient. and if your s/w architecture is _right_, you can scale quite easily.

by shykes 10y ago

signa11 10y ago

> 1) Modern hardware is getting much better at multiplexing resources (aka "making the things play nice with each other"). See for example sr-iov. sorry, but with sr-iov, you still have one single 'control' application muxing resources for applications sitting above it. for example, with dpdk, you more or less take over the complete network card. the 'control' might be even human in some cases :) which/who carefully lays out the resource mapping. another example, intel's cmt/cat techniques (https://github.com/01org/intel-cmt-cat/wiki https://github.com/01org/intel-cmt-cat/wiki) map nic's rx/tx rings to processors l3 cache etc. this is ofcourse assuming that userland applications have complete control over pci-e lanes etc. now, if you have two such control applications e.g. one doing packet-io and the other doing disk-io, how do you ensure that these control applications don't starve each other out ? in canonical settings, kernel would be democratizing (is that even a word ?) access to the underlying h/w. but since that is bypassed, we are in a strange new world...

benlwalker 10y ago
Sharing CPU resources between user space drivers is certainly a challenge. The best way to view this is that the kernel provides a general purpose solution for sharing resources with some associated overhead. Tools like DPDK and SPDK let you opt out of that, but now you are responsible for intelligently sharing the hardware. You, as the application developer, have a distinct advantage though - you only need to solve the problem for your application, and using that knowledge can often lead to more efficient solutions. This may mean dedicating cores to the network or disk, or it may mean working in fixed sized batches, etc.
- signa11 10y ago
  > ... you only need to solve the problem for your application, and using that knowledge can often lead to more efficient solutions... this ! _exactly_ this :) imho, the fundamental re-architecture of I/O subsystem for x86 machines has kind of relegated this playing field now to mostly solving _only_ s/w problem, rather than a combination of h/w and s/w. for example, earlier if you wanted to write a very high performance node in, say the epc-core e.g. SGW/PGW/MME etc. you would assemble a bunch of folks with very diverse set of expertise. right from h/w i/o subsystem designers who could do npu's, switch-fabrics etc. to driver dudes, to 'infrastructure' folks to application programmers etc. etc. in the current incarnation, a vanilla off the shelf x86 machine is more than sufficient. and if your s/w architecture is _right_, you can scale quite easily.