4 ms·

Is this being considered for merging upstream? What's the tech behind it, what makes it faster?

ericfrederich 12y ago
The tech behind it is backdoors. This seems to be developed in China afterall. You thought the OpenSSL and Bash bugs were bad... wait till we have one at the socket level.
- MoOmer 12y ago
  Have you identified a vulnerability, or is that pure conjecture?
  - ericfrederich 12y ago
    the latter
corbet 12y ago
The developers have not posted it to the relevant mailing lists or asked that it be merged, so, no, it is not being considered.
- harry8 12y ago
  Maybe it's up to the more sensible in the kernel "community", to reach out to the developers of code known to be interesting to discuss what's in it for them to do the work required to get it merged, the probability of doing a ton of work, and then being ignored etc etc. There's a sense in the above of "They haven't submitted us so we don't care." It might not be the best way to make the kernel as good as it can be, if that is the goal of anyone active in the kernel "community." (And maybe it is). I have a lot of sympathy for someone publishing their code and their results and then saying "I won't play stupid kernel politics, your move." I don't know if that's what is happening here or it's cultural differences or something I haven't thought of. Nor do I know if this particular development is worthwhile merging, but hey, neither does the kernel "community" right?
minimax 12y ago
It looks like there are like 3 separate optimizations, but I think the most important one is the "enable_listen_spawn" feature. Here is how they describe it: Fastsocket creates one local listen socket table for each CPU core. With this feature, application process can decide to process new connections from a specific CPU core. It is done by copying the original listen socket and inserting the copy into the local listen socket table. When there is a new connection on one CPU core, kernel tries to match a listen socket in the local listen table of that CPU core and inserts the connection into the accept queue of the local listen socket if there is match. Later, the process can accept the connection from the local listen socket exclusively. This way each network softirq has its own local socket to queue new connection and each process has its own local listen socket to pull new connection out. When the process is bound with the CPU core specified, then connections delivered to that CPU core by NIC are entirely processed by the same CPU core with in all stages, including hardirp, softirq, syscall and user process. As a result, connections are processed without contension across all CPU cores, which achieves passive connection locality. The kernel in its normal configuration will try to spread the IRQs evenly across CPUs. So for their use case, where you have one worker thread per CPU handling zillions of short lived TCP connections, they can eliminate a bunch of locking and cache thrashing that would otherwise happen when handling new connections and dispatching the related epoll events within the kernel.

The tech behind it is backdoors. This seems to be developed in China afterall. You thought the OpenSSL and Bash bugs were bad... wait till we have one at the socket level.

MoOmer 12y ago
Have you identified a vulnerability, or is that pure conjecture?
- ericfrederich 12y ago
  the latter

corbet 12y ago

The developers have not posted it to the relevant mailing lists or asked that it be merged, so, no, it is not being considered.

harry8 12y ago
Maybe it's up to the more sensible in the kernel "community", to reach out to the developers of code known to be interesting to discuss what's in it for them to do the work required to get it merged, the probability of doing a ton of work, and then being ignored etc etc. There's a sense in the above of "They haven't submitted us so we don't care." It might not be the best way to make the kernel as good as it can be, if that is the goal of anyone active in the kernel "community." (And maybe it is). I have a lot of sympathy for someone publishing their code and their results and then saying "I won't play stupid kernel politics, your move." I don't know if that's what is happening here or it's cultural differences or something I haven't thought of. Nor do I know if this particular development is worthwhile merging, but hey, neither does the kernel "community" right?

minimax 12y ago

It looks like there are like 3 separate optimizations, but I think the most important one is the "enable_listen_spawn" feature. Here is how they describe it: Fastsocket creates one local listen socket table for each CPU core. With this feature, application process can decide to process new connections from a specific CPU core. It is done by copying the original listen socket and inserting the copy into the local listen socket table. When there is a new connection on one CPU core, kernel tries to match a listen socket in the local listen table of that CPU core and inserts the connection into the accept queue of the local listen socket if there is match. Later, the process can accept the connection from the local listen socket exclusively. This way each network softirq has its own local socket to queue new connection and each process has its own local listen socket to pull new connection out. When the process is bound with the CPU core specified, then connections delivered to that CPU core by NIC are entirely processed by the same CPU core with in all stages, including hardirp, softirq, syscall and user process. As a result, connections are processed without contension across all CPU cores, which achieves passive connection locality. The kernel in its normal configuration will try to spread the IRQs evenly across CPUs. So for their use case, where you have one worker thread per CPU handling zillions of short lived TCP connections, they can eliminate a bunch of locking and cache thrashing that would otherwise happen when handling new connections and dispatching the related epoll events within the kernel.