3 ms·

The biggest thing I'm waiting for with FreeBSD is the TLS sendfile work that Netflix has done. They have said a few times they are open to open sourcing it, but

The biggest thing I'm waiting for with FreeBSD is the TLS sendfile work that Netflix has done. They have said a few times they are open to open sourcing it, but the code needs to be cleaned up. Also I'm interested in whats been going on with concurrencykit[1]? I saw it got imported into the kernel a couple years ago[2]. What's the progress on incorporating concurrencykit? [1]: http://concurrencykit.org/ http://concurrencykit.org/ [2]: https://www.freshbsd.org/commit/freebsd/src/309260 https://www.freshbsd.org/commit/freebsd/src/309260

loeg 8y ago
Netflix's TLS sendfile probably won't ever be imported. It isn't full TLS, just single-key output. It works for Netflix because they control the client software and if the client requests a re-key or other session change, they can just drop the connection and let the client reconnect. It isn't really obvious how it would be cleaned up to be general enough to import, and it would be a ton of work with little payoff. ConcurrencyKit is imported and used for efficient read-mostly datastructures in FreeBSD 12, mostly in networking applications such as IP routing and link-level address caching (i.e., ARP in IPv4 Ethernet). It's also used in network interface drivers, HWPMC, TCP, and the linuxkpi emulation layer (to emulate RCU). You might be interested in this blog post: http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-liveness-guarantees http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-livene... and https://www.freebsd.org/releases/12.0R/relnotes.html https://www.freebsd.org/releases/12.0R/relnotes.html .
- deleted 8y ago
  [deleted]
- toast0 8y ago
  I don't think it would be desirable for the kernel to handle rekeying, the socket should get sent back to userspace for that. That would probably require some wrangling, and it's certainly easier to do it in a good enough fashion rather than try to clean it up and get it upstreamed, but it's not impossible. I'm certainly guilty of not getting enough energy around upstreaming patches too, though.
  - loeg 8y ago
    > I don't think it would be desirable for the kernel to handle rekeying, the socket should get sent back to userspace for that. That would probably require some wrangling, and it's certainly easier to do it in a good enough fashion rather than try to clean it up and get it upstreamed, but it's not impossible. Sure; I agree. I tried to address that under "It isn't really obvious how it would be cleaned up to be general enough to import, and it would be a ton of work with little payoff." After all, you have to integrate with OpenSSL (or whatever) on the userspace side somehow too. And OpenSSL is contrib code, so we can't just hack it willy-nilly.
    - caf 8y ago
      OpenSSL already has integration for Linux's similar kTLS support (bulk transmit encryption only; all other alerts, renegotiation etc punted back to userspace). If you wrote it to provide the same API then the OpenSSL support side would be easy. https://github.com/openssl/openssl/blob/d6c3c1896cf3c0d69bc27da923d63f8130b13ca0/include/internal/ktls.h https://github.com/openssl/openssl/blob/d6c3c1896cf3c0d69bc2...
      - loeg 8y ago
        Thanks, I didn't know about that. That makes it seem a lot more viable.
GautamGoel 8y ago
I'm confused: wasn't it already mainlined? https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-Faster-Sendfile https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-...
- 2trill2spill 8y ago
  That's async sendfile, it doesn't do TLS.

loeg 8y ago

Netflix's TLS sendfile probably won't ever be imported. It isn't full TLS, just single-key output. It works for Netflix because they control the client software and if the client requests a re-key or other session change, they can just drop the connection and let the client reconnect. It isn't really obvious how it would be cleaned up to be general enough to import, and it would be a ton of work with little payoff. ConcurrencyKit is imported and used for efficient read-mostly datastructures in FreeBSD 12, mostly in networking applications such as IP routing and link-level address caching (i.e., ARP in IPv4 Ethernet). It's also used in network interface drivers, HWPMC, TCP, and the linuxkpi emulation layer (to emulate RCU). You might be interested in this blog post: http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-liveness-guarantees http://scalebsd.org/blog/2018/06/16/UDP-and-epoch-for-livene... and https://www.freebsd.org/releases/12.0R/relnotes.html https://www.freebsd.org/releases/12.0R/relnotes.html .

deleted 8y ago
[deleted]
toast0 8y ago
I don't think it would be desirable for the kernel to handle rekeying, the socket should get sent back to userspace for that. That would probably require some wrangling, and it's certainly easier to do it in a good enough fashion rather than try to clean it up and get it upstreamed, but it's not impossible. I'm certainly guilty of not getting enough energy around upstreaming patches too, though.
- loeg 8y ago
  > I don't think it would be desirable for the kernel to handle rekeying, the socket should get sent back to userspace for that. That would probably require some wrangling, and it's certainly easier to do it in a good enough fashion rather than try to clean it up and get it upstreamed, but it's not impossible. Sure; I agree. I tried to address that under "It isn't really obvious how it would be cleaned up to be general enough to import, and it would be a ton of work with little payoff." After all, you have to integrate with OpenSSL (or whatever) on the userspace side somehow too. And OpenSSL is contrib code, so we can't just hack it willy-nilly.
  - caf 8y ago
    OpenSSL already has integration for Linux's similar kTLS support (bulk transmit encryption only; all other alerts, renegotiation etc punted back to userspace). If you wrote it to provide the same API then the OpenSSL support side would be easy. https://github.com/openssl/openssl/blob/d6c3c1896cf3c0d69bc27da923d63f8130b13ca0/include/internal/ktls.h https://github.com/openssl/openssl/blob/d6c3c1896cf3c0d69bc2...
    - loeg 8y ago
      Thanks, I didn't know about that. That makes it seem a lot more viable.

GautamGoel 8y ago

I'm confused: wasn't it already mainlined? https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-Faster-Sendfile https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-...

2trill2spill 8y ago
That's async sendfile, it doesn't do TLS.