3 ms·

We constantly think about the possibilities of using exotic hardware for acceleration. So far we've got a very, very long way with "commodity" hardware and the

We constantly think about the possibilities of using exotic hardware for acceleration. So far we've got a very, very long way with "commodity" hardware and the Linux kernel. One day, we'll probably do something exotic. https://blog.cloudflare.com/cloudflares-gen-x-servers-for-an-accelerated-future/ https://blog.cloudflare.com/cloudflares-gen-x-servers-for-an... https://blog.cloudflare.com/tubular-fixing-the-socket-api-with-ebpf/ https://blog.cloudflare.com/tubular-fixing-the-socket-api-wi...

junon 4y ago
Just curious, is there a high cost overhead (aside from the cost of actual units) of e.g. custom silicon?
- jgrahamc 4y ago
  It’s just a question of “is it worth it?”. How much does this cool accelerator thing cost? How much does it save?
Thaxll 4y ago
Why not using the same kind of solution as Netflix does where they offload TLS and the like in hardware? https://www.nvidia.com/en-us/networking/ethernet/connectx-6-dx/ https://www.nvidia.com/en-us/networking/ethernet/connectx-6-...
- nothasan 4y ago
  Pretty sure they analyse TLS handshakes for their Bot Management product and probably other things.
anonymoushn 4y ago
Have you tried out io_uring? My impression is that these Rust libs end up making different syscalls that are less great.
- nevi-me 4y ago
  Tokio supports io_uring (https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring), so perhaps when it's mature and battle-tested, it'd be easier to transition to it if Cloudflare aren't using it already.
- jlokier 4y ago
  Don't assume io_uring is an automatic win. Last time I benchmarked io_uring, it was good enough, but I was able to get measurably faster from syscalls and a well-crafted userspace thread pool for storage to NVMe. (Note: the I/O thread pool I wrote came out noticably faster than fio, the benchmark tool, so don't assume fio results are the best possible.) For that reason, I have a small application which can be configured for either io_uring or the thread pool, and the thread pool is preferred for performance.
londons_explore 4y ago
I don't think the hardware need be very exotic - most network hardware already has firmware capable of various types of acceleration, and all you would have to do is partner with the network card vendor and have a joint team write firmware that has all the features you need. With shared source, the network card vendor can even repackage some of the types of acceleration you write for their other clients - and I bet they're already looking into acceleration of HTTP/3. It's a win-win - because cloudflare gets massive compute savings, and the network card vendor knows they've built quite a high moat to buying someone else's hardware.

junon 4y ago

Just curious, is there a high cost overhead (aside from the cost of actual units) of e.g. custom silicon?

jgrahamc 4y ago
It’s just a question of “is it worth it?”. How much does this cool accelerator thing cost? How much does it save?

Thaxll 4y ago

Why not using the same kind of solution as Netflix does where they offload TLS and the like in hardware? https://www.nvidia.com/en-us/networking/ethernet/connectx-6-dx/ https://www.nvidia.com/en-us/networking/ethernet/connectx-6-...

nothasan 4y ago
Pretty sure they analyse TLS handshakes for their Bot Management product and probably other things.

anonymoushn 4y ago

Have you tried out io_uring? My impression is that these Rust libs end up making different syscalls that are less great.

nevi-me 4y ago
Tokio supports io_uring (https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring), so perhaps when it's mature and battle-tested, it'd be easier to transition to it if Cloudflare aren't using it already.
jlokier 4y ago
Don't assume io_uring is an automatic win. Last time I benchmarked io_uring, it was good enough, but I was able to get measurably faster from syscalls and a well-crafted userspace thread pool for storage to NVMe. (Note: the I/O thread pool I wrote came out noticably faster than fio, the benchmark tool, so don't assume fio results are the best possible.) For that reason, I have a small application which can be configured for either io_uring or the thread pool, and the thread pool is preferred for performance.

londons_explore 4y ago

I don't think the hardware need be very exotic - most network hardware already has firmware capable of various types of acceleration, and all you would have to do is partner with the network card vendor and have a joint team write firmware that has all the features you need. With shared source, the network card vendor can even repackage some of the types of acceleration you write for their other clients - and I bet they're already looking into acceleration of HTTP/3. It's a win-win - because cloudflare gets massive compute savings, and the network card vendor knows they've built quite a high moat to buying someone else's hardware.