3 ms·

Cloudflare architecture and how BPF eats the world

96 points by migueldemoura 7y ago

mratsim 7y ago
I probably missed something very obvious as it's not defined in the article but what is BPF?
- detaro 7y ago
  https://en.wikipedia.org/wiki/Berkeley_Packet_Filter https://en.wikipedia.org/wiki/Berkeley_Packet_Filter A small bytecode VM for filtering data.
  - stingraycharles 7y ago
    Could anyone elaborate why it seems that the BPF VM is talked about so much nowadays? To me as an outsider it almost appears as if the VM is treated as a stand-alone thing, rather than just a component of a firewall.
    - jakeogh 7y ago
      Some examples: https://github.com/iovisor/bcc/tree/master/tools https://github.com/iovisor/bcc/tree/master/tools
    - wolfgang42 7y ago
      "Packet Filters" is a bit of a misnomer; as I understand it, it originally started out as a way to write complex firewall rules but the ability to safely run arbitrary code in the kernel has proven so useful that other Linux subsystems are gradually adding eBPF support as well. For example, you can now write a BPF program to filter what system calls a process can execute.
    - majke 7y ago
      Author here. For one - read the article! BPF was originally used purely for tcpdump, but has been re-used since then. Nowadays it's used in surprising number of Linux subsystems. eBPF allows us to do things previously impossible - you can execute untrusted code in the context of ring 0 kernel. This allows for: - exposing details of high-bandwidth stuff, like packets - advanced actions - XDP allows for packet forwarding or dropping, not only filtering and sending to userspace; TCP-BPF enables tuning finely-grained tcp performance details. These things fundamentally can't be exposed to userspace in easy way (or would be very hard to implement right). Userspace program can always not-respond,be de-scheduled, while eBPF program will have a response ready very quick. eBPF can be run in critical places of the kernel. This allows for very powerful use cases.
- skybrian 7y ago
  Also see: https://lwn.net/Articles/740157/ https://lwn.net/Articles/740157/
nahtnam 7y ago
Unrelated but I'm interested to see what stack all of the things mentioned in the article are on. Handling (I'm guessing billions) of requests while getting sub-ms (maybe even microseconds) latencies. Is it all written in Go? If so that is quite impressive.
- ec109685 7y ago
  This discusses their stack: https://blog.cloudflare.com/end-of-the-road-for-cloudflare-nginx/ https://blog.cloudflare.com/end-of-the-road-for-cloudflare-n... C, Go, Rust Lua and other tech.
  - nahtnam 7y ago
    It would be interesting to see what kinds of optimizations they have done in terms of programming languages. For example, Discord releases blog posts now and then on how they squeeze performance out of Elixir

96 points by migueldemoura 7y ago

mratsim 7y ago

I probably missed something very obvious as it's not defined in the article but what is BPF?

detaro 7y ago
https://en.wikipedia.org/wiki/Berkeley_Packet_Filter https://en.wikipedia.org/wiki/Berkeley_Packet_Filter A small bytecode VM for filtering data.
- stingraycharles 7y ago
  Could anyone elaborate why it seems that the BPF VM is talked about so much nowadays? To me as an outsider it almost appears as if the VM is treated as a stand-alone thing, rather than just a component of a firewall.
  - jakeogh 7y ago
    Some examples: https://github.com/iovisor/bcc/tree/master/tools https://github.com/iovisor/bcc/tree/master/tools
  - wolfgang42 7y ago
    "Packet Filters" is a bit of a misnomer; as I understand it, it originally started out as a way to write complex firewall rules but the ability to safely run arbitrary code in the kernel has proven so useful that other Linux subsystems are gradually adding eBPF support as well. For example, you can now write a BPF program to filter what system calls a process can execute.
  - majke 7y ago
    Author here. For one - read the article! BPF was originally used purely for tcpdump, but has been re-used since then. Nowadays it's used in surprising number of Linux subsystems. eBPF allows us to do things previously impossible - you can execute untrusted code in the context of ring 0 kernel. This allows for: - exposing details of high-bandwidth stuff, like packets - advanced actions - XDP allows for packet forwarding or dropping, not only filtering and sending to userspace; TCP-BPF enables tuning finely-grained tcp performance details. These things fundamentally can't be exposed to userspace in easy way (or would be very hard to implement right). Userspace program can always not-respond,be de-scheduled, while eBPF program will have a response ready very quick. eBPF can be run in critical places of the kernel. This allows for very powerful use cases.
skybrian 7y ago
Also see: https://lwn.net/Articles/740157/ https://lwn.net/Articles/740157/

nahtnam 7y ago

Unrelated but I'm interested to see what stack all of the things mentioned in the article are on. Handling (I'm guessing billions) of requests while getting sub-ms (maybe even microseconds) latencies. Is it all written in Go? If so that is quite impressive.

ec109685 7y ago
This discusses their stack: https://blog.cloudflare.com/end-of-the-road-for-cloudflare-nginx/ https://blog.cloudflare.com/end-of-the-road-for-cloudflare-n... C, Go, Rust Lua and other tech.
- nahtnam 7y ago
  It would be interesting to see what kinds of optimizations they have done in terms of programming languages. For example, Discord releases blog posts now and then on how they squeeze performance out of Elixir