6 ms·

eBPF – The Future of Networking and Security

195 points by genbit 6y ago

ryanmarsh 6y ago
I'm not an expert in BPF by any means. My gut tells me that the hype of eBPF is an example of Hyrum's law. That is, eBPF will be leveraged beyond its design intent, as an in-kernel JIT engine. This is more a comment on human nature than the technology itself.
- joestringer 6y ago
  Even looking at the original BPF which focused on filtering packets as they are forwarded to userspace (think tcpdump)[1] and looking at the extensions that eBPF provides on top to hook into various subsystems[2,3], it's clear that this is going far beyond the use cases originally envisioned. I'd love to see an eBPF paper to follow up / contrast with the '93 USENIX BPF paper. [1]: https://www.tcpdump.org/papers/bpf-usenix93.pdf https://www.tcpdump.org/papers/bpf-usenix93.pdf [2]: https://ebpf.io/what-is-ebpf#hook-overview https://ebpf.io/what-is-ebpf#hook-overview [3]: http://www.brendangregg.com/BPF/bpf_performance_tools_book.png http://www.brendangregg.com/BPF/bpf_performance_tools_book.p...
  - tptacek 6y ago
    FWIW: I just wrote a long-ish post on the history from BPF (and before BPF) to eBPF and XDP: https://fly.io/blog/bpf-xdp-packet-filters-and-udp/ https://fly.io/blog/bpf-xdp-packet-filters-and-udp/ An interesting fact is that packet filtering as a problem domain has been dominated by in-kernel virtual machines going back into the 1980s; it's an idea that comes all the way from Xerox.
    - teleforce 6y ago
      Need to know what's type of water the people at Xerox Palo Alto were drinking. They pioneered many groundbreaking and game changing works on computing including (but not limited to) windowing desktop environment, integrated programming/structural editor with CEDAR/Tioga, SQL (team moved to Oracle), Ethernet networks, laser printer, VLSI and Jupiter operational transform for distributed computing (precursor to CRDT). Each of this technology is now an industry of its own.
      - vvanders 6y ago
        I kinda feel like Dealers of Lightning should be required reading at this point[1], both for the breadth of invention and how they squandered it. [1] https://www.amazon.com/Dealers-Lightning-Xerox-PARC-Computer/dp/0887309895 https://www.amazon.com/Dealers-Lightning-Xerox-PARC-Computer...
      - pjmlp 6y ago
        Unfortunately we are still quite far from the safe computing platforms they were using at Xerox (Interlisp-D, Smalltalk, Mesa and Mesa/Cedar). The best we have gotten so far are the hybrids .NET/Windows, JME, Android Java/Linux, Chrome/Linux, Swift/iOS/macOS.
  - tgraf 6y ago
    The shift from BPF to eBPF was less of an evolutionary step as the name might indicate. The overlap with the name BPF is primarily due to the requirement for eBPF to be a superset of BPF in order to avoid having to maintain two virtual machines long-term. This was one of the conditions for eBPF to be merged and in that context, the name eBPF made sense.
    - tptacek 6y ago
      Disagree (see sibling post). Classic BPF could have been translated into any virtual machine design they came up with (because classic BPF is incredibly simple). When McCanne came up with the same design in 1998, his team called it "BPF+", for the same reason eBPF is called eBPF --- because it is pretty much an evolution of the earlier idea.
      - tgraf 6y ago
        I'm not going to argue with you. You can read up on initial naming and framing in slides of netconf and plumbers conferences as well as LKML archives.
        gonzo 6y ago
        Remember when Microsoft claimed to invent various computing technologies, even though they had been around since the 70s or earlier? That’s the type of history you’re articulating here.
        tptacek 6y ago
        To be clear: the dispute over the history of BPF/eBPF is not interesting, and I don't want to litigate it anymore than they do. I'm just here to say that eBPF and BPF are in fact pretty closely related. The eBPF design is uncannily similar to Begel, McCanne, and Graham's BPF+ design[1]; in particular, the BPF+ paper spends a fair amount of time describing an SSA-based compiler for a RISC-y register ISA, and eBPF... just uses (at this point) LLVM for a RISC-y register ISA. Most notably, the fundamental execution integrity model has, until pretty recently, remained the same --- forward jumps only, limited program size. And that's to me the defining feature of the architecture. The lineage isn't important to me, so much as the sort of continuous unbroken line from BPF to eBPF, regardless of what LKML says. [1]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.597.3024&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.597...
- tptacek 6y ago
  Easy to predict something that's already happening. :) https://github.com/xdp-project/xdp-tutorial https://github.com/xdp-project/xdp-tutorial It's a good thing, I think! Compared to loading new unmanaged C code into the kernel, BPF is a really nice way to add functionality to Linux.
F2hn 6y ago
Fuck to HN for shadow banning
tgraf 6y ago
Disclaimer: I wrote the post. Happy to answer any questions.
- plesiv 6y ago
  First of all, congrats. The tech is great and I hope you'll be able to make a company around it. As for the question: How are you looking to make money?
  - tgraf 6y ago
    I'm not going to spam this forum with a marketing pitch so I'll just refer to https://www.isovalent.com/product https://www.isovalent.com/product and add that you can buy a Cilium Enterprise distribution with enterprise specific add-ons from us.
- rurban 6y ago
  At first two annoying lies in the title alone. The Future of Networking? Networking is not only linux. eBPF is linux-only. Everyone else uses the secure variant dTrace, which has even wide-spread user-space support. So you can trace across the kernel, processes and its extensions/scripts. For decades. Future of Security? eBPF is insecure. User-accessible arrays in the kernel can never be secure. dTrace did not do that for a reason, it was already compromised with the spectre-like attacks, and the fixes were laughable at best to safe face. Linux might be advised to do better (or is just NIH?), but advertising Worse as Better was fashionable in the 80ies only.
  - tgraf 6y ago
    I personally think that networking will be almost exclusively based Linux in some form. If you want to interpret it as "eBPF - The Future of Linux Networking" then that is totally fine as well. That said, eBPF-based networking can be offloaded to SmartNICs already so it may be less Linux specific than you seem to assume right now. Comparing dTrace and eBPF is definitely a very interesting question. I've actually asked Brendan Gregg in the Q&A of his keynote at eBPF summit this year how he compares dTrace and eBPF these days. Here is his answer (jumps right to the specific question): https://youtu.be/jw8tEPP6jwQ?t=4618 https://youtu.be/jw8tEPP6jwQ?t=4618 I doubt that eBPF will remain a Linux-only technology. Ports to FreeBSD are already underway it seems [0] and Microsoft declared intent to invest into eBPF [1]. I'm not sure what that means on timeline for eBPF availability on Windows though. There are also several user space implementations for eBPF which could become interesting to provide a universal programmability approach across traditional kernels like Linux, microkernels like Snap and application kernels like gVisor. [0] https://papers.freebsd.org/2018/bsdcan/hayakawa-ebpf_implementation_for_freebsd/ https://papers.freebsd.org/2018/bsdcan/hayakawa-ebpf_impleme... [1] https://twitter.com/markrussinovich/status/1283039153920368651?s=20 https://twitter.com/markrussinovich/status/12830391539203686...
Ericson2314 6y ago
The future ought to be capabilities. All this policy scripting stuff is just drudgery make-work that gets us nowhere.
- tptacek 6y ago
  BPF wasn't originally conceived of as a reference monitor or ACL system; in fact, originally, it was believed that operating systems would use BPF-style packet filters to do pretty much all their demuxing.
  - Ericson2314 6y ago
    That's all true. I'm worried about what people will do with this stuff in practice (more rope to hang themselves) not eBPF fundamentally is.
  - trasz 6y ago
    AFAIK BPF wasn’t conceived as anything security-related, it was just an optimization.
    - tptacek 6y ago
      It was, but it was an optimization over earlier VM-based packet filters, which were definitely not optimizations; they were pursued as elegant system design, not high-performance networking.
    - silly-silly 6y ago
      From the article: Buggy kernel code will crash your machine. The kernel is not protected from a buggy kernel module. I think people assumed that this is just how things are; that's the price to do kernel programming. eBPF changed this dogma. It brought safety to kernel programming. "It brought safety to kernel programming" , if you use eBPF and don't expose bugs in the parser, or checking or validation systems. (These have already happened).
      - tptacek 6y ago
miohtama 6y ago
eBPF is also used high throughput blockchain, Solana https://github.com/solana-labs/rbpf https://github.com/solana-labs/rbpf Unlike more common Rust + LLVM + WASM toolchain, Solana smart contracts use Rust + LLVM + eBPF.
- chc4 6y ago
  Solana uses a custom Rust re-implementation of a custom C re-implementation of the Linux BPF VM for what appears to be licensing reasons. Notably, it's jitting all bytecode without a verifier or emitting runtime bounds checks[0]. I suspect you can pop a shell on every single computer on their testnet somewhere between "trivially" and "extremely trivially". They appear to be running some kind of "open security test"[1] but are only paying out their own imaginary funny money. I'd suggest you run for the hills as fast as you can instead of considering Solana. 0: https://github.com/solana-labs/rbpf/blob/f7007d6ae8728e61401f3ec0aa1008e05d167508/src/jit.rs#L374 https://github.com/solana-labs/rbpf/blob/f7007d6ae8728e61401... 1: https://forums.solana.com/t/tour-de-sol-stage-1-details/317 https://forums.solana.com/t/tour-de-sol-stage-1-details/317
  - miohtama 6y ago
    Interesting. I am not sure if your comment Without a verifier make sense. Because AFAIK you need to verify the contract only once, when it is deployed. Not every time it is invoked. Verifying a contract should be super cheap compared to executing it, unless eBPF verification is somehow super expensive.
ninjha 6y ago
I was under the impression that Cilium was one of the more common choices for Kubernetes CNI but judging by the other comments... maybe not? We’re currently moving to Kubernetes for our infrastructure at the Berkeley OCF (https://ocf.berkeley.edu/ https://ocf.berkeley.edu/), and picked Cilium for all the networking things. It’s good to see that there’s a company backing it now!

195 points by genbit 6y ago

I'm not an expert in BPF by any means. My gut tells me that the hype of eBPF is an example of Hyrum's law. That is, eBPF will be leveraged beyond its design intent, as an in-kernel JIT engine. This is more a comment on human nature than the technology itself.

joestringer 6y ago
Even looking at the original BPF which focused on filtering packets as they are forwarded to userspace (think tcpdump)[1] and looking at the extensions that eBPF provides on top to hook into various subsystems[2,3], it's clear that this is going far beyond the use cases originally envisioned. I'd love to see an eBPF paper to follow up / contrast with the '93 USENIX BPF paper. [1]: https://www.tcpdump.org/papers/bpf-usenix93.pdf https://www.tcpdump.org/papers/bpf-usenix93.pdf [2]: https://ebpf.io/what-is-ebpf#hook-overview https://ebpf.io/what-is-ebpf#hook-overview [3]: http://www.brendangregg.com/BPF/bpf_performance_tools_book.png http://www.brendangregg.com/BPF/bpf_performance_tools_book.p...
- tptacek 6y ago
  FWIW: I just wrote a long-ish post on the history from BPF (and before BPF) to eBPF and XDP: https://fly.io/blog/bpf-xdp-packet-filters-and-udp/ https://fly.io/blog/bpf-xdp-packet-filters-and-udp/ An interesting fact is that packet filtering as a problem domain has been dominated by in-kernel virtual machines going back into the 1980s; it's an idea that comes all the way from Xerox.
  - teleforce 6y ago
    Need to know what's type of water the people at Xerox Palo Alto were drinking. They pioneered many groundbreaking and game changing works on computing including (but not limited to) windowing desktop environment, integrated programming/structural editor with CEDAR/Tioga, SQL (team moved to Oracle), Ethernet networks, laser printer, VLSI and Jupiter operational transform for distributed computing (precursor to CRDT). Each of this technology is now an industry of its own.
    - vvanders 6y ago
      I kinda feel like Dealers of Lightning should be required reading at this point[1], both for the breadth of invention and how they squandered it. [1] https://www.amazon.com/Dealers-Lightning-Xerox-PARC-Computer/dp/0887309895 https://www.amazon.com/Dealers-Lightning-Xerox-PARC-Computer...
    - pjmlp 6y ago
      Unfortunately we are still quite far from the safe computing platforms they were using at Xerox (Interlisp-D, Smalltalk, Mesa and Mesa/Cedar). The best we have gotten so far are the hybrids .NET/Windows, JME, Android Java/Linux, Chrome/Linux, Swift/iOS/macOS.
- tgraf 6y ago
  The shift from BPF to eBPF was less of an evolutionary step as the name might indicate. The overlap with the name BPF is primarily due to the requirement for eBPF to be a superset of BPF in order to avoid having to maintain two virtual machines long-term. This was one of the conditions for eBPF to be merged and in that context, the name eBPF made sense.
  - tptacek 6y ago
    Disagree (see sibling post). Classic BPF could have been translated into any virtual machine design they came up with (because classic BPF is incredibly simple). When McCanne came up with the same design in 1998, his team called it "BPF+", for the same reason eBPF is called eBPF --- because it is pretty much an evolution of the earlier idea.
    - tgraf 6y ago
      I'm not going to argue with you. You can read up on initial naming and framing in slides of netconf and plumbers conferences as well as LKML archives.
      - gonzo 6y ago
        Remember when Microsoft claimed to invent various computing technologies, even though they had been around since the 70s or earlier? That’s the type of history you’re articulating here.
        tptacek 6y ago
        To be clear: the dispute over the history of BPF/eBPF is not interesting, and I don't want to litigate it anymore than they do. I'm just here to say that eBPF and BPF are in fact pretty closely related. The eBPF design is uncannily similar to Begel, McCanne, and Graham's BPF+ design[1]; in particular, the BPF+ paper spends a fair amount of time describing an SSA-based compiler for a RISC-y register ISA, and eBPF... just uses (at this point) LLVM for a RISC-y register ISA. Most notably, the fundamental execution integrity model has, until pretty recently, remained the same --- forward jumps only, limited program size. And that's to me the defining feature of the architecture. The lineage isn't important to me, so much as the sort of continuous unbroken line from BPF to eBPF, regardless of what LKML says. [1]: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.597.3024&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.597...
tptacek 6y ago
Easy to predict something that's already happening. :) https://github.com/xdp-project/xdp-tutorial https://github.com/xdp-project/xdp-tutorial It's a good thing, I think! Compared to loading new unmanaged C code into the kernel, BPF is a really nice way to add functionality to Linux.

F2hn 6y ago

Fuck to HN for shadow banning

tgraf 6y ago

Disclaimer: I wrote the post. Happy to answer any questions.

plesiv 6y ago
First of all, congrats. The tech is great and I hope you'll be able to make a company around it. As for the question: How are you looking to make money?
- tgraf 6y ago
  I'm not going to spam this forum with a marketing pitch so I'll just refer to https://www.isovalent.com/product https://www.isovalent.com/product and add that you can buy a Cilium Enterprise distribution with enterprise specific add-ons from us.
rurban 6y ago
At first two annoying lies in the title alone. The Future of Networking? Networking is not only linux. eBPF is linux-only. Everyone else uses the secure variant dTrace, which has even wide-spread user-space support. So you can trace across the kernel, processes and its extensions/scripts. For decades. Future of Security? eBPF is insecure. User-accessible arrays in the kernel can never be secure. dTrace did not do that for a reason, it was already compromised with the spectre-like attacks, and the fixes were laughable at best to safe face. Linux might be advised to do better (or is just NIH?), but advertising Worse as Better was fashionable in the 80ies only.
- tgraf 6y ago
  I personally think that networking will be almost exclusively based Linux in some form. If you want to interpret it as "eBPF - The Future of Linux Networking" then that is totally fine as well. That said, eBPF-based networking can be offloaded to SmartNICs already so it may be less Linux specific than you seem to assume right now. Comparing dTrace and eBPF is definitely a very interesting question. I've actually asked Brendan Gregg in the Q&A of his keynote at eBPF summit this year how he compares dTrace and eBPF these days. Here is his answer (jumps right to the specific question): https://youtu.be/jw8tEPP6jwQ?t=4618 https://youtu.be/jw8tEPP6jwQ?t=4618 I doubt that eBPF will remain a Linux-only technology. Ports to FreeBSD are already underway it seems [0] and Microsoft declared intent to invest into eBPF [1]. I'm not sure what that means on timeline for eBPF availability on Windows though. There are also several user space implementations for eBPF which could become interesting to provide a universal programmability approach across traditional kernels like Linux, microkernels like Snap and application kernels like gVisor. [0] https://papers.freebsd.org/2018/bsdcan/hayakawa-ebpf_implementation_for_freebsd/ https://papers.freebsd.org/2018/bsdcan/hayakawa-ebpf_impleme... [1] https://twitter.com/markrussinovich/status/1283039153920368651?s=20 https://twitter.com/markrussinovich/status/12830391539203686...

Ericson2314 6y ago

The future ought to be capabilities. All this policy scripting stuff is just drudgery make-work that gets us nowhere.

tptacek 6y ago
BPF wasn't originally conceived of as a reference monitor or ACL system; in fact, originally, it was believed that operating systems would use BPF-style packet filters to do pretty much all their demuxing.
- Ericson2314 6y ago
  That's all true. I'm worried about what people will do with this stuff in practice (more rope to hang themselves) not eBPF fundamentally is.
- trasz 6y ago
  AFAIK BPF wasn’t conceived as anything security-related, it was just an optimization.
  - tptacek 6y ago
    It was, but it was an optimization over earlier VM-based packet filters, which were definitely not optimizations; they were pursued as elegant system design, not high-performance networking.
  - silly-silly 6y ago
    From the article: Buggy kernel code will crash your machine. The kernel is not protected from a buggy kernel module. I think people assumed that this is just how things are; that's the price to do kernel programming. eBPF changed this dogma. It brought safety to kernel programming. "It brought safety to kernel programming" , if you use eBPF and don't expose bugs in the parser, or checking or validation systems. (These have already happened).
    - tptacek 6y ago

miohtama 6y ago

eBPF is also used high throughput blockchain, Solana https://github.com/solana-labs/rbpf https://github.com/solana-labs/rbpf Unlike more common Rust + LLVM + WASM toolchain, Solana smart contracts use Rust + LLVM + eBPF.

chc4 6y ago
Solana uses a custom Rust re-implementation of a custom C re-implementation of the Linux BPF VM for what appears to be licensing reasons. Notably, it's jitting all bytecode without a verifier or emitting runtime bounds checks[0]. I suspect you can pop a shell on every single computer on their testnet somewhere between "trivially" and "extremely trivially". They appear to be running some kind of "open security test"[1] but are only paying out their own imaginary funny money. I'd suggest you run for the hills as fast as you can instead of considering Solana. 0: https://github.com/solana-labs/rbpf/blob/f7007d6ae8728e61401f3ec0aa1008e05d167508/src/jit.rs#L374 https://github.com/solana-labs/rbpf/blob/f7007d6ae8728e61401... 1: https://forums.solana.com/t/tour-de-sol-stage-1-details/317 https://forums.solana.com/t/tour-de-sol-stage-1-details/317
- miohtama 6y ago
  Interesting. I am not sure if your comment Without a verifier make sense. Because AFAIK you need to verify the contract only once, when it is deployed. Not every time it is invoked. Verifying a contract should be super cheap compared to executing it, unless eBPF verification is somehow super expensive.

ninjha 6y ago

I was under the impression that Cilium was one of the more common choices for Kubernetes CNI but judging by the other comments... maybe not? We’re currently moving to Kubernetes for our infrastructure at the Berkeley OCF (https://ocf.berkeley.edu/ https://ocf.berkeley.edu/), and picked Cilium for all the networking things. It’s good to see that there’s a company backing it now!