5 ms·

I've been hearing more and more about eBPF, especially here on HN. I haven't yet watched the documentary so perhaps it is answered there. But, the analogy of J

• lmm 3y ago
  > But, the analogy of JavaScript inside the kernel is great and I'm left wondering: what was the way to do it previously? Userland network tool? This standardizes on a interface to the kernel, not a language, right? Guess/sketch: It's a language in most senses. Previously the kernel had APIs for packet filtering rules for iptables etc., but the set of rules you could use was somewhat "static" - rules would have parameters, so you could do things like if the source IP is in this range then rewrite it as this and direct it to this interface, but it was kind of like one of those visual flowchart languages where you can drag and drop the available boxes in a given order, but if there isn't a box to do what you want then you're stuck. Whereas with eBPF it really is scriptable - rather than a specific rule type you can just submit the script you want it to run - and nowadays it's become kind of a general kernel scripting language rather than just for networking. I'd draw a parallel with how 3D graphics programming has shifted from "you can do these kinds of transformations, submit a list of what you want to run in what order" to "this is our shader programming language, just write whatever you want to do as a program in this language".
  • bhickey 3y ago
    iptables is definitely limited in comparison to eBPF, but that isn't the innovative step. BPF was around for more than twenty years before eBPF came around. Around 2013 I worked on a packet analysis pipeline that generated BPF code dynamically at runtime. eBPF isn't more scriptable than BPF in this sense. The language does add some opcodes and loops that weren't available in the original, but this is relatively modest. The real genius of what these folks did was extending the usefulness of BPF beyond the network stack. Without a provably safe language it would've been impossible to enable flexible kernel tracing.
    • nailer 3y ago
      Out of interest - it's a while since I've done infra work - is iptables still around? Is it rewritten to be based on eBPF now? Do people just make .bt or .py files that call eBPF instead to implement packet filtering?
      • deleted 3y ago
        [deleted]
      • suprjami 3y ago
        No, iptables is dead and the interface to netfilter is nftables.
        • nailer 3y ago
          Thanks. Does nftables use eBPF?
          • suprjami 3y ago
            No
• ants_everywhere 3y ago
  > This standardizes on a interface to the kernel, not a language, right? It's a VM that runs JIT-compiled eBPF programs. You can write code in C or Golang or other languages that compiles down to eBPF. I did a video looking at the kernel eBPF code here: https://youtu.be/hznUH_zP77U?t=1165 https://youtu.be/hznUH_zP77U?t=1165 > wondering: what was the way to do it previously? Userland network tool? My understanding is that userland network tools were common in fields like finance that needed fast custom networking and wanted to eliminate the overhead of context switching. I don't know how common they are/were in other fields though.
  • shp0ngle 3y ago
    golang compiles to eBPF? including the whole golang runtime with garbage collector? that... surprises me edit: I don't see anything on the web that would show golang compiles to eBPF. I see bpf2go which is the other way around.
    • thatloststudent 3y ago
      This [1] makes me think gobpf uses C bindings to bcc, so golang does not compile to eBPF, bcc does. [1] - https://github.com/iovisor/gobpf/tree/master/bcc https://github.com/iovisor/gobpf/tree/master/bcc
    • ants_everywhere 3y ago
      Oh, no I don't mean that arbitrary Go compiles to eBPF. Apologies if I gave that impression. I meant that there are libraries that let you compose eBPF programs in other languages. But you're still putting together an eBPF program, just like you can assemble JSON with Go but you can't compile an arbitrary Go program to JSON. Cilium's eBPF library is the Go one I had in mind: https://github.com/cilium/ebpf https://github.com/cilium/ebpf Here's an example from that repo: https://github.com/cilium/ebpf/tree/main/examples/ringbuffer https://github.com/cilium/ebpf/tree/main/examples/ringbuffer
    • spaceywilly 3y ago
      BPF works more like Java, where you have your high-level source code, then a JIT compiler translates it to bytecode that runs on a VM. Even BPF “programs” written in C aren’t using a standard C compiler. The BPF JIT compiler translates them to bytecode. The subset of C you can use is fairly limited, only what their compiler supports. So what people have done is written compilers that can take Go or Python and compile it to BPF bytecode.
• jeremiahbuckley 3y ago
  One of the big wins is not so much “build and run your own stuff” but there are very nice low-cost (in terms of compute) performance utilities built on eBPF https://github.com/iovisor/bcc https://github.com/iovisor/bcc There are so many utilities in that list; there’s a diagram midway down the readme which tries to help show their uses. bcc-tools should be available in any distro. Also, Brendan Gregg does a ton of performance stuff that is worth knowing about if you check out his other work. Not eBPF only. Flame graphs are useful.
• tptacek 3y ago
  Ad-hoc kernel extensions were a pretty common answer, and one thing a lot of people love about eBPF is that it subsumes most of the reasons people wrote lkms commercially.
• suprjami 3y ago
  > what was the way to do it previously? SystemTap, and it still is. BPF can only do so much.