4 ms·

The BPF capability should really only be given to root. I don't think it really gives any new attack surface. All I could see is it giving black-hats an easier

The BPF capability should really only be given to root. I don't think it really gives any new attack surface. All I could see is it giving black-hats an easier interface to "kernel-level-fuckery".

_3u10 5y ago
It’s not an easier interface. It’s much easier to write kernel modules than to mess around with eBPF.
- stormbrew 5y ago
  Easier to get started maybe, but there's something to be said for the 'ease' of having to worry a lot less about crashing the kernel you're working on.
  - xyzzy_plugh 5y ago
    I've crashed multiple kernels with eBPF programs. I don't buy this argument at all.
    - justicezyx 5y ago
      Could you share the ebpf program and the external dependencies that can reproduce the crash? It'll be valuable to learn this, so that we might be able to proactively address them. Ebpf is the core of our product.
    - deleted 5y ago
      [deleted]
    - roblabla 5y ago
      Those would be considered kernel bugs though, and should be getting fixed when reported. While a crash in a kernel module is just a bug in the kernel module itself
    - stormbrew 5y ago
      I said 'less' rather than 'at all' for a reason. I'm not really sure what to tell you if you disagree that it's a lot easier to accidentally crash your kernel with a from-scratch module than ebpf though. That's an experience pretty drastically at odds with mine.
    - zokula 5y ago
      I call bullshit! Ebpf programs cannot crash the kernel since they cannot be run if they contain bugs in the first place.
      - daenz 5y ago
        That assumes that the virtual machine has no bugs
        tptacek 5y ago
        Not so much (eBPF VM bugs are pretty rare, as you'd expect, since the VM is very simple) --- you're much more likely to run into bugs in the C-code helpers the kernel exports. If you're malicious, you can also hit verifier bugs that'll give your eBPF code raw pointers, but I don't think you're likely to stumble on them accidentally.
      - xyzzy_plugh 5y ago
        The kernel, however, can and does have bugs. I've seen several thousand hosts get taken down with a perfectly correct eBPF program due to a buggy kernel.
    - tptacek 5y ago
      Can you describe some of those bugs? What helpers were you using? I've been doing bonkers stuff with eBPF for the last year and, while I've definitely had bugs, none of them took my kernel down.
      - xyzzy_plugh 5y ago
        I no longer have access to the code, and the kernel has since been patched. Mostly doing some observability around new network connections. We definitely hit this one at some point: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454 We also ran into a couple of similar but unrelated panic bugs on much newer kernels on non-Ubuntu distros.
  - deleted 5y ago
    [deleted]
- tptacek 5y ago
  That seems wrong. The build environment for eBPF programs is simpler (you don't even need a working kernel tree), and, much more importantly, eBPF programs are constrained and can't crash the kernel, which is easy to do accidentally when writing an freestyle C LKM. You can learn enough to get stuff done with eBPF inside of a couple days; the same is absolutely not true of kernel modules.
  - _3u10 5y ago
    I find the verifier to be extremely frustrating. It's also very difficult to debug eBPF programs, especially in regards to why it's not passing verifier. Combine with ridiculous restrictions from BCC like not being able to use functions in macros, or the vagueries of the bpf target in clang and it's extremely frustrating. I've personally never found obtaining a working kernel tree to be difficult, certainly easier than a working BCC toolchain. Or all of the various compiler flags needed for clang not to emit code incompatible with the verifier.
    - tptacek 5y ago
      I don't use BCC or really understand why anyone would; I use clang, and just produce .o's. The verifier is definitely annoying, especially at first, but I found myself sort of quickly working out the verifier's expected idiom, and a lot of it can be wrapped with macros. All of this drama pales in comparison to writing freestyle C code in the Linux kernel without causing random panics.
staticassertion 5y ago
I'd recommend to anyone running Linux that they do exactly that - disable unprivileged eBPF. For a server you shouldn't need that, you can just drop privs for your service after setting up the filter. They should probably put it behind its own capability like CAP_LOAD_EBPF. Forcing signed ebpf will also help though.

The BPF capability should really only be given to root. I don't think it really gives any new attack surface. All I could see is it giving black-hats an easier interface to "kernel-level-fuckery".

_3u10 5y ago

It’s not an easier interface. It’s much easier to write kernel modules than to mess around with eBPF.

stormbrew 5y ago
Easier to get started maybe, but there's something to be said for the 'ease' of having to worry a lot less about crashing the kernel you're working on.
- xyzzy_plugh 5y ago
  I've crashed multiple kernels with eBPF programs. I don't buy this argument at all.
  - justicezyx 5y ago
    Could you share the ebpf program and the external dependencies that can reproduce the crash? It'll be valuable to learn this, so that we might be able to proactively address them. Ebpf is the core of our product.
  - deleted 5y ago
    [deleted]
  - roblabla 5y ago
    Those would be considered kernel bugs though, and should be getting fixed when reported. While a crash in a kernel module is just a bug in the kernel module itself
  - stormbrew 5y ago
    I said 'less' rather than 'at all' for a reason. I'm not really sure what to tell you if you disagree that it's a lot easier to accidentally crash your kernel with a from-scratch module than ebpf though. That's an experience pretty drastically at odds with mine.
  - zokula 5y ago
    I call bullshit! Ebpf programs cannot crash the kernel since they cannot be run if they contain bugs in the first place.
    - daenz 5y ago
      That assumes that the virtual machine has no bugs
      - tptacek 5y ago
        Not so much (eBPF VM bugs are pretty rare, as you'd expect, since the VM is very simple) --- you're much more likely to run into bugs in the C-code helpers the kernel exports. If you're malicious, you can also hit verifier bugs that'll give your eBPF code raw pointers, but I don't think you're likely to stumble on them accidentally.
    - xyzzy_plugh 5y ago
      The kernel, however, can and does have bugs. I've seen several thousand hosts get taken down with a perfectly correct eBPF program due to a buggy kernel.
  - tptacek 5y ago
    Can you describe some of those bugs? What helpers were you using? I've been doing bonkers stuff with eBPF for the last year and, while I've definitely had bugs, none of them took my kernel down.
    - xyzzy_plugh 5y ago
      I no longer have access to the code, and the kernel has since been patched. Mostly doing some observability around new network connections. We definitely hit this one at some point: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454 https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1763454 We also ran into a couple of similar but unrelated panic bugs on much newer kernels on non-Ubuntu distros.
- deleted 5y ago
  [deleted]
tptacek 5y ago
That seems wrong. The build environment for eBPF programs is simpler (you don't even need a working kernel tree), and, much more importantly, eBPF programs are constrained and can't crash the kernel, which is easy to do accidentally when writing an freestyle C LKM. You can learn enough to get stuff done with eBPF inside of a couple days; the same is absolutely not true of kernel modules.
- _3u10 5y ago
  I find the verifier to be extremely frustrating. It's also very difficult to debug eBPF programs, especially in regards to why it's not passing verifier. Combine with ridiculous restrictions from BCC like not being able to use functions in macros, or the vagueries of the bpf target in clang and it's extremely frustrating. I've personally never found obtaining a working kernel tree to be difficult, certainly easier than a working BCC toolchain. Or all of the various compiler flags needed for clang not to emit code incompatible with the verifier.
  - tptacek 5y ago
    I don't use BCC or really understand why anyone would; I use clang, and just produce .o's. The verifier is definitely annoying, especially at first, but I found myself sort of quickly working out the verifier's expected idiom, and a lot of it can be wrapped with macros. All of this drama pales in comparison to writing freestyle C code in the Linux kernel without causing random panics.

staticassertion 5y ago

I'd recommend to anyone running Linux that they do exactly that - disable unprivileged eBPF. For a server you shouldn't need that, you can just drop privs for your service after setting up the filter. They should probably put it behind its own capability like CAP_LOAD_EBPF. Forcing signed ebpf will also help though.