3 ms·

Compiling to eBPF is hard! The compiler must: - avoid loops (backwards jumps) - unroll everything - inline everything - no function calls Basically, it's imp

by majke 7y ago

Compiling to eBPF is hard! The compiler must: - avoid loops (backwards jumps) - unroll everything - inline everything - no function calls Basically, it's impossible to use clang to generate eBPF without -O2. Sorry.

justinclift 7y ago
>Basically, it's impossible to use clang to generate eBPF without -O2. Sorry. That's a bit confusing, as the comment by praseodym shows the output from doing just that. ???
- Dangeranger 7y ago
  I believe he is saying that eBPF will disallow certain code to run if it doesn’t meet the standards listed by the parent comment. The article also mentions that eBPF will check whether code is “worthy” or running or rearrange it so that it is worthy. The idea is to compile to assembly which both fixes the bug, and is runnable by eBPF.
  - majke 7y ago
    Apologies. Indeed, not all generated clang eBPF is loadable eBPF. So we can sum up: - Clang can compile C - eBPF backend can generate the eBPF bytecode for subset of IR (AFAIU not all IR, not all "C" is okay for eBPF) - verifier can accept or reject the generated eBPF. The goal is to write C in such a way that will be generally sensible C but that will consistently yield loadable eBPF programs. This is not easy! Just recently I was debugging code where clang optmizer decided to merge two loads of 2 bytes into one load of 4 bytes. This made the verifier unhappy - the specific loads are rewritten to helper calls by verifier. The verifier knows what to do on 2-byte load at offset X, and 2-byte load at offset X+2. But it has no idea what to do with 4-byte load at offset X! Writing C programs that will compile to valid, loadable eBPF requires a fair amount of voodoo.
    - justinclift 7y ago
      Thanks, that makes sense. :) Kind of sounds like having a specific eBPF transformation pass available in LLVM would be useful. eg having it generate known-to-work-with-eBPF output
lukego 7y ago
Interestingly, you are basically describing the way that LuaJIT compiles Lua code :)
sreque 7y ago
I'm new to this area of work, but has anyone stepped back and thought: "this is not the right way to build eBPF programs"? Would it be better to create a new high-level language and toolset? All this voodoo hackery to try to trick a C compiler into making eBPF-compatible code feels unsustainable.
- asgeir 7y ago
  bcc https://github.com/iovisor/bcc https://github.com/iovisor/bcc if you need low-level abstractions or bpftrace https://github.com/iovisor/bpftrace https://github.com/iovisor/bpftrace if you need something simpler (similar to dtrace)

by majke 7y ago

justinclift 7y ago

>Basically, it's impossible to use clang to generate eBPF without -O2. Sorry. That's a bit confusing, as the comment by praseodym shows the output from doing just that. ???

Dangeranger 7y ago
I believe he is saying that eBPF will disallow certain code to run if it doesn’t meet the standards listed by the parent comment. The article also mentions that eBPF will check whether code is “worthy” or running or rearrange it so that it is worthy. The idea is to compile to assembly which both fixes the bug, and is runnable by eBPF.
- majke 7y ago
  Apologies. Indeed, not all generated clang eBPF is loadable eBPF. So we can sum up: - Clang can compile C - eBPF backend can generate the eBPF bytecode for subset of IR (AFAIU not all IR, not all "C" is okay for eBPF) - verifier can accept or reject the generated eBPF. The goal is to write C in such a way that will be generally sensible C but that will consistently yield loadable eBPF programs. This is not easy! Just recently I was debugging code where clang optmizer decided to merge two loads of 2 bytes into one load of 4 bytes. This made the verifier unhappy - the specific loads are rewritten to helper calls by verifier. The verifier knows what to do on 2-byte load at offset X, and 2-byte load at offset X+2. But it has no idea what to do with 4-byte load at offset X! Writing C programs that will compile to valid, loadable eBPF requires a fair amount of voodoo.
  - justinclift 7y ago
    Thanks, that makes sense. :) Kind of sounds like having a specific eBPF transformation pass available in LLVM would be useful. eg having it generate known-to-work-with-eBPF output

lukego 7y ago

Interestingly, you are basically describing the way that LuaJIT compiles Lua code :)

sreque 7y ago

I'm new to this area of work, but has anyone stepped back and thought: "this is not the right way to build eBPF programs"? Would it be better to create a new high-level language and toolset? All this voodoo hackery to try to trick a C compiler into making eBPF-compatible code feels unsustainable.

asgeir 7y ago
bcc https://github.com/iovisor/bcc https://github.com/iovisor/bcc if you need low-level abstractions or bpftrace https://github.com/iovisor/bpftrace https://github.com/iovisor/bpftrace if you need something simpler (similar to dtrace)