12 ms·

Will there every be way to write probes/tracing scripts without dropping into C? I don't mind C in general, but I don't want to have to dig out the documentatio

Will there every be way to write probes/tracing scripts without dropping into C? I don't mind C in general, but I don't want to have to dig out the documentation for the eBPF C library and start writing hundreds of lines of C every time I want to run a trace. DTrace made this really nice, because you would write your tracing scripts in a high-level, awk-like language, which is the sort of thing well-suited to the purpose.

brendangregg 10y ago
Yes, see the section "A higher-level language", which mentions at least two projects: SystemTap+BPF and ply. Think of the current bcc/Python/C interface as a lightweight skin that was necessary during BPF development to kick the tires on various features, prototype tools, see what else needed to be done, etc. It may be good enough to stay around, as lots of tools have been written for it that will get used and be valuable. But there's room for higher-level languages too. If Sasha keeps developing his "trace" tool (and its summary counterpart, argdist), that may serve many such custom needs (as another option). See the various examples: https://github.com/iovisor/bcc/blob/master/tools/trace_example.txt https://github.com/iovisor/bcc/blob/master/tools/trace_examp... , like: # trace 'sys_read (arg3 > 20000) "read %d bytes", arg3' TIME PID COMM FUNC - 05:18:23 4490 dd sys_read read 1048576 bytes 05:18:23 4490 dd sys_read read 1048576 bytes 05:18:23 4490 dd sys_read read 1048576 bytes
- caf 10y ago
  How feasible would it be to compile the dtrace language itself to an eBPF backend?
  - qwertyuiop924 10y ago
    Huh. Is there a formal definition anywhere? If we have some kind of spec (even if it's not formal), it might be possible, since they are roughly equivalent, AFAIK. However, since I haven't worked in-depth in either, I'm unsure what work would be involved: it's possible that only a subset would compile. Anyways, this is probably a good goal to shoot for. Dtrace is the system tracer on pretty much all of the other big unixes, so it's a good idea to support as much of its language as possible. Plus, there are a lot of scripts already written in DTrace's language: having access to them would be invaluable.
    - brendangregg 10y ago
      Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. DTrace is Oracle copyrighted, and re-implementing a DTrace front-end on Linux sounds a lot like re-implementing an Oracle-copyrighted API.
      - Annatar 10y ago
        What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.
        qwertyuiop924 10y ago
        Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
        Annatar 10y ago
        What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
        qwertyuiop924 10y ago
        I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
- qwertyuiop924 10y ago
  Thanks. That looks very exciting, and it's a relief to know that the eBPF team has thought about these things, and are actually doing work on them.
Roxxik 10y ago
It shouldn't be that difficult to write some simple compiler to translate a scripty language that looks like awk to BPF-Assembly (or C and stuff that into LLVM). I might look into this stuff for my Bachelor Thesis ;)
bonzini 10y ago
Yes, people are working on an eBPF backend for systemtap.
- qwertyuiop924 10y ago
  Well, that will at least stop systemtap bringing your system to a crashing halt (literally). Although I believe they may have fixed that already. But what would you expect for software coming out of Red Hat? Their projects are usually a mixed bag at best.
vavrusa 10y ago
It wasn't mentioned in the article, but I've recently merged LuaJIT to BPF compiler. So you just write Lua and the kernel bits get compiled into BPF bytecode and loaded. No C. See https://github.com/iovisor/bcc/blob/master/src/lua/README.md#luajit-bpf-compiler https://github.com/iovisor/bcc/blob/master/src/lua/README.md... or examples https://github.com/iovisor/bcc/blob/master/examples/lua/tracepoint-offcputime.lua https://github.com/iovisor/bcc/blob/master/examples/lua/trac... (this one is for tracing).
- qwertyuiop924 10y ago
  Okay, that's really cool. Thanks. It does need more documentation, but this is an excellent step. It also allows for building higher level APIs atop BCC's low-level lua integration, as opposed to writing yet another DSL.

brendangregg 10y ago

Yes, see the section "A higher-level language", which mentions at least two projects: SystemTap+BPF and ply. Think of the current bcc/Python/C interface as a lightweight skin that was necessary during BPF development to kick the tires on various features, prototype tools, see what else needed to be done, etc. It may be good enough to stay around, as lots of tools have been written for it that will get used and be valuable. But there's room for higher-level languages too. If Sasha keeps developing his "trace" tool (and its summary counterpart, argdist), that may serve many such custom needs (as another option). See the various examples: https://github.com/iovisor/bcc/blob/master/tools/trace_example.txt https://github.com/iovisor/bcc/blob/master/tools/trace_examp... , like: # trace 'sys_read (arg3 > 20000) "read %d bytes", arg3' TIME PID COMM FUNC - 05:18:23 4490 dd sys_read read 1048576 bytes 05:18:23 4490 dd sys_read read 1048576 bytes 05:18:23 4490 dd sys_read read 1048576 bytes

caf 10y ago
How feasible would it be to compile the dtrace language itself to an eBPF backend?
- qwertyuiop924 10y ago
  Huh. Is there a formal definition anywhere? If we have some kind of spec (even if it's not formal), it might be possible, since they are roughly equivalent, AFAIK. However, since I haven't worked in-depth in either, I'm unsure what work would be involved: it's possible that only a subset would compile. Anyways, this is probably a good goal to shoot for. Dtrace is the system tracer on pretty much all of the other big unixes, so it's a good idea to support as much of its language as possible. Plus, there are a lot of scripts already written in DTrace's language: having access to them would be invaluable.
  - brendangregg 10y ago
    Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. DTrace is Oracle copyrighted, and re-implementing a DTrace front-end on Linux sounds a lot like re-implementing an Oracle-copyrighted API.
    - Annatar 10y ago
      What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.
      - qwertyuiop924 10y ago
        Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
        Annatar 10y ago
        What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
        qwertyuiop924 10y ago
        I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
qwertyuiop924 10y ago
Thanks. That looks very exciting, and it's a relief to know that the eBPF team has thought about these things, and are actually doing work on them.

Roxxik 10y ago

It shouldn't be that difficult to write some simple compiler to translate a scripty language that looks like awk to BPF-Assembly (or C and stuff that into LLVM). I might look into this stuff for my Bachelor Thesis ;)

bonzini 10y ago

Yes, people are working on an eBPF backend for systemtap.

qwertyuiop924 10y ago
Well, that will at least stop systemtap bringing your system to a crashing halt (literally). Although I believe they may have fixed that already. But what would you expect for software coming out of Red Hat? Their projects are usually a mixed bag at best.

vavrusa 10y ago

It wasn't mentioned in the article, but I've recently merged LuaJIT to BPF compiler. So you just write Lua and the kernel bits get compiled into BPF bytecode and loaded. No C. See https://github.com/iovisor/bcc/blob/master/src/lua/README.md#luajit-bpf-compiler https://github.com/iovisor/bcc/blob/master/src/lua/README.md... or examples https://github.com/iovisor/bcc/blob/master/examples/lua/tracepoint-offcputime.lua https://github.com/iovisor/bcc/blob/master/examples/lua/trac... (this one is for tracing).

qwertyuiop924 10y ago
Okay, that's really cool. Thanks. It does need more documentation, but this is an excellent step. It also allows for building higher level APIs atop BCC's low-level lua integration, as opposed to writing yet another DSL.