5 ms·

She mentioned it, just not explicitly. BPF is the super efficient optimized compiler that tptacek tweeted about.

by dfc 11y ago

jsnell 11y ago
No, BPF is the extremely simple and limited virtual machine. The optimized compiler is in libpcap and does some neat stuff to eliminate redundant code. There are implementations of the pcap filtering language that do not compile to BPF, and there are other languages compiling to BPF which do not benefit from the compiler in libpcap. It's worth noting that optimizer is also absolutely necessary since the design of the pcap language is such that very simple filters are easy, anything even remotely complicated becomes very verbose and repetitive.
- tptacek 11y ago
  I'm pretty sure the three things (BPF, tcpdump, and the pcap compiler) were developed in tandem, or, at least, the latter two were. I had a job, about 15 years ago, hacking on a customized version of libpcap (mostly to do filter merges). There is a surprising amount of stuff going on there.
  - jsnell 11y ago
    Indeed. There's a great talk about those early days at https://www.youtube.com/watch?v=XHlqIqPvKw8 https://www.youtube.com/watch?v=XHlqIqPvKw8 ; I'd recommend anyone who is interested in the design and implementation of little languages to watch that. Few things will make a language designer throw out everything and restart from scratch, but I guess having Van Jacobson dismiss the initial design as unusable would do it! But after that common starting point they quickly developed lives of their own, and should not be treated as a single unit. You'll get things like pflua as a completely distinct implementation of the filtering language compiling to lua instead of BPF, pfmatch with language extensions that are not really compilable to BPF, seccomp and other uses of BPF in the Linux kernel for things that have nothing at all to do with packet processing, a (e)BPF code generator backend in LLVM, and so on.
    - tptacek 11y ago
      BPF and libpcap are certainly separable, but tcpdump is practically just the test driver program for libpcap. :)
- cthalupa 11y ago
  > BPF is the extremely simple and limited virtual machine No longer true in modern Linux. http://www.brendangregg.com/blog/index.html http://www.brendangregg.com/blog/index.html
  - Dylan16807 11y ago
    Yes it is. Going from 2 to 10 registers and getting map data structures and compilation to machine code doesn't change that BPF is intentionally simple and even disallows loops. "simple and limited" is a good thing.
    - cthalupa 11y ago
      It's simple, sure. And it's limited, in that there are sane limits in place, because you don't want an application running in the kernel to loop for forever, but 'extremely simple and limited' I don't think is fair. You can do a hell of a lot with BPF, and it's not like work to extend it's functionality is slowing down, either. We've had tons of features implemented in the 4.x family, stack walking is likely to be implemented in the future, etc. When you have people like Alexei Starovoitov and Brendan Gregg saying it can do all sorts of things including "crazy stuff", I think we've moved beyond 'extremely simple and limited'
      - Dylan16807 11y ago
        You're reading a negativity into that phrase that isn't really there. It's harshly limited, but in a careful way. Despite having few parts and not being turing complete it has a massive number of use cases.

She mentioned it, just not explicitly. BPF is the super efficient optimized compiler that tptacek tweeted about.

by dfc 11y ago

She mentioned it, just not explicitly. BPF is the super efficient optimized compiler that tptacek tweeted about.

No, BPF is the extremely simple and limited virtual machine. The optimized compiler is in libpcap and does some neat stuff to eliminate redundant code. There are implementations of the pcap filtering language that do not compile to BPF, and there are other languages compiling to BPF which do not benefit from the compiler in libpcap. It's worth noting that optimizer is also absolutely necessary since the design of the pcap language is such that very simple filters are easy, anything even remotely complicated becomes very verbose and repetitive.

tptacek 11y ago
I'm pretty sure the three things (BPF, tcpdump, and the pcap compiler) were developed in tandem, or, at least, the latter two were. I had a job, about 15 years ago, hacking on a customized version of libpcap (mostly to do filter merges). There is a surprising amount of stuff going on there.
- jsnell 11y ago
  Indeed. There's a great talk about those early days at https://www.youtube.com/watch?v=XHlqIqPvKw8 https://www.youtube.com/watch?v=XHlqIqPvKw8 ; I'd recommend anyone who is interested in the design and implementation of little languages to watch that. Few things will make a language designer throw out everything and restart from scratch, but I guess having Van Jacobson dismiss the initial design as unusable would do it! But after that common starting point they quickly developed lives of their own, and should not be treated as a single unit. You'll get things like pflua as a completely distinct implementation of the filtering language compiling to lua instead of BPF, pfmatch with language extensions that are not really compilable to BPF, seccomp and other uses of BPF in the Linux kernel for things that have nothing at all to do with packet processing, a (e)BPF code generator backend in LLVM, and so on.
  - tptacek 11y ago
    BPF and libpcap are certainly separable, but tcpdump is practically just the test driver program for libpcap. :)
cthalupa 11y ago
> BPF is the extremely simple and limited virtual machine No longer true in modern Linux. http://www.brendangregg.com/blog/index.html http://www.brendangregg.com/blog/index.html
- Dylan16807 11y ago
  Yes it is. Going from 2 to 10 registers and getting map data structures and compilation to machine code doesn't change that BPF is intentionally simple and even disallows loops. "simple and limited" is a good thing.
  - cthalupa 11y ago
    It's simple, sure. And it's limited, in that there are sane limits in place, because you don't want an application running in the kernel to loop for forever, but 'extremely simple and limited' I don't think is fair. You can do a hell of a lot with BPF, and it's not like work to extend it's functionality is slowing down, either. We've had tons of features implemented in the 4.x family, stack walking is likely to be implemented in the future, etc. When you have people like Alexei Starovoitov and Brendan Gregg saying it can do all sorts of things including "crazy stuff", I think we've moved beyond 'extremely simple and limited'
    - Dylan16807 11y ago
      You're reading a negativity into that phrase that isn't really there. It's harshly limited, but in a careful way. Despite having few parts and not being turing complete it has a massive number of use cases.