2 ms·

I am surprised ebpf wasn't mentioned. it's made for exactly that and you get bonus features like lite weight JIT and safety given that it supports a strict subs

I am surprised ebpf wasn't mentioned. it's made for exactly that and you get bonus features like lite weight JIT and safety given that it supports a strict subset of asm commands that are deemed safe. The arguments against JIT aren't backed by any prototypes or proofs. it felt like the author has a bias towards dynamic dispatch and walked backwards towards it. There is no mention how frequently these filters change nor the possible perf advantage that you'd get from running compiled expressions that would compensate for the accumulated compile time of these filters over time. Often you need 2 approaches together and you switch from interpretation to JIT after a condition. (in db case after the same query shows up multiple times for example). In their case it can be something as simple as if filter hasn't updated in x time then consider it stable and compile it. On the other hand, PGs dynamic dispatch is a fun read just like any other code in PGs repo. https://github.com/postgres/postgres/blob/master/src/backend/executor/execExprInterp.c https://github.com/postgres/postgres/blob/master/src/backend...

buro9 8y ago
In the other blog post (linked from the top of this blog post) I wrote about the motivation for using the Wireshark-like syntax and how we also have a few customers with a lot of rules (tens of thousands and greater). That said, it seems obvious to us that now we have a Rust library that can parse a Wireshark-like syntax into an AST... that we don't have to just perform the matching in Rust. i.e. that we can ask the library to produce translations of the expression as SQL (for our ClickHouse), GraphQL (for our analytics API), or even eBPF. We can't run everything in eBPF, but we could check the list of the fields within an expression to see whether it could be run in eBPF, and then look at heavy hitter rules and promote the ones doing the most work inside L7 to be eBPF and to run in XDP. Even if we don't do this for customer configured rules, this might be something we do for handling denial of service attacks using the same Wireshark-like expression syntax throughout our system.
- halayli 8y ago
  Got it. Just to clarify, I didn't mean to implement the filters using ebpf inside the kernel but rather use ebpf's engine as a lib at the application layer.

buro9 8y ago

In the other blog post (linked from the top of this blog post) I wrote about the motivation for using the Wireshark-like syntax and how we also have a few customers with a lot of rules (tens of thousands and greater). That said, it seems obvious to us that now we have a Rust library that can parse a Wireshark-like syntax into an AST... that we don't have to just perform the matching in Rust. i.e. that we can ask the library to produce translations of the expression as SQL (for our ClickHouse), GraphQL (for our analytics API), or even eBPF. We can't run everything in eBPF, but we could check the list of the fields within an expression to see whether it could be run in eBPF, and then look at heavy hitter rules and promote the ones doing the most work inside L7 to be eBPF and to run in XDP. Even if we don't do this for customer configured rules, this might be something we do for handling denial of service attacks using the same Wireshark-like expression syntax throughout our system.

halayli 8y ago
Got it. Just to clarify, I didn't mean to implement the filters using ebpf inside the kernel but rather use ebpf's engine as a lib at the application layer.