4 ms·
The model I'm describing contains two pieces: 1) Moving away from sidecars to per-node proxies that can be better integrated into the Linux kernel concept of na
by tgraf 5y ago
The model I'm describing contains two pieces:
1) Moving away from sidecars to per-node proxies that can be better integrated into the Linux kernel concept of namespacing instead of artificially injecting them with complicated iptables redirection logic at the network level.
2) Providing the HTTP awareness directly with eBPF using eBPF-based protocol parsers. The parser itself is written in eBPF which has a ton of security benefits because it runs in a sandboxed environment.
We are doing both. Aspect 2) is currently done for HTTP visibility and we will be working on connection splicing and HTTP header mutation going forward.
- tptacek 5y agoWhat does an HTTP parser written in BPF look like? Bounded loops only --- meaning no string libraries --- seems like a hell of a constraint there.
- tgraf 5y agoIt looks not too different from the majority of HTTP parsers out there written in C. Here is an example of NodeJS [0]. [0] https://github.com/nodejs/http-parser/blob/main/http_parser.c https://github.com/nodejs/http-parser/blob/main/http_parser....
- tptacek 5y agoNode's HTTP parser doesn't have to placate the BPF verifier, is why I'm asking.
- Matthias247 5y agoanother challenges I can see is wheee to actually store the state of a connection. Even if we just focus on http/1.1 then not all headers will be received at one, and data from previous segments needs to be carried forward. Would it be eBPF maps? Those also seem rather limited for this usecase, and are probably also not extremely fast. I can imagine getting something to work for http/1.1 - but http/2 with multiplexing and stateful header compression is a completely different beast.
- justicezyx 5y agoBounded loop plus 1M instruction limits in the 5.4 kernel (no record at hands about the exact version), gives a large range of supported headers. Also note that these BPF code are on the network level, which is subject to the MTU limit as well, which usually is 1500 and now can be 10s of KBs (65,525 bytes maxmial in theory accroding to https://www.lifewire.com/definition-of-mtu-817948 https://www.lifewire.com/definition-of-mtu-817948, but my networking knownledge is poor). These makes it possible to effectively handle all possible headers. HTTP is actually fine. HTTP2 will be a bigger issue as it has HPACK, and Huffman coding, that would be very complicated to maintain inside BPF runtime. I haven't thought about it closely yet. But based on our experience at http://px.dev http://px.dev, I am not aware of any glaring technical obstacles.
- tptacek 5y agoThis is interesting and all, but I've also written bounded loop BPF code on 5.6 kernels, and it is not easy to get the verifier to accept seemingly obvious loops. I'm not saying it's impossible, I'm saying I'd like to see what this code actually looks like. I'd be a little shocked if it just looked exactly like Node's HTTP parser.
- justicezyx 5y agoI need to double verify when was the bounded loop patch got into the kernel, I suppose it's 5.6 as you mentioned above. What I actually was thinking is that one can write C code and ask the compiler to unroll it. ``` pragma(unroll) for (..., i < 100; ++I) { parsing code } ``` Also the other comment note the stake bookkeeping for HTTP to maintain the state when the parsing spans multiple packets, assuming here we are talking about XDP probes. One quick idea is to use BPF_TABLE(, uint128_t, some data structure) I haven't tested if uint128_t is OK as key type. And the data structure in the value needs more thoughts. Roughly I am thinking turn any state bookkeeping into some BPF tables, and keyed through whatever data that matches the context. This probably means uint128_t as Ipve/6 address, and a nested map with key as the port. Or combined v4 IP & port. It'll be interesting. I suppose the code from Isovalent will eventually be open sourced. Or is it already so? Haven't checked yet.