9 ms·
eBPF Documentary
- deleted 3y ago[deleted]
- bch 3y agoI was a little disappointed DTrace[0] was not mentioned at all. The instrumentation (not the SDN) isn’t novel, not even to Linux (DTrace is available on Linux - I understand licensing is at least questionable (for some distros), but that aside…). [1] [0] https://en.wikipedia.org/wiki/DTrace https://en.wikipedia.org/wiki/DTrace [1] https://docs.oracle.com/en/operating-systems/oracle-linux/dtrace-tutorial/dtrace-tutorial-IntroducingDTrace.html https://docs.oracle.com/en/operating-systems/oracle-linux/dt...
- kragen 3y agoi feel like the fact that the page mentions brendan gregg is a pretty strong reference to dtrace already. actually it doesn't just mention him, it's completely written by him
- bch 3y agoI think people that know, know he was involved (in the userland aspect) of DTrace, but I get a sense for some reason there’s no love lost between Brendan and “DTrace”.
- ta8645 3y agoDTrace was not the inspiration for eBPF at all, so it's not obvious it is relevant to mention. As the documentary mentions, the initial impetus for eBPF, was software defined networking. eBPF is a much bigger and more comprehensive infrastructure piece (networking, tracing, security, etc) than DTrace. And thanks to licensing issues, even within the limited domain of tracing, DTrace will likely become a footnote in history, while eBPF becomes available on every major OS platform.
- bcantrill 3y agoThere aren't "licensing issues" with DTrace -- you are merely referring to the fact that it is licensed under the MPL-derived CDDL and not the GPL. But it is definitely true that they are not seeking to solve the same problems! Safety is very core to DTrace[0]; the difference here is entirely deliberate. [0] https://bcantrill.dtrace.org/2005/07/19/dtrace-safety/ https://bcantrill.dtrace.org/2005/07/19/dtrace-safety/
- ta8645 3y agoAFAIU, it goes beyond safety; eBPF is designed to run entirely in the kernel rather than requiring a user-space bridge. So it's able to operate at line speeds in networking scenarios etc. > you are merely referring to the fact that it is licensed under the MPL-derived CDDL and not the GPL. Not really. I'm referring to the fact the eBPF will be available across Microsoft, Apple, and Linux, and there is no other technology that will be able to offer that. > not seeking to solve the same problems! Exactly. eBPF has a much broader and more significant problem set, not just tracing, but also security modules, software defined networking, and an almost unlimited future potential as well.
- bcantrill 3y agoA few points of clarification: - DTrace does execute entirely in the kernel. Indeed, anonymous tracing is instrumenting the system without any corresponding user process whatsoever - DTrace exists on quite a few systems, including (with caveats) Windows, MacOS and Linux. - The unbounded nature of eBPF very much runs contrary to the safety that DTrace assures; DTrace isn't seeking to augment the system, but merely to understand it
- tdullien 3y agoeBPF has only recently become less bounded, at 4.15 for example it was still very bounded ;). (I assume you mean maximum # of instructions etc?)
- bch 3y ago
- deleted 3y ago[deleted]
- monocasa 3y agoAFAICT DTrace is more a parallel branch. Both inspired by cBPF. What should get more talk IMO, is the exokernel XOK's kernel VMs which went way harder than even eBPF does towards user space programmability for the kernel as a core primitive. For instance instead of sleep(2) or futex(2) calls, XOK exposed "wake programs" that user space would register for the scheduler to run to answer "is this blocked thread runnable again". Would have solved the collabra's need to change futex(2) to work more nicely with wine/windows primitives in a more general way.
- xrd 3y agoI've been hearing more and more about eBPF, especially here on HN. I haven't yet watched the documentary so perhaps it is answered there. But, the analogy of JavaScript inside the kernel is great and I'm left wondering: what was the way to do it previously? Userland network tool? This standardizes on a interface to the kernel, not a language, right? Feels off to say it is JavaScript because that comes with a lot of baggage, but also (as a versatile and ubiquitous language) incredibly powerful and useful tool. Is that intentional by the author?
- lmm 3y ago> But, the analogy of JavaScript inside the kernel is great and I'm left wondering: what was the way to do it previously? Userland network tool? This standardizes on a interface to the kernel, not a language, right? Guess/sketch: It's a language in most senses. Previously the kernel had APIs for packet filtering rules for iptables etc., but the set of rules you could use was somewhat "static" - rules would have parameters, so you could do things like if the source IP is in this range then rewrite it as this and direct it to this interface, but it was kind of like one of those visual flowchart languages where you can drag and drop the available boxes in a given order, but if there isn't a box to do what you want then you're stuck. Whereas with eBPF it really is scriptable - rather than a specific rule type you can just submit the script you want it to run - and nowadays it's become kind of a general kernel scripting language rather than just for networking. I'd draw a parallel with how 3D graphics programming has shifted from "you can do these kinds of transformations, submit a list of what you want to run in what order" to "this is our shader programming language, just write whatever you want to do as a program in this language".
- bhickey 3y agoiptables is definitely limited in comparison to eBPF, but that isn't the innovative step. BPF was around for more than twenty years before eBPF came around. Around 2013 I worked on a packet analysis pipeline that generated BPF code dynamically at runtime. eBPF isn't more scriptable than BPF in this sense. The language does add some opcodes and loops that weren't available in the original, but this is relatively modest. The real genius of what these folks did was extending the usefulness of BPF beyond the network stack. Without a provably safe language it would've been impossible to enable flexible kernel tracing.
- James_K 3y agoIt has always seemed quite obvious to me that dealing in machine code is a flawed approach for distributing software. At the most basic level, it entails giving someone else near unfettered access to the hardware of your computer and simply hoping that they do nothing malicious or malformed. Yet the software world as a whole seems continually shocked at the idea of using anything else. Perhaps someday we will learn this lesson in its entirety and begin to share code rather than blobs.
- yjftsjthsd-h 3y agoI don't think machine code is the problem? Running a native binary doesn't give it access to anything inherently; the OS gives access through syscalls, and can impose restrictions - and indeed, does; it's not like running a binary on Linux automatically gives it access to anything under /dev. Also, it's not that the whole software world thinks blob-only software is normal; I'm typing this on a nice comfy GNU/Linux box where the only blobs are some firmware. (Edit: And to be quite clear, a good chunk of this community would really like to get rid of those blobs too, it's just that we don't have a fix at this point.)
- James_K 3y ago> Running a native binary doesn't give it access to anything inherently While this is true from a certain perspective, machine code creates a system which must grand access to many things to become usable. A shared file system is a good example of this. Some software could easily echo a line into you .profile that tries to launch a key-logger, and this works in many cases. The expectation of software existing as opaque files creates a huge amount of work for the OS in verifying the exact behaviour of the software as it runs (and in ways which can often be circumvented), rather than a source-based approach in which malware is never allowed to touch the processor. > I'm typing this on a nice comfy GNU/Linux box where the only blobs are some firmware So you suffer the worst of both worlds then. You've had to download and compile the source yourself, but as the software is designed around being distributed as blobs, so you enjoy none of the benefits that might come from source distribution.
- 3y ago
- softirq 3y agoThe genius of ebpf is allowing for pluggable policy in a world where the kernel API is very slow to change and can’t meet everyone’s needs. Whether it’s how the kernel handles packets off the wire, how it controls traffic, scheduling entities, or instrumentation, ebpf lets you provide logic rather than turn a bunch of knobs or use a bespoke syscall that only handles one case. It also moves the processing logic to the data in the kernel rather than having the kernel have to do expensive copies to and from userspace. ebpf isn’t really novel beyond the interfaces it provides. They are just kernel modules that have been vetted and are sandboxed. Inserting executable code has been part of the kernel since forever in module form and kprobes.
- yjftsjthsd-h 3y ago> the kernel API is very slow to change and can’t meet everyone’s needs Better yet - eBPF provides a stable ABI:) It makes things that were formerly kernel-internal possible to work with from a stable ~userspace interface.
- dilyevsky 3y agobpf tooling generally provides no stability guarantees when you interact with kernel primitives. See [0], for example. Tho things have improved somewhat with CO-RE [0] - https://lore.kernel.org/lkml/93a20759600c05b6d9e4359a1517c88e06b44834.camel@fb.com/ https://lore.kernel.org/lkml/93a20759600c05b6d9e4359a1517c88...
- stefan_ 3y agoI'm curious what this guarantee includes - the bytecode? Because the actual in-kernel eBPF API is famously unstable, with eBPF-based applications usually requiring a cutting-edge kernel version (for industry anyway). And of course the eBPF programs themselves rely on accessing structures for which no stability guarantees are made whatsoever.
- bidandanswer 3y ago> ebpf isn’t really novel beyond the interfaces it provides. They are just kernel modules that have been vetted and are sandboxed. Inserting executable code has been part of the kernel since forever in module form and kprobes. This should be sung from the mountaintops. This concisely summarizes nearly everything that uninformed reader should take away from the comment section.
- cyberax 3y ago"eBPF Documentary: An exciting train wreck in progress" That would be a better title. eBPF started as a small extension to just be able to insert small trivial hooks. It's now basically a hacked-up broken WebAssembly clone, with zero forethought put into it. NIH syndrome at its worst. It has recently grown unlimited loops with runtime metering, making the static verifier basically a worthless complexity. Before that, it had acquired exceptions and stack unwinding.
- akira2501 3y agoArchitecture independent eBPF "rescue binaries" could be an interesting part of a distribution toolset.. or attack rootkit. It's hard to have nice things.
- pjmlp 3y agoWhile I agree its design isn't the best, versus something like Solaris DTrace, it predates WebAssembly, so hardly a clone.
- cyberax 3y agoeBPF itself is indeed ancient. However, the explosion of NIH-ed functionality is not. Most of it happened after the standardization of WASM.
- pjmlp 3y agoBytecode based execution goes back to the 1960's, yet another thing that WebAssembly advocates tend to forget when writing comments and blog posts about its greatness.
- cyberax 3y agoBytecode interpreters/JITs are indeed ancient. However, JITs designed to safely confine malicious bytecode are most definitely not. Java or .NET VMs are probably the earliest examples, and the WASM infrastructure is using experience gained there. My problem with eBPF is that it just ignored all of it, and proceeded to feature-creep an initially "simple" solution into a monster. The initial justification for the NIH stuff was: "but we need those static guarantees provided by the verifier, nothing else can do that". But of course, these justifications were forgotten as soon as the verifier became the limiting factor.
- brcmthrowaway 3y agoBrendan Gregg will get a Turing Award for this, congrats!
- bidandanswer 3y agoA Turing Award for avoiding a context switch? I mean, I love eBPF more than most, but this is a practical engineering solution to a logistical problem that didn't really need to exist in the first place. This is not genius and not an order-of-magnitude improvement to an important computer science problem; it's an improvement to a costly artifact of the Linux kernel.
- spaceywilly 3y agoAs someone who works with BPF every day, I don’t think comparing orders of magnitude is right. It just enables things that can be done now that you wouldn’t dream of doing before. I think a good analogy is going from horse drawn carriages to the automobile. In a carriage going from Boston to New York would be a multi day journey that you would have to plan for weeks in advance. With a car you just get in and go. BPF is like that, suddenly you have all these options for things that you can do instantly, that you’d only do before if you were ready to dedicate years to getting support into the kernel. It’s a superpower for kernel development.
- witx 3y agoebpf seems to be a very interesting idea and have been experimenting with it. Still I find it weird that we're doing documentaries on software "frameworks"
- a2800276 3y ago"Documentary" may be a bit of a strong word. It's more of an image film / ad for Cilium and/or the ebpf foundation.
- bingleflip 3y agoIt makes me wonder, what was the point of introducing Rust into the kernel when they already have eBPF? Couldn't they have just used the existing eBPF compiler for creating safe kernel code? Maybe the distant future of the Linux kernel is for almost everything to compile down to eBPF bytecode except the part that runs the bytecode.
- bidandanswer 3y agoThat's a cool conjecture but it kind of misses the point of eBPF. The point of it is that you can run user-defined programs while avoiding the costly context switch between user space and kernel space. The kernel already is the kernel. Compiling kernel code to eBPF programs would offer seemingly no performance gains, since you're already in kernel space; there is no costly context switch to avoid.
- bidandanswer 3y agoeBPF allows user-defined programs to run in the kernel. This is huge for performance-sensitive code that executes against network packets: you don't have to context switch between kernel space and user space. It's worth pointing out Solana's extreme competitive advantage over other chains is almost entirely due to it running on a variant of eBPF. † This is an order-of-magnitude leap over other implementations and essentially the way you should do it, if you were to write it from scratch, aside from special purpose hardware fabrication. † The second reason Solana is so fast is extreme parallelism: all accounts that are used in a transaction must be marked as either "read-only" or "writeable" before sending the transaction, allowing the runtime to parallelize all reads and only solve write contention when necessary.
- suprjami 3y agoThis comment seems like half GPT nonsense and half unsubtle advertisement for whatever Solana is.
- phendrenad2 3y agoeBPF modules can be closed-source, right? I can see a future where things like ZFS are paid eBPF modules, or games ship with proprietary anti-cheat eBPF modules.
- tdullien 3y agoIn theory, yes. In practice pretty much all useful applications need to call kernel APIs that mandate the eBPF code being GPL.
- phendrenad2 3y agoYou can't call kernel APIs from eBPF without triggering the license, I suppose?
- lexarflash8g 3y agoI saw the airing of it at Kubecon -- also met the Finnish guy there in the video. I mistook him for another Finnish guy with blond hair. Didn't meet the Russian though.