5 ms·
A Beginner's Guide to eBPF
- wittjeff 4y agoThe entire first page begs the question -- "what is eBPF?"
- CoastalCoder 4y agoIt's actually answered (indirectly) on that page: "My report "What is eBPF?" and in-depth book "Learning eBPF" are both available for download [0] from Isovalent or with your subscription to O'Reilly's learning platform. You can buy "Learning eBPF" from any good bookstore (support your local bookshop by ordering it there!)" IIUC, you need to give contact info on that page to get the PDF. So this [1] might be a better starting point. [0] https://isovalent.com/ebpf/ https://isovalent.com/ebpf/ [1] https://ebpf.io/ https://ebpf.io/
- Dylan16807 4y agoIs a joke going over my head here, because the first sentence links to a document titled "what is eBPF?"
- yawnxyz 4y agoThe joke is you have to give them all your information in a form to then download some B2B report... rather than add two sentences about what ePBF is
- autoexec 4y agoSeriously. I really don't know why people think that's acceptable. Who sees a website titled "A beginner's guide to X" that makes people hand over their data to even get "X" defined, and thinks "This looks good to me! Sign me up!"
- tptacek 4y agoThis is a Github repository accompanying a book. Neither of those things were submitted here by their author. A real problem HN has sometimes is treating the entire Internet as if it was was written for HN specifically.
- tptacek 4y agoYou can just type "What is eBPF" into Google, and the first hit will decisively answer this question for you.
- aptwebapps 4y agoAlthough the fact that Google puts the Wikipedia BPF article in the sidebar might confuse you a little.
- cinntaile 4y agoYou can work around pretty much any issue like this, but you shouldn't have to. A beginner's guide should define the concept.
- CTDOCodebases 4y agoI find this same issue with many things posted here. Even businesses marketing their product. My favourite is when they give you an abstraction salad instead of explaining something. I clicked the first link so I got the gist but how many other people just give up and disengage with their post? Or click the link then just close the tab without reading further.
- noduerme 4y agoAfter looking at the link first, this comment cracked me up. I skimmed most of the GH content without figuring out the answer; I wasn't going to click an O'Reilly link. So I gave up. Then my OCD kicked in and I got annoyed with myself for giving up, since if I'm going to waste time on HN I should really get to the bottom of things. So searched it. I found this: https://ebpf.io/what-is-ebpf/ https://ebpf.io/what-is-ebpf/ It runs sandboxed kernel extensions? Or it's a VM? Something like that? The writing of the page itself doesn't inspire a lot of confidence, but then maybe it's just over my head.
- Dylan16807 4y ago> It runs sandboxed kernel extensions? Or it's a VM? Yes to both of those. The kernel has a bunch of extension points that can run eBPF code in a VM. That code can make decisions for the kernel and/or track events. eBPF code can do basically any calculation you want, but it can't have infinite loops. It's loaded as bytecode, with a spec for how it's formatted and what the instructions do and what data structures are built in to the VM. The main benefit is that it runs in the kernel, so it can be triggered very very often with minimal performance impact.
- johnea 4y agoLooks like a "buy my book" ad to me...
- tptacek 4y agoHey. Please be more careful. The author of this repository isn't the story submitter. People write things that end up on Hacker News all the time with no idea that it's even happened, let alone any intention to promote things to you.
- deleted 4y ago[deleted]
- JohnDeHope 4y agoThis is not eBNF. That is something completely different.
- zquestz 4y agohttps://ebpf.io/what-is-ebpf/ https://ebpf.io/what-is-ebpf/ is quite useful.
- zero-dark 4y agoThe GitHub page linked here mentions the word “eBPF” 18 times and yet not once does it expand on what this abbreviation means.
- phillipcarter 4y agoIt's extremely berkeley packet fun.
- Palomides 4y agowhile originally it's "extended Berkeley Packet Filter", at this point it does so much more than packet filtering that spelling out the words probably increases confusion rather than clarifying anything it's like a reverse backronym
- Paul-Craft 4y agoThis may prove useful: > eBPF (often aliased BPF)[2][5] is a technology that can run sandboxed programs in a privileged context such as the operating system kernel.[6] It is used to safely and efficiently extend the capabilities of the kernel at runtime without requiring to change kernel source code or load kernel modules.[7] Safety is provided through an in-kernel verifier which performs static code analysis and rejects programs which crash, hang or otherwise interfere with the kernel negatively.[8][9] Examples of programs that are automatically rejected are programs without strong exit guarantees (i.e. for/while loops without exit conditions) and programs dereferencing pointers without safety-checks.[10] Loaded programs which passed the verifier are either interpreted or in-kernel JIT compiled for native execution performance. The execution model is event-driven and with few exceptions run-to-completion,[2] meaning, programs can be attached to various hook points in the operating system kernel and are run upon triggering of an event. eBPF use cases include (but are not limited to) networking such as XDP, tracing and security subsystems.[6] Given eBPF's efficiency and flexibility opened up new possibilities to solve production issues, Brendan Gregg famously coined eBPF as "superpowers for Linux".[11] Linus Torvalds expressed that "BPF has actually been really useful, and the real power of it is how it allows people to do specialized code that isn't enabled until asked for".[12] Due to its success in Linux, the eBPF runtime has been ported to other operating systems such as Windows.[4] https://en.wikipedia.org/wiki/EBPF https://en.wikipedia.org/wiki/EBPF
- awesomegoat_com 4y agoHaven't sandboxed programs in a privileged context been the root cause of me seeing BSOD so often in the late 90ties?
- upghost 4y agoSomething’s off here. I’m reasonably well read and literate on computer topics, I’ve worked in cyber security for over 5 years now, and extremely open-minded to new ideas — this reads at best like derivative marketing jargon and little in the way of technical.
- tptacek 4y agoeBPF is an extremely big deal in computer and network security, so, I assure you, this isn't "derivative marketing jargon", and it is very technical.
- upghost 4y agoWell that is wild and kinda cool. I don’t often find things that are so foreign to me they appear fake! I’ll have to poke around a little more, thanks for the correction.
- cookiengineer 4y agoThe game changing idea behind eBPF is XDP in my opinion. Lots of network drivers and NICs support offloading XDP programs ("xdp_prog") to the network controller's chipset, which results in zero CPU i/o interrupts if you e.g. use an XDP_DROP to block traffic. Being able to block network traffic _before_ it reaches even kernelspace is a game changer.
- PhilipRoman 4y agoI've found kernel documentation under Documentation/bpf to be the best resource available. Clear, concise and no marketing-speak As for this repository - the README triggered a false alarm in my bullshit sensors, but the code example is pretty nice.
- _def 4y agowelcome to cloud engineering.
- mooreds 4y agoIf you are into podcasts, she was on "Screaming in the Cloud" recently, and talks about what eBPF enables, how to teach technologies and more. https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/learning-ebpf-with-liz-rice/ https://www.lastweekinaws.com/podcast/screaming-in-the-cloud...
- DethNinja 4y agoI’m new to Linux kernel programming & eBPF (just started last week) and I’m having major troubles with eBPF verifier. I honestly feel like it would be easier for me to write a kernel module than eBPF code. I do wonder if this is the case for many people. It seems verifier is a bit unpredictable and makes eBPF programming quite painful.
- tptacek 4y agoIt would be much easier to write a kernel module than an eBPF program. But the eBPF program is unlikely to panic your machine, and the kernel module is almost certain to.
- israrkhan 4y agoAlso In cloud environments, I would much rather trust an eBPF program, than a kernel module.
- anonymousDan 4y agoWhy in cloud environments?
- Temporary_31337 4y agoYou don’t want kernel panic affecting other users
- cassianoleal 4y agoWhy would your cloud instance panicking affect other users of the cloud provider? Or do you mean something else?
- bionsystem 4y agoAnecdotically, I had the case on VMWare 4 (that was in 2012 or 2013) that a Solaris 11 VM managed to reboot the entire ESX it was hosted on. Very weird bug where ESX passed through some interrupt or something. But in this case I think they mean on the same machine. "In production" would be more accurate than "in a cloud environment". And yeah I wouldn't load custom kernel modules in production just to do observability.
- tomgs 4y agoIf you’re looking for a slightly higher level look, Unzip does it well: http://unzip.dev/0x00c-ebpf/ http://unzip.dev/0x00c-ebpf/
- funerr 4y agoThanks for the shout-out!<3
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- kqr 4y agoEver since Brendan Gregg started using eBPF for observability back in 2015 I've had the sense that eBPF is an extremely underrated tool of the future. I really would like to learn it, but beyond some improvised bpftrace scripting and the tools that come with bcc, I've not really had the need. What custom usage do you have for it?
- blipvert 4y agoI’ve used it to write a layer 4 load balancer to replace old hardware appliances. Similar to Facebook’s Katran, but switches traffic at layer 2 (rather than L3) for compatibility and in Go rather than C++,’cos I’m not a great coder ;-)
- anonymousDan 4y agoAs I understand it, eBPF is primarily an observation tool and thus is quite limited in the modifications it can make to kernel memory. Does it have any generic way to make arbitrary modifications to kernel memory? Obviously this would invalidate any verification guarantees, but I would expect this to be very minor modifications in practice. For example, if I wanted to hook a page fault handler to change the behaviour without having to pay the cost of signal handlers.
- deleted 4y ago[deleted]
- madog 4y agoWhat you can do is quite limited. You're restricted to some preset eBPF program types, and each program type has a restricted set of operations it can perform (eBPF helper methods). So arbitrary modifications, absolutely not without adding a helper method and/or program type for this purpose. More program types and helper methods are being added all the time but overall it's pretty limited in use cases and operations. If you want full control then kernel module is the way to go, but this doesn't have the same security and stability guarantees.
- anonymousDan 4y agoRight, that's what I thought. So at a high level helper methods are equivalent in some sense to 'unsafe' code in rust and require manual validation for security (i.e. the verifier ignores them other than to check they are in some helper method whitelist for the program type)?
- tptacek 4y agoeBPF has pretty significant latitude with network traffic, but for everything else the idiom is to use it to do fast kernel telemetry to a userland process that does the actual acting.
- kjuulh 4y agoJust read the book, I like the way she did it, it starts out easy with what eBPF is, how it works, using short examples for both the low-level under the hood stuff, but also how to actually use it. It doesn't deviate much from this pattern, which is a good thing for such a relatively short book, and she gets to cover a lot of ground that way. It is still a "beginners" eBPF book, but sets the stage for further development through the references or alternative books. I can highly recommend it if you're eBPF curious =D I guess my only gribe is that the latter parts of the book, is a bit product heavy, but done in the most tasteful manner it could probably have been done, i.e. using it as examples on how to use eBPF.
- ilitirit 4y agoHer implementation of a rudimentary load balancer (in C) demonstrates eBPF in practice: https://www.youtube.com/watch?v=L3_AOFSNKK8 https://www.youtube.com/watch?v=L3_AOFSNKK8
- jcwayne 4y agoThe parody here is so perfect, they even created the technology they don't bother to define.
- witx 4y agoHow so?
- qubit0ne 4y agoIn case anyone is interested in writing eBPF programs in Rust https://github.com/vishpat/oxidize-ebpf https://github.com/vishpat/oxidize-ebpf
- jrpelkonen 4y agoI don’t have any experience with this project, but Aya seems more vibrant, https://aya-rs.dev/ https://aya-rs.dev/
- qubit0ne 4y agoAya is one of the most under-rated eBPF projects.
- philpearl1 4y agoYay!
- jooz 4y agocompared to making a module, apart the fact that the module can end in kernel panic (thats my problem), is there any significance performance difference between BPF and kernel modules ? If my goal is to reduce the cpu cloud bill, is BPF good enough compared to making a kernel module ?