4 ms·
Can someone elaborate on why he takes a weird detour in the middle of the post to discourage forks? Is there some particular issue with supporting bpf tooling f
by kortilla 5y ago
Can someone elaborate on why he takes a weird detour in the middle of the post to discourage forks? Is there some particular issue with supporting bpf tooling forks?
As someone who has spent a lot of time in open source, forks are not a problem, they are indicative of a problem.
Don’t cry when people fork and go a different direction, try to figure out why and see if you’re willing to change the project to accommodate them. Dropping chastising “more wood behind fewer arrows” platitudes is pointless when half of the wood wants to break off in a different direction anyway.
- kbenson 5y ago> As someone who has spent a lot of time in open source, forks are not a problem, they are indicative of a problem. Forks because the main project is not responding to a need are not a problem. Forks just because some project or product believes it gives them more control and they don't interface with the original project usefully are a sort of a problem. Those aren't necessarily started because of a community need, but some business need or perceived business need which may have nothing to do with the reality. I think that's the situation he's asking to avoid here, as he mentions observability products. He's saying please don't fork just to have it under your own repo and for no other reason, when it can be developed jointly.
- kortilla 5y agoWhy? If someone wants more control their interests already aren’t aligned with the community. Developer resources aren’t just some faucet easily redirected. It’s worse to have people trying to jam shit into the main project if they don’t actually care about the main project. That’s how you get contributions that are huge hacks and require more work to review and iterate on than is worth it to the community. If some company forks for an observability product and doesn’t contribute back, they clearly don’t want to. Don’t try to force them. git is an incredible tool for managing forks and this anti-fork mindset is right out of old school open source culture from the early 2000s. Demanding people contribute to your project instead of forking just sounds like demands to pay homage more than anything.
- kbenson 5y ago> That’s how you get contributions that are huge hacks and require more work to review and iterate on than is worth it to the community. And yet the author here is saying please work together. Presumably he or the community is okay with picking apart those huge hacks. I'm not sure why you should care what that community is willing to accept, unless you're part of it. > If some company forks for an observability product and doesn’t contribute back, they clearly don’t want to. Don’t try to force them. Who's forcing anyone? Did you not notice the "please" he starts that statement with? > Demanding people contribute to your project instead of forking just sounds like demands to pay homage more than anything. That's just a straw man, nobody demanded anything, and I'm not sure why you'd even try to insinuate he was when he literally says please. If I had to paraphrase the part of the post you're critiquing, I would do so as "If you're using these tools, don't feel like you have keep any stuff you do separate. We'd like to build something better for everyone, so feel free to contribute to the project rather than keeping it separate and we can all build on each other's work." That's pretty standard open source ideals IMO.
- brendangregg 5y agoI appreciate the discussion as I want to do a follow on post about this in more detail. It doesn't do the BPF community any good to have people running old broken versions of my tools (just as back in 2005, Sun was selling DTrace by distributing my half-finished socketsnoop.d). Now, some people are going to port anyway regardless of what's good for BPF. But if I can reach developers who may be on the fence about it, I can explain the pros of just building upon the existing tools. To start with, you get updates for free by kernel engineers at Facebook, Netflix, etc. Also, by pushing fixes back you'll find people will try to help you in return for when you need it (some startups, like Cilium, have done so much for BPF that I'd happily help them with anything they need (provided there isn't a clash with my current employment)).
- kortilla 5y ago> If I had to paraphrase the part of the post you're critiquing, I would do so as "If you're using these tools, don't feel like you have keep any stuff you do separate. We'd like to build something better for everyone, so feel free to contribute to the project rather than keeping it separate and we can all build on each other's work." That's pretty standard open source ideals IMO. Sure, that’s just a completely different writing of what’s actually there. The original post is an instruction not to do something. Your interpretation is a much more open “pull requests accepted”. His reply clarifies that he doesn’t want broken shit out there so your reading is incorrect. He does want to discourage forks because they are subtle to get correct and doesn’t want shit out their sullying bpf’s reputation.
- brabel 5y agoThe criticism is of forks that are done in order to re-brand and re-sell a project, and then never contribute anything back to the original, and likely, never pull improvements from the originally forked project (which can include critical bug fixes) after that.
- kortilla 5y agoSerious question, how does that hurt the original project? This has never been a concern in the projects I’ve worked on. Getting mad every time some developer writes code that didn’t go to your project is a good way to be mad all of the time.
- brendangregg 5y agoJust a couple of example scenarios: - Someone sells you BPF observability that's buggy and dissapointing, and you avoid BPF in the future. So in that case it's hurt the BPF community. - Funding goes to a project that gives nothing back, instead of funding a project that does. Again, it hurts the community as funding isn't infinite.
- kortilla 5y ago> Someone sells you BPF observability that's buggy and dissapointing, and you avoid BPF in the future. So in that case it's hurt the BPF community. Ok, so someone sells me closed source tooling that uses BPF and it’s buggy and disappointing, so I avoid BPF in the future. This association problem exists regardless of closed source vs a fork vs an old release. > Funding goes to a project that gives nothing back, instead of funding a project that does. Again, it hurts the community as funding isn't infinite. This only hurts if it’s a closed/hidden fork. This also presumes that the fork isn’t just stripping out tons of upstream features (I’ve had to do this for clients because “security”). My view is that forks of open source (GPL in particular) should be encouraged and done in public. Iterating on central projects only is just so slow and stifling. Maybe your projects have very little churn so lots of people experimenting in parallel doesn’t make sense, but this has been my experience with larger infrastructure projects. Let a thousand flowers bloom.
- brendangregg 5y agoYes, there can be good reasons to fork (especially after making a fair effort to have things fixed), and bad reasons. But also yes: There are two particular issues with bpf tooling forks. 1) They look deceptively simple, but those that are kprobes-based are really kernel-specific and brittle, and need ongoing maintenance to match the latest changes in the kernel. One ftrace(/kprobe) tool I wrote has already been ported a bunch of times, and I know it doesn't always work and one day I'll go fix it -- but how do I get all the ports updated? No one porting it has noticed it has a problem, and so the same problem is just getting duplicated and duplicated. Which is also issue 2) Unlike lots of other software, when observability tools become broken it may not be obvious at all! Imagine a tool prints a throughput that captures 90% of activity and no longer 100% (because there's now a fast-path taking 10%). So the numbers for some deep kernel activity are now off by 10%. It's hard to spot, and that increases the risk people keep deploying their old broken ports without realizing there's a problem.
- kortilla 5y ago> They look deceptively simple, but those that are kprobes-based are really kernel-specific and brittle, and need ongoing maintenance to match the latest changes in the kernel It seems like there is a missing formal interface here if this is so brittle, no? If it’s hitting a bunch of internal kernel stuff shouldn’t this stuff just live with the kernel itself?
- brendangregg 5y agoThe formal interface is tracepoints. So tracepoints in theory aren't brittle (they are best-effort stable) and don't need so much expert maintenance (which is mostly the case). In theory, someone could port tracepoint-based tools and almost never need maintenance. But kprobes is basically exposing raw kernel code that the kernel engineers bashed out with no idea that anyone might trace it. And they can change it from one minor release to another. And change it in unobvious ways: Add a new codepath somewhere that takes some of the traffic, so gee, seems like my tool still works but the numbers are a bit lower. Or maybe I measured queue latency and now there's two queues but the tool is only tracing the first one, or now there's no queues so my tools blows up as it can't find the functions to trace (that's actually preferable, since it's obvious that something needs fixing!). I really don't like using kprobes if it can be avoided (instead use tracepoints, /proc, netlink, etc). But sometimes it's solve the problem with kprobes or not at all. Now, normally such code-specific-brittle things should indeed live with the code like you say, so normally I'd think about putting the tools in the kernel code. But we don't want to add so much user space to the kernel, and, it also opens the door as to whether these should actually be tracepoints instead (which begins long discussions: Maintainers don't want to be on the hook to maintain stable tracepoints if they aren't totally needed). Another scenario where the tools should ship with the code base would be user space applications. E.g., if someone wrote a bunch of low-level tracing tools for the Cassandra database that used uprobes and were code specific, then they would be too niche for bcc, and would probably be best living in the Cassandra code base itself.