7 ms·
Bus Number – The GitHub plugin my coworkers asked me not to write
- philipwhiuk 2y agoFrom the original paper the key argument is: > Our estimation relies on a coverage assumption: a system will face serious delays or will be likely discontinued if its current set of authors covers less than 50% of the current set of files in the system An author of a file is defined as a user who has made significant (based on pre-calculated weights) contributions to a file
- stevage 2y agoThanks, this is the key info I was looking for here.
- shiroiushi 2y agoFrom TFA: >In 2015 or so, my employer had layoffs. One of them was the only contributor to part of the codebase that made money for our company. Maybe I missed it, but I didn't see the author mention what came of this. I'm very curious: did someone else take it over, or did the employer go down in flames?
- chuckadams 2y agoI imagine the employer limps on with the current codebase and/or is spending beaucoup bucks to rewrite the codebase while still using the old one.
- shiroiushi 2y agoProbably, but I'm always hoping to hear a story about a company laying off a highly valuable employee(s) (usually for cost-cutting reasons) and then going bankrupt as a result.
- serbrech 2y agoThe Codescene product provides an in-depth analysis of knowledge distribution over time: https://codescene.io/docs/guides/social/knowledge-distribution.html https://codescene.io/docs/guides/social/knowledge-distributi... I recall seeing the linux kernel repo analysis as a show case, but I can't find it anymore.
- thepuppet33r 2y agoOn the one hand, I would be very surprised if this sort of dashboard metric didn't already exist in some enterprise software somewhere. My management at a previous company literally asked me if we could do a daily report on who sent and received the most email in our department. I refused to do it because I didn't like where I saw it going, but another coworker did. Unsurprisingly, the person who got the most email and sent the most email was the Sysadmin whose email account was set as sender for all the automated emails from the various servers, as he would literally email himself hundreds of alert emails a day, on top of all the crap newsletters and digests he subscribed to. On the other hand, all of your coworkers asking you to not do something that could potentially impact their jobs, and you do it anyway as a hobby project? Sounds like kind of a jerk move.
- jschrf 2y agoKudos to you. Bullshit metric.
- shae 2y agoI didn't write it in 2015 when my coworkers asked me not to. I do want to see if open source software I use has spread the knowledge to increase survival. I refused the jerk move!
- thepuppet33r 2y agoNice!
- grogenaut 2y agoAmazon has these numbers easily accessible as reports on their code systems runnable at any manager level, and many other ways to inspect what the team is doing and the risks you might have. I find them useful. Bus factor is one way to think of it. Another is it lets you spot silos, or engineers who aren't working with others, or places where you can't as easily move engineers around (so you can fix that). Some developers fear fungability, they think that that one system only they know is job security. I see it the other way, I see that as a technical risk, but also a thing that might be keeping a great engineer from working on more important projects. Or the way to work on something else when you get fed up with that one system you hate.
- fn-mote 2y ago> Some developers [...] think that that one system only they know is job security "You can't be promoted if you can't be replaced."
- null0pointer 2y agoPerhaps another framing of the same idea is that you can’t work on other (bigger and better) things if you can’t hand your work off to someone else.
- efitz 2y agoI don’t want to frame it positively. In my experience, people who hoard knowledge and/or relationships (as with other teams or customers) are toxic. I hate all sorts of gatekeeping behavior but I especially despise having to wheedle information or effort out of someone like that. They almost invariably have an ego that is out of proportion to their contributions. Side note: Occasionally I have seen people who take on the thankless task of being a maintainer of something no one else wants- those people tend to welcome others and share as much as others are willing to listen to, so being the single person who knows something doesn’t make you a gatekeeper. It’s the gatekeepers that I hate.
- shae 2y agoArticle author here: I've always said I want to be able to go on vacation and not get a call. I like your framing better!
- bunsenhoneydew 2y agoThis is one of the features of https://codescene.com/ https://codescene.com/ It looks for knowledge islands and relates those to frequently modified code, to identify hotspot, or areas of high risk due to low knowledge distribution in areas of high change. Another use is if someone hands in their notice you can easily see all the code that only they know, so your handover planning is mapped out easily. I’ve never thought of it being used maliciously, it’s for visibility. It would be a shitty manager that would use it that way and if they’re already shitty then this tool won’t change that.
- llm_trw 2y ago>I’ve never thought of it being used maliciously, it’s for visibility. It would be a shitty manager that would use it that way and if they’re already shitty then this tool won’t change that. You are a member of the intelligence community of a country, let's call it Tussia, which has been locked out of the leading kernel for military hardware in the world. Let's call that Kinux. You know that the guy down the office has started a project to fork that kernel for your countries own internal usage. You're an over achiever and want a promotion before he gets one. You call acquisitions for 8 female agents with special training for intimacy with nerds, you also make a back up call for 8 doses of polonium in case the agents aren't successful. In case you think the above is fiction I know a CEO of a unicorn startup who got the first part of the treatment when he was looking for seed funding.
- 1123581321 2y agoDid it work?
- aorloff 2y agoThe sad irony of this is that its still the wrong question. When startups have to do layoffs, the question isn't "who can I fire and keep the existing business going" it is instead "who is the team I want building the next version of the product quickly enough to not go out of business" But every fork in the road is just exactly that, and not choosing a path quickly enough has been the death of many
- nextlevelwizard 2y agoHow is this ironic?
- bhaney 2y ago> Why does gnu parallel use all the cores when we tell it to use only 8? We only saw eight git clone processes at a time, but a large number of git index-pack processes that maxed out all 32 cores on my laptop. I’m guessing git’s index-pack is a forked subprocess and allows parallel to start another git clone? Parallel is running 8 git clone jobs at once (as asked for) and each git clone is starting as many index-pack threads as it wants. Temporarily setting pack.threads to 1 (via git config) would help here.
- chrismorgan 2y agoThis is an increasingly common problem: two layers, both trying to utilise the entire machine by parallelising by the number of CPUs/cores, and so the inner layer gets N² threads/processes. Being quadratic growth, this means that the problem becomes worse the more CPUs you have. With 32 cores, 32² = 1024, though in practice Parallel was told 8 so it’d cap out at only 256 index-pack processes. But that’s a lot of memory needed to support that, especially given that it’s doing no good. The solution is for only one of the two layers to parallelise. —⁂— Given that you talked about pack.threads, here’s its description from `man git-config`: > Specifies the number of threads to spawn when searching for best delta matches. This requires that git-pack-objects(1) be compiled with pthreads otherwise this option is ignored with a warning. This is meant to reduce packing time on multiprocessor machines. The required amount of memory for the delta search window is however multiplied by the number of threads. Specifying 0 will cause Git to auto-detect the number of CPUs and set the number of threads accordingly.
- PhilipRoman 2y ago>The solution is for only one of the two layers to parallelise If you have a common scheduling API, you can manage this much more elegantly. For example make(1) can control the concurrency level across recursive invocations by using a "job server" https://www.gnu.org/software/make/manual/html_node/Job-Slots.html https://www.gnu.org/software/make/manual/html_node/Job-Slots... With this, you can have, for example, 3 top level subprocesses, each spawning multiple threads of their own, but never exceeding the CPU count. Alternatively, parallel could make it's subprocesses think that they're running on a 1-core machine, although this may have some subtle side affects.
- joshdavham 2y agoIt would be cool if you guys produced a graph for truck factor vs number of github stars. It could roughly model the importance per team member or the net impact it would have on the "stargazers" if the maintainers were hit by a truck.
- shae 2y agoCould happen? mclare published the other half of the blog post with nice graphs a few hours later: https://mclare.blog/posts/the-bus-factor/ https://mclare.blog/posts/the-bus-factor/ I'd like to build a tree of open source dependencies and find out which parts need the most knowledge spreading.
- chuckadams 2y agoCPAN has been tracking the Bus Factor for a long time now. For example, https://metacpan.org/pod/Moose https://metacpan.org/pod/Moose shows a Bus Factor of 5 in the left column info.
- oalders 2y agoAlso, we did document how we arrive at the number: https://www.olafalders.com/2021/06/30/cpan-bus-factor/ https://www.olafalders.com/2021/06/30/cpan-bus-factor/
- jschrf 2y agoI don't like this take. This post is for any engineering leader. The Bus Factor is how hard your team would suffer if you - or anyone else on the team - got hit by a bus. Ideal Bus Factor for all team members is 0. This might sound counter-intuitive at first, almost like "make everybody expendable", but it's quite the opposite and kind of the point. Teams should be good enough that they are a) autonomous and b) there are no mysteries. In the ideal state, everyone understands how everything works. New employees should hit the ground running and be able to produce value immediately. Departing employees should feel comfort in knowing that there are no unknowns. An ideal team with 0 BF across the board is desirable. It means that team members are fungible. It means that every single team member can fill in the gaps if someone is ill, or on vacation, or actually leaves or is removed. More importantly, a 0 BF is a reflection of simplicity. The software, its build/test/deployment pipelines, documentation, and support, should all be cohesive and coherent. Siloing information in team members is bad, everyone should be able to build and deploy. 0 BF is a healthy metric, but it is absolutely 100% not measured in email rate, commit rate, PR rate, lines of code, timeliness, GitHub heatmaps, etc. Those metrics indicate nothing at all. Quite the oppositve. They are harmful, awful metrics. Measuring people by these metrics is just monkeys on typewriters. More startups need to hear this.
- saghm 2y ago> The Bus Factor is how hard your team would suffer if you - or anyone else on the team - got hit by a bus. > Ideal Bus Factor for all team members is 0. This might sound counter-intuitive at first, almost like "make everybody expendable", but it's quite the opposite and kind of the point. I've always heard bus factor described in the inverse fashion, as in "how many people would need to get hit by a bus for the project not to be able to continue", with the optimal number being the same as the number of people on the team. It sounds like the idea is the same, but I'm surprised to find out that the number people to convey the concept isn't always the same.
- Moru 2y agoA number between 0 and 1 can easily scale to whatever company you have. A number that is different depending on how many is on your team is harder to compare between teams. I guess it depends on if you ask a programmer, administrator or mathematician what a logical system would be :-)
- d--b 2y agoNot only is this a metric that doesn’t make any sense, simply floating the concept that this metric corresponds to anything in real life is harmful. Teams are social constructs, and you simply cannot apply an algo to some observable code metric and get any kind of proper result. People leave, others step up, or don’t. End of story. The problem with these kinds of metrics is that even if people know they are ridiculously off what they mean, some still think that the idea they convey is correct: ie. That if x people were to leave, the project would stall. That premise is simply not true. Don’t even think about this stuff. It’s stupid. If you want to know more about these risks, talk to the team. People on the team know if anyone is irreplaceable.
- jedberg 2y agoWe like to call it the lottery factor. Would the project keep going if someone won the lottery and took off to an off-grid tropical island. Less morbid that way.
- ainiriand 2y agoI'd argue that it is easier being hit by a bus so you have to factor that.
- jedberg 2y agoTrue, it's about 1000 times more likely to get hit by a bus, but only 5% of those are fatal. So it's not that far off. And still less morbid. :)
- sam_bristow 2y agoI tend to phrase it as hit by a bus OR jump on a bus to another company.
- EVa5I7bHFq9mnYK 2y agoDon't know anyone who won a lottery big, or got hit by a bus, but know several people who quit cause of crypto riches.
- erik_seaberg 2y agoA good employee will want to hand off his projects and take questions, but you need to be ready if he can't.
- rightbyte 2y agoYe using sudden death as an eupheism for changing employer leaves a bad taste. It is like when people complain about people using sports metaphors. I say that at least it is not military metaphores. Also, there is usually no handover anyways. So the suddeness factor is not that important.
- 2y ago
- nextlevelwizard 2y agoso where is the "plugin" mentioned? I want to see for funsies what my projects look like, but I dont want to run some random crusty java.
- gklitz 2y agoPut a positive spin on it. We don’t talk about coworkers getting hit by a bus, we talk about them winning the lottery and quitting on the spot due to their newfound fortune. Same discussion but it’s less morbid, and doesn’t end up sounding like your prioritizing the health of your project about the literal lives of your employees.
- Boltgolt 2y agoThe implications are different. There has to be a high euro sum you can offer to get someone who won the lottery to still brief a colleague. You can throw money at a dead person all you want but that will not have the same result. It is a sudden end in knowledge that can not be restored with time or money (like running out of lottery funds)
- hnbad 2y agoAlso, the idea that the only reason people work on the project is that they need the money surely seems less appealing than that the only thing stopping them from working on it is death? Or are we now just okay with saying the quiet part out loud and acknowledging the exploitative nature of the economic system?
- gklitz 2y ago> Also, the idea that the only reason people work on the project is that they need the money No one has presented this idea. Your arguing that we should stop paying employees, because they will obviously continue to work unless they are only working because they need the money? Saying that if someone wins the lottery they’ll likely leave to Perdue the opportunities that bag of money will present them to is not the same as saying the only thing keeping them at the job is money.
- gklitz 2y agoYou don’t need to plan for people leaving the company if you have a emergency reserve big enough to literally compete with the lottery payout wise in emergency. If someone leaves you can just hire a couple of hundred people to replace them and still save money.
- letters90 2y agoIf you wanted to view this from a darker side it could also be titled: How to generate a hit list to hurt a global economy
- shae 2y agoTrue! The original 2015 article I found on Linux Weekly News had comments that worried about that https://lwn.net/Articles/651384/ https://lwn.net/Articles/651384/ I think Linux is safe because everyone uses it.
- DeathArrow 2y agoThe title and the first paragraph mentioned a plugin but the article is not about a plugin but about trying to replicate some old results, and they've failed at that since they didn't 100% respect the method of original authors.
- aa-jv 2y agoThere's a very quick fix to this, which also radically improves the strength and potency of any organization: Do not ever accept management directives from someone who couldn't do your job. Yes, you heard me. If your manager cannot do what they are asking you to do, fire them immediately. This has the added bonus of ensuring that Peters Principle doesn't become a major mountain instead of a mole-hill. This does not mean that the manager has to do your job. It does mean that making sure your manager knows how to do your job is your responsibility and something that you, yourself as a worker-cog in a large machine, actually do control. I have shipped software all over the planet in all kinds of markets for all kinds of users. The most successful project is composed of individuals who have great deals of trust in each others' ability to perform, and who share the load in ways adequate to the task. In the most successfully managed projects, those managers who I knew could do my job, but didn't (because I did it), were absolutely the best to work for .. whereas those who had no idea how to wrangle a single line of code, yet gave themselves the altitude required to be a 'manager' were, across the board, a catastrophe. BTW, if you feel 'seen' by this comment - i.e you are a manager who feels a bit imposter'ish - don't worry, this is Peters Principle at work, and you can easily fight this by better communication with the folks whose work you cannot do ..
- kookamamie 2y ago> As you go up the career ladder, developers should do more review and less hands on keyboard. A common misunderstanding in tech companies, I think. You don't want to exchange great developers for mediocre managers.
- result2vino 2y agoIf you think that code review is “management” then that’s very very concerning.
- swiftcoder 2y agoWell, it certainly isn't "development". It's a hoop we jump through so that management can tick a box labelled "all commits were reviewed by another engineer" on the SOC2 audit...
- akerl_ 2y agoThat’s a really weird way to view code review. Everywhere I’ve been part of development, code review is part of ensuring code quality, catching bugs, and avoiding silos.
- swiftcoder 2y agoIf a non-trivial percentage of your code review feedback is about code quality and bugs, you are severely underinvesting in autoformatters/linters/strong type systems/testing/continuous-integration. It's not a cost-effective use of (expensive) software engineers to have them scanning every PR on the off-chance they notice a typo. I'll grant you they can help break down silos, but the question you should be asking, is why your codebase is so convoluted that silos are developing in the first place?
- danielvf 2y agoJust going to second this. Good code reviews (not just typo nitpicking) can be a great way to simplify down code, and spread knowledge horizontally across the org. Not to mention catching bugs. Unit test aren't a substitute because unit tests check that the success paths are good. That's a good start, but it's not the same as verifying all the possible ways code could go wrong in a complex system, and one of the cheapest ways to spot those problems is with people familiar with complex system looking at new code. Code review give you the double benefit of building more people who understand the whole system, and having the code looked at by people who understand the whole system.
- burnt-resistor 2y agoStep 0. Document everything in an organized manner, create runbooks, and forbid knowledge hoarding and manual processes only (so-and-so) knows.
- swiftcoder 2y agoWhile it is the platonic ideal for operating an existing service, this is also a great way to kill your velocity if you are an early-stage project which needs to deliver results yesterday, and/or pivot on a dime, in order to stay funded...
- burnt-resistor 2y agoOr kill your velocity by playing with creaky, fragile hacks only you understand that duplicate code and are a huge mess. I've seen countless wannabe startups that stupidly accumulated so much technical debt with a "let's worry about that tomorrow" techbro attitude that they had a relative velocity of zero when asked to make minor changes.
- swiftcoder 2y agoBuilding for rapid iteration is a skill like any other, and a lot of engineers who work in big companies never learned it. Blindly staffing one's early stage startup with BigTech engineers is a pretty good way to tank a startup.
- shae 2y agoPost Author here! mclare posted the other half with visualizations: https://mclare.blog/posts/the-bus-factor/ https://mclare.blog/posts/the-bus-factor/ Check it out!
- egeozcan 2y agoI really don’t like the term "bus factor" because I lost someone in a bus accident. On the other hand, millions of lives have literally been lost on various hills on various battlefields, yet no one seems to mind when someone says, "this is the hill I’m willing to die on." :)
- woodrowbarlow 2y agoi worked a software job that sometimes involved literally going to the busyard to debug on-board equipment. people at that job used the phrase attrition risk rather than bus factor because the latter was just a little too "real".
- jjkaczor 2y agoThese days the "politically correct" phrasing is: "Lottery winner" Someone wins the lottery. (I am old - I start with the poor bus driver scenario - but then try to inject the new phrasing into the conversation - as typically I am the only technical person on my team and they begin to rely on me for EVERYTHING)
- pabs3 2y agoWonder how many managers are thinking about the bus/lottery numbers of all the open source projects their developers are relying on, and then doing something about it.
- m-clare 2y agoThis is what I was most interested in with this project (especially in light of the https://opensourcepledge.com/ https://opensourcepledge.com/). We didn't run it against any new libraries, so it would be interesting to see the state of the most starred libraries.
- pabs3 2y agoThe Nebraska problem really is terrifying, I wonder how to get it in front of the eyes of CEOs and the like. PS: I suggested to LWN https://lwn.net/ https://lwn.net/ that they might want to contact you about the Linux numbers.
- RIMR 2y ago"Bus" and "Truck" terminologies invite you to imagine your coworkers being crushed to death. This isn't great. I've always preferred the term "lottery factor". If this employee won the lottery (almost certainly leaving their day job behind), would you be able to survive their sudden departure?
- dragonwriter 2y agoOver 30-ish years of work, I’ve lost about 0 coworkers without warning or transition to lotteries or similar events, and more than a few (whether permanently or for an extended period) to unexpected sudden death, injury (traffic related being particularly common), or other unheralded misfortune (law enforcement involvement, on a short out of country trip and then encountered visa issues, whatever.) “Bus factor” is a more realistic reflection of the threat model, and resonates with experiences anyone who has been working more than a short time probably has more than “lottery factor”.