9 ms·
Code review review is the manager’s job
- sgt101 8y agoAre there any field studies that demonstrate the impact and efficiency of these practices? Is there a body of evidence to support the assertions that are made in this post?
- johnb 8y agoHi - author here. The quality assertions are the most defensible empirically. I wrote the post at the office away from my bookshelf which was the main thing keeping me from including them up front (flicking through my old copy of code complete now). Capers Jones' "Software Defect-Removal Efficiency" (https://ieeexplore.ieee.org/document/488361/ https://ieeexplore.ieee.org/document/488361/) would be the main one on the effectiveness rate of code inspections. I've collected a few other relevant papers over at https://github.com/joho/awesome-code-review https://github.com/joho/awesome-code-review The second two themes around education and culture come more from my personal perspective and experience - largely from running the engineering teams at Envato and 99designs respectively. My feeling is the educational and safety elements of code review have come up online a lot more over the past couple of years - but that's just my anecdotal view.
- blazespin 8y agoUnfortunately, productivity is always context dependent.
- sgt101 8y agoSome good points - many thanks. I think that there is a market failure / exec education issue around these point though. The impact of the improvements measured/evidenced is in the maintenance cost of the delivered artefact and the improvement of the process for future delivery. These are "over the fence" issues for the owners of the delivery shop. We need to get it into the bosses minds that they are paying for things that will be either good in the long term or expensive in the long term - I think that there is a lot of hiding and hedging and shifting of burden (to users) that goes on around that.
- johnb 8y agoI agree and trying to cover that education gap is what I'm trying to achieve with the post (and the product behind it)
- JackMorgan 8y agoHere's the thing, there's no way to measure programming productivity at all, so, no, there's no valid studies about anything ever. The studies that try always suffer from this inherent flaw. They count bugs, lines of code, function points, all metrics that aren't actual business value. Pair programming vs code review, Nodejs vs Ruby, React vs Vue, Vim vs Atom, etc etc etc are all 100% emotional and subjective decisions. Therefore they are subject to fashion, being cool, and herd mentality. Accepting this is an essential step on the path to become a well-rounded engineer.
- johnb 8y agoThat's not really true. A better formulation is that there's no way to manage programming productivity cheaply and/or non-intrusively. I agree that the bulk of X vs Y comparisons are emotive and subject to what Alan Kay calls the "pop culture" but that in no way demonstrates that measurement is out of reach. It's just in most contexts we don't care to do the work to measure it out of personal preference or economic reality.
- JackMorgan 8y agoGood point, productivity measurement is probably possible, but so vastly expensive enough that it's not happened yet. I really like this post and the one about managers programming btw. Your common sense explanation to why most new managers end up coming back and reporting that they shouldn't code is exactly what I've seen as well.
- johnb 8y agoThere was an old Tom DeMarco article I'm a fan of: http://www.systemsguild.com/pdfs/DeMarcoNov2011.pdf http://www.systemsguild.com/pdfs/DeMarcoNov2011.pdf In it he has a pretty interesting insight about late projects. "If a project offered a value of 10 times its estimated cost, no one would care if the actual cost to get it done were double the estimate. On the other hand, if expected value were only 10 percent greater than expected cost, lateness would be a disaster." I imagine as our field matures measuring productivity and hitting estimates will get more important once software has finished eating the world. The unlimited (or close to) upside dev projects will be largely behind us and we'll need to hit tighter economic targets. I'm just glad I'm working in the era where I don't need to measure it :D Glad you like the posts. Thank you!
- sonnyblarney 8y agoThis is a very good article.
- deleted 8y ago[deleted]
- crdoconnor 8y ago>Code review, whether via a pull request or other means, still is one of the best bang for buck methods for finding defects in code. I don't find that to be true. It's a good way of keeping code quality in check, which has an indirectly positive effect on catching defects (as well as a number of other positive effects), but it's not a good way of catching defects itself.
- johnb 8y agoAuthor here. The post is largely written from a point of personal experience and philosophy but that's the one statement I'm comfortable saying has empirical backing. I link to https://blog.codinghorror.com/code-reviews-just-do-it/ https://blog.codinghorror.com/code-reviews-just-do-it/ in the post which has an excerpt from code complete: > … software testing alone has limited effectiveness – the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent.
- JackMorgan 8y agoAs a firm believer in TDD and pair programming, I would caution you that since ultimate goal of programming productivity is unmeasurable, defect rate could also just be a measure of how much slower those practices are. I don't think that's the case, but really we have no way to prove that it's true. Just a word of caution if you are ever trying to convince someone, it's best to be up front with the possible flaws of those studies.
- crdoconnor 8y agoMy experience is that having a really great design eliminates a lot of bugs (60% is perhaps not unreasonable, although pretty hard to measure), but having an awesome design is not even close to being the same thing as doing regular code reviews. I'm often thrown in to a project with bad to mediocre design and I am not able to just change the design to become "good" just by reviewing PRs. I can only attempt to push the design into a slightly better direction over a long period of time. Similarly, just because it's reviewed doesn't mean that the design is great. I am probably able to hit something approaching 45% defect detection rate with integration testing on most projects though. With code reviews I'd say it's about 1 or 2%. Bug-wise, I usually only spot fairly obvious "language" gotchas (e.g. initializing an empty list in a method signature in python).
- maxxxxx 8y agoHow do these benefits compare to what you get from pair programming? If the pair works well together I think pairing provides more learning and better quality.
- lovelle 8y agoI think code review is the async way to share thoughts about the code and managers (in my experience) tend to be busy with many meetings making 1 on 1 code review more difficult therefore in this case pull request might not be the best option but a good one.
- johnb 8y agoI'm not super across the evidence on pair programming so I'd hate to try and have a definitive answer. https://www.cs.utexas.edu/users/mckinley/305j/pair-hcs-2006.pdf https://www.cs.utexas.edu/users/mckinley/305j/pair-hcs-2006.... suggests it's pretty good for quality and so-so for utilisation. My anecdotal experience with my own teams is relying too much on pairing as a training mechanism, especially between senior and junior, has a couple of issues. The first is tradeoff between training quality and utilisation - it may shorten the training cycle but the payoff isn't high enough for the output constraint on the more senior member. The second is I've sometimes had trouble weaning the junior off always having guidance on tap and it takes them longer to develop confidence to break through certain problems alone.
- Cthulhu_ 8y agoCode review is (from my point of view) a way to have a second or third person look at the code from a distance, in his own time, with less context than the author. When I write code I'm far into it and stop seeing the bigger picture. When you pair program, I think the same thing happens. Finally there's the end product, which can be looked at from a distance by a reviewer. While yes, pair programming is preferred over just pushing to master, I think code reviews (and moreover formal code reviews) are better than pair programming for the final OK.
- maxxxxx 8y agoHow much time do you take for a code review? If I did a full review that really tries to understand the code it would take me close to the time it took to write the code. But I don't get that time. So I pretty much nitpick on some style issues but rarely find any real problems.
- barrkel 8y agoThe manager can do code review if they're a player-manager. If they're a people manager, it's a lot harder to meaningfully review. Managers usually hired in to manage after the team grows (and splits), rather than trying to convert developers into managers, so they may not have a good idea about the code at all. I'm more inclined to think that it's senior technical responsibility to review, since they'll have a lot more knowledge of the codebase, its evolution and future direction, potential gotchas and various cultural dos and don'ts. For sure, it's important to keep an eye on the code review culture, though. It's easy for code review to degenerate into nitpicking small things like whitespace, naming, coding conventions - these should be handled by tools instead. And of course if people are being assholes, they need to be pulled up on it.
- telltruth 8y agoIf your team has pure “people manager”, run!
- barrkel 8y agoNot a pure people manager - managers with a background of coding are important so they won't swallow BS, can understand tradeoffs between e.g. refactoring vs accepting tech debt to get a feature finished quickly, etc., understand the reliability or otherwise of dev estimates, etc. But few strong developers are also strong people managers. The two attributes seldom overlap. And I think being good with people is more important in a manager than being good with code.
- quantumhobbit 8y agoOne of the best managers I ever worked for was a “people manager”. He knew his limitations and worked within them. He was reliant on the tech lead for technical knowledge but between the two of them made a pretty good team. Some of the worst managers I’ve worked for have been technical people who don’t really want to manage but git Peter Pricipled into management.
- TickleSteve 8y agoManagement skills are orthogonal to technical skills. People who are good technically are not necessarily good at management and should not be put in that position. They should however be put into a code-reviewing position.
- agounaris 8y agoNo its not, code review is the team's job, including the manager.
- EliRivers 8y agoIt says code review review, not code review.
- EliRivers 8y agoBefore jumping in with your thoughts on how managers should or should not be involved in code review, note that this is about code review review. Reviewing the code review process, and reviewing the code reviews.
- quii 8y agoHoping there will be a thought leader who will show us how to conduct code review, review, review.
- gypsydave5 8y ago10 Agiles to Griffindor!
- dbcurtis 8y agoHis name was Andy Grove.
- kjeetgill 8y agoI think you've said the most key thing here. Much of the HN conversation has drifted out of focus. The best management I've seen was tapped into the pulse of the team and knew how people interacted and the problems we were struggling with. Keeping an ear to code reviews is no different than observing how your team interacts in person or in emails.
- ahartmetz 8y agoYeah, I see more projects with bad code review culture than good. There are many things that managers can do about it. * Assign enough time for code reviews - otherwise developers will just care about their own code submissions and let reviews rot. * Establish productive code review rules. E.g. always go from big picture to details, otherwise you'll waste time on details that you end up throwing away. * Make reviewers co-responsible for the quality of code reviewed, otherwise instant acceptance for everything becomes attractive.
- johnb 8y agoThose are all really good points. I have a recollection of seeing a study once that found the best balance point of review time vs dev time but haven't found it again. I recall it being somewhere around 20-30% but now that I can't find it I'm questioning my recall.
- johnb 8y agoI definitely overstated here sorry. Found the excerpt I was remembering in my copy of code complete "On a project that uses inspections for design and code, the inspections will take up about 10-15 percent of project budget and will typically reduce overall project cost". So assuming the same payoff benefit (big if, but let's go with it) between PR style reviewing and more formal inspections, every engineer should be spending around four or five hours a week performing reviews to hit the sweet spot.
- quantumhobbit 8y agoI just left a job with a truly toxic code review culture. - managers would swoop in and leave incorrect criticisms. Asserting plainly false things and in one case reviewing python code as though it was JavaScript. - devs would make conflicting requirements for improvement. Dev A demands change A. Dev B wants change B. A and B are incompatible and neither dev will back down. Management of course refused to help settle the argument. - change requests completely unrelated to the PR being reviewed would get crammed into the PR. So every small code change could balloon into weeks on refactoring work. - Often times code review comments were made without ever reading the code being reviewed. Such as a comment of “Why aren’t you using library X?” sandwiched in between invocations of library X. The culture was clearly rewarding people for shitting on others work instead of doing any work themselves. Getting even the simplest code changes past required endless meetings explaining the most basic facts about computers. So glad to be gone from there.
- johnb 8y agoPost author here - logging off to go to bed because it's bedtime down under. Will pop by again in the morning. Thanks for reading and all your comments.
- something2332 8y agoThis is fine if the manager is also shipping code. If the manager is not shipping code then it's really not cool to do code review. You won't earn respect that way.
- DanielBMarkham 8y agoI agree with this in principle. Gets a lot tougher in practice, though. "Manager" really shouldn't be a job in tech, at least when it comes to individual teams. Management should be a skill, just like all the other dozens of skills tech teams manage in various ad-hoc ways. Groups of teams plays out differently, since you're really starting to work at the meta level. Even then, though, management is just one of a bunch of skills necessary. (At that point, however, it may be so much of a primary skill to warrant a separate role)
- pavel_lishin 8y ago> Management should be a skill, just like all the other dozens of skills tech teams manage in various ad-hoc ways. I would phrase it as "like all the other dozens of skills tech teams half-ass in various broken ways".
- DanielBMarkham 8y agoI think an important point in modern technology development is that to make anything of value happen, you constantly deal with dozens of various silos, each of which can be a lifelong specialty area. You have to learn to skim most of the time and be able to dive deep -- a bit -- as necessary. We may be saying the same thing, but I like my way better :)
- pavel_lishin 8y agoI agree that everyone needs some management skills, but I'm not convinced that most teams can be run without a dedicated manager.
- DanielBMarkham 8y agoAgreed. The same as most teams cannot be run without a dedicated DBA. The question is whether or not the DBA does other things, not whether that skill is needed or not. If you've got six or seven people that need constant management, you're working at a burger joint, not a tech shop. If I remember correctly, in the book "In the Plex", Google scaled out to thousands of developers without having a dedicated management slot. (This is a good time to mention that all generalizations are false, including this one) In general, in the tech world I find the biggest supporters of "X should be its own job" are either the people who do X -- or train people to do X. Customers don't care one way or another. Neither should we. If we need an X, we get one.
- wavefunction 8y agoOn my team we're all managers. We manage ourselves and our output, and we manage our team, team's output and each other by supporting and assisting each other. So from that standpoint I agree. I don't see the need for dedicated managers though I do value our dedicated scrum/agile coach though, who encourages all of that.
- bradenb 8y agoThis is a really interesting read. Thanks. I have found in my own experience that developers tend to include the same people over and over again in their PRs and what ends up happening is people tend to just "trust" what the submitter has committed. This often leads to PRs that are almost blindly approved. By involving management to keep the PRs as a tool for discussion (social and technical), I think this could be avoided. It's probably also a good idea to mix up the reviewers (a healthy mix of people familiar with the code and those not familiar with it).
- mychael 8y agoPlease ignore the advice in that article. It's hinged on the false idea that a manager always knows best. Imagine for a second that the manager is wrong in his/her judgement. Now the developer has to choose between arguing with the manager and risk being viewed as insubordinate or just going along with the feedback and letting the software suffer the consequences.
- hyperpape 8y agoHow is an un-self-aware manager ever going to be effective at anything? The author notes your objection, and addresses it: "There’s a fine line to be walked between managing the pull request process and micro-managing it. What happens within any specific pull request is the domain of the team so you should resist the urge to intervene frequently within the process itself. The behavioural trends in the process are your domain and if you’re running a meta-review process well your work will be predominantly behind the scenes."
- mychael 8y ago> meta review Is "meta review" really a value add for software companies? Is this advice for Managers of peace-time companies who can't find anything more valuable to do with their time?
- AlexCoventry 8y agoI've seen pull request interactions where a moderating authority would have made all the difference. We mostly accept the need these days to moderate other online social behavior, like on HN, to reduce potential damage from ego/politics. Hopefully the need is lower in a professional environment like SD, but I could still see it being useful when things go off the rails.
- hyperpape 8y agoI suspect manager here does not mean high level manager, but someone who has at most a dozen or two people under them. And yes, effective culture around PRs seems valuable. My current team seems to have very inconsistent expectations, and would benefit from someone with authority steering people in the right direction.
- 0x8BADF00D 8y agoThis is a terrible idea because it will lead to a more hierarchical corporate environment. My manager has been coding for 15 years in the embedded space, but he is rarely on my PRs (unless we happen to be working on an overlapping feature). At my company, PRs will automerge if at least one reviewer with write access approves and CI checks pass. This is the best way IMO. It’s fast, too.
- OnlyRepliesToBS 8y agohah no
- atomicbeanie 8y ago-1 Moving the evolution of a groups code into the hands of the person who works with it the least is an anti-pattern. Driving change from within a team can be facilitated by a manager who reads the group's code, but any manager who thinks they are going to be the judge of what the team is achieving is deluded and overly enthusiastic about their authority.
- ndh2 8y agoThe author tried to be clever with the title, but failed to take into account how human perception works. The second "review" gets filtered out like that gorilla in the basketball video.
- techopoly 8y agoOh my gosh...I didn't even realize the word repeats...
- andyidsinga 8y agoholy shit - got me too!
- drharby 8y agoYa got me
- amenod 8y agoFWIW, I did catch it - at first I thought it's a typo, and after a few seconds realized it was intentional. It is a fitting title though, if a bit... smart.
- dave_sid 8y agoBut is the code review review review the manager’s manager’s job?
- pvarangot 8y agoOf course it is it is
- sorval 8y agoA manager sitting in meetings all day will be poor at responding to review requests. Its the same reason they make crappy oncalls.