10 ms·
In the last engineering team I managed (by far the most successful one up to date), we discarded most velocity measures in favour of a simple solution: every Fr
by hliyan 3y ago
In the last engineering team I managed (by far the most successful one up to date), we discarded most velocity measures in favour of a simple solution: every Friday, each team sends a brief "here's what we delivered this week" email to the whole company. It contains some screenshots and instructions like "agents can now update the foobar rate from their mobile app without having to call in". After a month or two of these going out like clockwork, it gave both management and business stakeholders a level of comfort that no amount of KPI dashboards ever could. KPIs compress away details that stakeholders need to form an understanding. A full blast from the task tracker is overkill. This solution worked for us. But of course, required a solid feature flag practice so that half-built features (story slices) could be released weekly for testing without affecting production. That said, we did maintain a quarter-level sum-of-t-shirt-size based KPI.
- cheschire 3y agoWas your sum KPI burned down on a report during the quarter or was it only reviewed once the quarter was complete?
- hliyan 3y agoI did keep an eye on it because it was a good indicator of problems. But I never drove the team based on it, any more than I would try to increase a car's speed by grabbing the speedometer needle.
- pydry 3y agoI like how the most convincing examples of "agile transformations" almost never use the word agile.
- lloydatkinson 3y agoIn those cases (the majority) the word Agile is used to describe it. Not actually agile, just whatever the hell Agile is. https://www.youtube.com/watch?v=a-BOSpxYJ9M https://www.youtube.com/watch?v=a-BOSpxYJ9M
- dustingetz 3y agoscale of team?
- hliyan 3y agoAbout 50 engineers spread across 7 teams
- meerita 3y agoKPIs encompass more than just team performance; they also encompass the performance of the product itself. Even if you excel at delivering, a poorly performing product resulting from flawed ideas renders delivery insignificant.
- deleted 3y ago[deleted]
- bluGill 3y agoKPI should do that. However it is very hard to measure the useful things in advance. The real measure is the bottom line after the product is released until it stops production. If you have an apprentice plumber cleaning out clogged drains it is easy as you work for a couple hours and then you get a check a month later. (this is a reasonable example of something easy to measure, but I doubt any plumbing company ever operated this way). However a lot of software is on a very long release cycle with hundreds of other developers. Even if you are Google and deploy to production often it can still be a long time before you start work on some feature and it is actually useful to customers, and then you are constantly enhancing things for the next version so that is even more noise making it hard to measure what has a useful impact.
- meerita 3y agoMeasuring is not difficult, unless you are attempting to measure something immeasurable. The appropriate sequence would be as follows: 1. Determine whether there is supporting data for the proposed change. 2. If no data is available, implement tracking to establish a reference point. 3. Measure the reference point and ascertain what could lead to success. 4. Implement the change. 5. Review the new data and compare it against past values.
- bluGill 3y agoSure, but the feedback loops for what I want to measure are 10 years long. You can go from fresh out of school to senior engineer in that time if we can measure your contributions faster than 10 years.
- chiefalchemist 3y ago> KPIs compress away details that stakeholders need to form an understanding. Comms. It always boils down to comms. Specifically, comms that drive understanding. KPIs = knowledge. But what you did, increased understanding. And what I like to say is: "(Knowledge isn't power.) Understanding Is Power."
- S_A_P 3y agoBut doesn’t knowledge imply understanding? Otherwise it would just be facts without context.
- yathaid 3y agoKPIs are facts. Facts without context don't lead to understanding.
- mejutoco 3y agoIMO KPIs are metrics to optimize (if they are objective). A set of values is already encoded in them by the specific formula being chosen.
- chiefalchemist 3y agoAt the extreme, watch Jeopardy. People spitting out their knowledge. Unfortunately, 99 out of 100 times, they have very little *understanding* of the answers they offer. Knowing the dots doesn't mean you can connect them. And thus, "Understanding Is Power."
- S_A_P 3y agoI totally get what you mean here, but I don’t know if Jeopardy is a great example. I think of that like trivia which is a skill that you have to practice and exercise. Rote memorization can increase knowledge and definitely helps. (Here comes the but :) ) But I don’t consider someone who has a head full of facts(only) to be knowledgeable. They have to be able to talk about the concepts and theory to back up the facts. The very act of connecting the dots demonstrates knowledge. Anyway I think we are probably saying the same thing and discussing semantics.(and to be fair it is probably just me:) )
- mdorazio 3y agoHow did the team report out things that weren’t new features? Performance improvements and bug fixes are often more important than adding more stuff to the product.
- yurishimo 3y agoI had a similar report structure, and generally we would relay the same information. "Here's a graph of measurements from before and after the code change, showing a 20% reduction in resources saving us $X/year". Generally that was good enough.
- mdorazio 3y agoI like this approach with a graph. In the past I’ve run into people ignoring text bullets about the “boring” development wins.
- cookie_monsta 3y agoIs there some reason you can't report on performance improvements and bug fixes?
- itsoktocry 3y ago>Is there some reason you can't report on performance improvements and bug fixes? Exactly. What does "performance improvement" even mean if you aren't measuring performance before and after?
- feoren 3y agoBecause a shiny new feature that takes 1/10th of the time gets 10x more praise and attention. Because naive senior vice president MBAs believe there never should have been bugs in the first place, and every fix is an admission that the bug existed at all. Because every email is now a weekly contest between teams, and when hard times come the ones who reported on performance improvements and bug fixes will lose their jobs long before the ones who made the header the president's favorite color. Because every email is your team trying to convince the company of your own worth and it's much harder to show a pretty screenshot of a performance improvement than a new feature (Graphs of before/after performance every week? Every week?) Because bugs don't always follow your bullshit schedule made by a bullshit manager who is looking for a pat on the back by replacing bullshit A with bullshit B, instead of anyone anywhere in the company ever just trusting anyone.
- gardenhedge 3y agoSo you planned, built, tested (and fixed any issues from testing) and released features (and supported them) each week? Edit: I see the team is made up of 50 people across multiple teams. That makes a bit more sense.
- switch007 3y agoI’m assuming each team has a PM because it’s impossible for an engineer to communicate therefore this couldn’t have happened without a PM being in place to play “telephone”.
- Aurornis 3y agoI’ve watched this backfire, too. It works great when everyone is delivering day-long or week-long incremental features that lend themselves to nice screenshots. But then you slowly start accumulating a backlog of difficult tasks that everyone avoids because they won’t sound good for the week. Technical debt accumulates as people cut corners in the interest of getting their weekly presentations out. You can theoretically avoid it with enough trust, but the trigger for our collapse was a round of layoffs where the people with the most visible features were spared while people working on important backend work and bug fixes got cut. After that, it was game over.
- yathaid 3y agoNothing in OPs comment says it should only be completed stuff that goes into the status. Also, if your team is optimising for the status report, your manager has already failed you.
- fjfuvucucuc 3y agoYes. The manager has failed you. Doesn't mean you should put your ass on the line to fix things for the business. That's why these kind of incentive structures are dangerous and why things like OKRs put such heavy emphasis on regular, company wide, failure. (I.e. they punish 100success rates)
- ranting-moth 3y agoAurornis didn't say that. He specifically said "because they won’t sound good for the week". With time and experience, you'll learn that those who can quickly churn out features and bug fixes are deemed extremely valuable to the company. Let's say we have a wicked bug that has very little chance of happening. But if it does, it'll bankrupt the company, no questions asked. Spending 3 weeks fixing it is nowhere as impressive to the company as Joe who's churned out 10 features per week while your status updates are "Hunting for the wicked bug", even if you describe it it more details.
- neeleshs 3y ago
- hankchinaski 3y agothat works well when the deliverable shipped and the value can be easily understood by the audience is targeted to, for example product team shipping updates to non-technical people. When working on a backend refactoring, CI/CD pipelines, infra work and other things it's only valuable if the audience is engineering, other people aren't gonna understand the value of a new CI/CD pipeline for example
- generic92034 3y ago> every Friday, each team sends a brief "here's what we delivered this week" email to the whole company I am not sure this scales very well with company size. ;)
- swalsh 3y agoHave each squad send the email to the platoon, and each platoon send a summarized email to the company.
- generic92034 3y agoDepending on company culture that can lead to uniformly very positive feedback on the lower levels (amazing new features, developers show excellent performance, all of them), which does not seem to fit the mess the top management can see at their level (customers complaining, sales dropping).
- jodrellblank 3y ago"the Ministry of Plenty's forecast had estimated the output of boots for the quarter at 145 million pairs. The actual output was given as sixty-two millions. Winston, however, in rewriting the forecast, marked the figure down to fifty-seven millions, so as to allow for the usual claim that the quota had been overfulfilled. In any case, sixty-two millions was no nearer the truth than fifty-seven millions, or than 145 millions. Very likely no boots had been produced at all. Likelier still, nobody knew how many had been produced, much less cared. All one knew was that every quarter astronomical numbers of boots were produced on paper, while perhaps half the population of Oceania went barefoot. And so it was with every class of recorded fact, great or small. [...preamble] it was not even forgery. It was merely the substitution of one piece of nonsense for another. Most of the material that you were dealing with had no connexion with anything in the real world, not even the kind of connexion that is contained in a direct lie. Statistics were just as much a fantasy in their original version as in their rectified version. A great deal of the time you were expected to make them up out of your head." - George Orwell, 1984. All one knew was that every quarter astronomical numbers of features and improvements were produced on paper, while perhaps half the users of SaaS products complained or left.
- nonameiguess 3y agoThis sort of practice killed me as a platform engineer when it was an org-wide expectation that we do these weekly demos of what a feature looks like to a user. Sure, let me just screen record a new environment buildout or a suite of applications running for ten hours and you can observe how often a problem occurs and what happens in response. Even better when an activity for a week is something like we upgraded a storage driver across all environments. What does a user see? Nothing, but the old one had bugs, including at least one CVE. Now we don't.
- 20after4 3y agoLean into it. "What does a user see? Nothing, but the old one had bugs, including at least one CVE. Now we don't." That seems like a reasonable content for demo and with just the right amount of self-deprecating humor it can be a slightly entertaining 2 minute presentation.
- justin_oaks 3y agoThe smartest folks will recognize "User sees no change" as a good thing. Keeping things stable is extremely important. Unfortunately, too few people think that way.
- osigurdson 3y agoDid anyone actually read it? The problem with this is the frequency is so high that it would just get tuned out.
- Consultant32452 3y agoMy first thought was that I would create a rule to dump these in a folder I never looked at. However, if I were on the team generating that email I would like it if it could truly replace the old ways of providing updates to leadership.
- alexchamberlain 3y agoThe irony is this is exactly what we did (albeit not the whole company) before adopting agile.
- js8 3y agoI was about to say the same. Before Agile, we had one Friday meeting with development manager where we pretty much did this.
- 20after4 3y agoThis seems perfect to me. It reminds me of deviantArt tech dev meetings when I worked there (~10 years ago) Every Monday, each tech/product team did a small demo for the whole tech org (maybe 50 people, growing to nearly 100 by the time I left). Teams without an interactive demo would just put together a web page with a few pictures and text describing what they accomplished and team lead would present it to the whole org. Teams competed for dubious accolades like most lines of code deleted, most embarrassing feature, best meme. Prizes were arbitrarily awarded by the VP of technology in the form of credits to buy art from their prints shop. Similarly to what you describe, this practice relied heavily on feature flags enabling us to release features for testing while they were still in early stages of development. This worked out really well for getting feedback and QA testing early on, while also keeping everyone up to date about everything that was happening outside of our immediate area of focus. It was fun and motivating as well. deviantArt did a lot of things really right though back in the early 2010s. IMO it was a really incredible engineering org and probably the best job I ever had.
- lamontcg 3y agoThat optimizes for Demo-driven development though. I've seen a lot of flashy "clever" stuff created for demos which were later totally abandoned. And a lot of basement-level code wrangling that is difficult to demo.
- oq_pmg 3y agoInteresting. I did the same within just my team. We used Confluence to publish those, describing accomplishments of every IC. This initiative was started to appreciate the work of the team, so I still was concerned if anyone were reading those reports apart from my team, but I never had a question from the Steering Committee "what all of these peeps doing there?" since I started. Question: what the format you have used (number of words)? Kind of a release notes? Was anyone actually reading those?
- hliyan 3y agoE.g. "Agents can now see their up-to-date balance on the app (previously they had to wait for the weekly email). Here's a screenshot. Here's a direct link to the QA environment so you can try it out yourself." That's usually it for a given feature.
- ramijames 3y agoI run the devrel team at Ultra.io, and tried this for 6 months or so at the beginning of the year. These types of updates only work in an environment where people actively read them.
- robertlagrant 3y agoUsing velocity as a KPI is to be prevented at all management levels.
- feoren 3y agoThis industry already has a huge problem with prioritizing shiny new features over fixing bugs, improving performance, verifying correctness, etc. This seems like a great way to enshrine that into your company culture. Team A: Added a wizzbazz button that says "Wizz!" when you push it with a cool noise! Look at this cool screenshot! Team B: Worked on fixing an elusive bug causing rare data corruption, but couldn't figure out what was causing it. Team A: Re-colored the header to the CEO's favorite color! Look at this cool screenshot! Team B: Fixed bug causing rare data corruption. Started looking into strange performance bottlenecks in UI. Team A: Added spiffy animations that play every time the user goes to the next page. Watch this cool video! Team B: Improved page responsiveness issue introduced by the wizzbazz button. Look at this graph; pay attention to the ninetieth-percentile time-to-first-render (dotted blue line) for dashboard users. It does actually go down quite a bit if you look. Team A: Added a thousand lines of code to the fluxborg module! Look at this graph! It went from FIFTY lines of code to a THOUSAND lines of code! Look at how much that is on this graph! Team B: Removed two thousand unnecessary lines of code from the wizzbazz module. We decided not to show a graph because the line goes down and we know you'll all think that's a bad thing. Which team is getting the axe when it comes time to lay off employees? You know. Come on. You know it's Team B. You know it's true.
- nine_zeros 3y agoThis is accurate. My director is obsessed with visible status updates aka UI changes or metrics. So anyone reporting, "fixed a 3 year old bug by refactoring old libraries, that improves maintenance" just doesn't cut it for him. And unfortunately for engineers, they have to dance to this director's negligence even if it comes at the cost of their own sanity.
- ChrisCinelli 3y agoCorporate dynamics make shiny object raise up even if it fool's gold. The real gold is in improving what in bad need of improvement. The companies that stay afloat are often those that are abundant in people that know what really matter and find a way to do it regardless what the middle management thinks.
- 3y ago
- AtNightWeCode 3y agoOverall this is my standpoint too. But it depends on how KPIs are set. I think it is more destroying to not have any general directions at all.
- 1270018080 3y agoNon frontend engineers probably despise this.