9 ms·
Time for a code-yellow?: A blunt instrument that works
- janice1999 2y agoComing from an old school engineering company, this article frankly reads as a little insane to me. I assumed "Code Yellow" was a catastrophe on par with a tornado hitting a factory or a massive security breach. Instead the examples are not hitting artificial growth metrics and needing to launch advertising. It's bad enough to blow up teams planned work but this is what you demand employees (not founders mind you) sacrifice their personal life for (i.e. get paid less per hour and not see their families)? The author not only acknowledges the time he is stealing from employees but that stressing them out is the point. If you're 8 years into a company and lurching from (artificial) crisis to crisis to "sweat the teams" something is seriously wrong.
- monkeydreams 2y agoI work in public health so our outages are critical to safe treatment of our patients. We have the concept of a Major Incident/Major Event (because the term Code Yellow is already co-opted to mean any administrative fault across a hospital which might impact the flow of patients). These are all-hands-on-deck moments. They may be called by any member of staff who discovers an event which will impact our service, though it will be ratified by a Major Incident Manager (MIM). While the event is underway the MIM is God; except when it affects staff welfare. If a member of staff says that they cannot attend a war room for whatever reason, then the MIM will move onto the next person up the chain, even calling in directors if they feel it is required. What is being described above is tech-bro shittery. Calling a major event because you haven't hit a sales target should keep the C-Suite up at night, sure, but calling in techs and devs and 'sacrificing the L and the B in Work Life Balance'? The C-Suite should be making the strategic decisions to reverse the decline, not suddenly drag everyone into a meeting to fix their lack of foresight and working towards an end that the average tech/dev cannot influence.
- ElevenLathe 2y agoI've been a major incident manager. It works great to have a hardass like me giving orders on a conference bridge: stuff gets fixed really fast! People pull out laptops at little league games and get it done. The outage is solved, and business goes on. I look like a genius. The problem is that, if a major incident is the only process in your organization that results in things getting done in a reasonable amount of time, there is a large temptation to declare them for everything. Somebody wasted 6 months failing to renew our contract with the payment processing vendor because other stuff was being prioritized above it? Open a major incident and we'll get them to call the vendor CEO at home on Sunday. We're a big account, so they'll answer. It's the end of the fiscal year and if we don't decomm this expensive system like was promised, a VP somewhere will lose their bonus? Declare a major incident and ElevenLathe will get it done in a few hours, even if it is Thanksgiving. I would page on-call people at any hour of the day for real emergencies without hesitation, but always asked for orders in writing to page people (not executives, but real people) who weren't on call, or for nonsense like this. This got annoying so they started assigning other incident managers to the nonsense.
- monkeydreams 2y ago>there is a large temptation to declare them for everything. Agreed. We have a very tight description for an Incident and, therefore, a major Incident. We don't call MI's often but anyone can call one.
- cbsmith 2y agoIt's not called a Code Red. Not all catastrophes are immediate in nature. Most organizations have incident response protocol which works well enough for an immediate catastrophe. Non-immediate, gradual threats can represent a much greater risk, because there's no inflection point (beyond "way too late") where the threat becomes so imminent as to trigger that response. Code Yellows are a mechanism to artificially force that inflection point before it is too late.
- Lammy 2y ago> It's not called a Code Red. The etymology is not green/yellow/red. It's just not-Yellow or yes-Yellow. See Stephen Levy's In The Plex (2011) pg186: “A Code Yellow is named after a tank top of that color owned by engineering director Wayne Rosing. During Code Yellow a leader is given the shirt and can tap anyone at Google and force him or her to drop a current project to help out. Often, the Code Yellow leader escalates the emergency into a war room situation and pulls people out of their offices and into a conference room for a more extended struggle.”
- cbsmith 2y ago> The etymology is not green/yellow/red. It's just not-Yellow or yes-Yellow. See Stephen Levy's In The Plex (2011) pg186: Yes, I know. I was making a rhetorical point using a metaphor.
- vitus 2y ago> The etymology is not green/yellow/red. It's just not-Yellow or yes-Yellow. Um, no. Today, Google has Code Reds, Code Yellows, Code Purples, and Code Greens... and this is after standardizing to remove other made-up terms like Code Mauve. I wish I were making up these terms.
- asciimike 2y agoNot sure why it's getting downvoted, definitely experienced Code Yellows, Code Reds and Code Purples at Google. Red is (obviously) worse than Yellow and IIRC was a total code freeze for a period of time. IIRC there was a Code Red around memory at some point because the supply chain was literally so backed up that google couldn't get enough DIMMs and reasonably sized services had to stop deploying because there wasn't enough compute capacity. Purple is/was "developer experience is so bad we need to stop developing new functionality and make the current functionality usable."
- AlexandrB 2y agoThere's definitely a "delusions of grandeur" thing that happens with some startups. "Come join our incredible journey to change the world by micro-optimizing ad placement on image boards."
- ponow 2y ago[flagged]
- HomeDeLaPot 2y agoUnhappy with your current government? You can always move to another country. Staying is acceptance of the current expression of terms of citizenship.
- snowfarthing 2y agoIt doesn't matter if it's literally stealing -- it's the mentality that is important here. Sure, you could always quit -- but the thing is, if you're pushing for these kinds of policies, you may very well push your best talent to accepting that option!
- nottorp 2y agoIs this an article about the virtues of mandatory overtime?
- DrillShopper 2y agoIt's from a manager / founder so yes, implicitly it is
- bawolff 2y agoThis feels like a blunt instrument to solve management failures.
- Waterluvian 2y agoAny time military language gets used like seen in this blog (eg. “War room”), it’s usually alongside management failure to be addressed by crunch of some sort.
- arkh 2y ago> Any time military language gets used And some military terms are never used. Reserve: the idle resource you have that you can deploy to stop a breach or to profit from an opportunity. Nope, we're using Human Resources like people used to use machines in the 80s: you have to get them working 100% of the time. And when there's an emergency? Code Yellow. When there's an unforeseen opportunity opening up? Well, too bad so sad your competitor got it.
- Lammy 2y ago> 2019: Building our fourth senior executive team in six years to lead us to the next plateau of scale. Sounds like nobody was sticking around long enough to even find out.
- cbsmith 2y agoThat seems about right. That's actually helpful. When you have a management failure, trying to solving it by "getting better at management" isn't really a solution.
- ponow 2y ago[flagged]
- bayindirh 2y ago> It is alluring because it allows for existing plans to be deprioritized, removes any/all ambiguity around what is most important at the moment, and strongly encourages the team to sacrifice the ‘L’ and ‘B’ from Work-Life-Balance. When I read this, I understand that when management fails, and wants something to be get done, because reasons, they can just pull all stops and declare temporary slavery until the problem is solved. This should be normally a "once in a decade" event. Not "once in a year" instrument. Being proud of removing "life" and "balance" from employees' reality reads like the worst power trip ever. I worked in similar environments. Never again. We have our "code yellow"s in my current job, but we know when it's going to come, and prepare ourselves and our lives. Go through it, pat ourselves on the back for the good job, learn our lessons for the things we fail along the way, and continue our journey.
- andrewjf 2y agoAlso the line about sending this email while on the weekend to his staff at his son's tae kwon do is particularly absurd.
- bayindirh 2y agoThe guy comes across as someone who thinks that non-work activities are just inconveniences for his and the company's well-being. Work is there to support other areas of life. Personal life is not something that we "do" in the time which is left from work. It's not the second class citizen when compared to work. Did he failed at this notion so badly, so he turned to this workaholic, everything else is secondary lifestyle, I don't know.
- roenxi 2y agoIt is hard to pick the most absurd line. I quite enjoyed "2018: Scaling our infrastructure to keep the site up on Sunday’s (we were having constant outages every week)". Take a moment to reflect that implies achieving one 9 of reliability required an emergency all-hands response that bypassed their normal processes. I may not be an expert in high-reliability systems but that isn't how I expect the problem to be tackled. My read is the management team are not very capable and that this gentleman is not a natural leader. Doesn't sound like he knows how to build systems in a planned and thoughtful manner and has embraced a classic lurching-into-crisis strategy for want of understanding any alternatives. EDIT Also juxtaposes beautifully with the "‘keeping the lights on’ fallacy" he mentions. Those crazy engineers with their belief that they need to try and keep the lights on. Also, when I forcefully pull them away from that, why do the lights go out on Sunday???
- mtrovo 2y agoI’ve experienced a Code Yellow myself and can definitely see its value. In large companies, you sometimes need that top-to-bottom rallying cry to overcome the bystander effect when big problems emerge. After ChatGPT’s initial release, Google famously sounded the alarm and brought Larry Page back into the mix, showing how this kind of all-hands effort can quickly organize everyone around a single goal.
- AlexandrB 2y agoBrought Larry Page back to do what? Google is worse than ever, Gemini is just one more piece of clutter ruining what used to be a useful tool.
- Blackthorn 2y ago> It is alluring because it allows for existing plans to be deprioritized, removes any/all ambiguity around what is most important at the moment, and strongly encourages the team to sacrifice the ‘L’ and ‘B’ from Work-Life-Balance. Why would anyone go along with this unless it came with a FAT bonus?
- 01HNNWZ0MV43FF 2y agoI also accept PTO credits as payment for voluntary overtime
- fn-mote 2y agoExcept not as a 1:1 trade!
- gregors 2y agoThe idea is that you put in the extra work and he keeps the extra profit. Sounds like a sweet deal.
- HL33tibCe7 2y agoWhen you’re on your deathbed, I’m sure you will be glad that you spent your kid’s taekwondo tournament writing an email so that an e-commerce company could make line go up.
- bayindirh 2y agoSome of these people don't understand that they won't see that tournament again, and they won't be able to make their relationship "line go up" if they don't build the foundations when their children are still children.
- DrillShopper 2y ago> Some of these people don't understand that they won't see that tournament again Just as many don't care or see it as an inconvenience to be there. They might do less damage to their families by just being honest and skipping.
- bayindirh 2y agoIf you don't care about your child's development and be there to support them, then you can just skip having children in the first place, or building a family altogether. There's no shame in being honest to yourself and others, as you said.
- Lammy 2y agoThe child is a future host for the wealth. In this sense the wealth's priorities still come first.
- bigiain 2y agoBut but but! My Personal Brand Consultant told me a beautiful wife with two kids and a late model luxury minivan are essential components of my fundraising endeavors! Round B investments are 27.3% less likely to succeed for single founders under 26 years old compared to founders with families.
- rdtsc 2y ago> At Beacon, we are not going to wait for the next crisis to sweat our biggest challenges; we will build a culture that makes Sweating The Problem our default. Brilliant! I love it. It's Code Yellow all the time. If it worked once a year at Instacart, and Google made so much progress with it, we'll just turn "the Yellow" on all day, every day. Then maybe they can have a Code Red for when it's super-duper important to "sweat" the problem. > you walk away with the confidence that you can handle whatever comes next. You don't even need a bonus, just enjoy your new found sense of accomplishment. And maybe a subscription to the jelly-of-the-month club [1] [1] https://kitchychristmas.com/jelly-of-the-month/ https://kitchychristmas.com/jelly-of-the-month/
- DrillShopper 2y agoPerson who closes the most tickets gets a car Second place gets a set of steak knives Third place is you're fired Always Be (C)losing
- tanseydavid 2y agoCoffee is for closers.
- marcus_holmes 2y agoYeah I thought the same. I would not want to work for this guy or any organisation that thinks like him. It's the manager's job to prioritise. There is always more work than can be done, and always more problems than can be solved with the available resource. Trying to solve that prioritisation problem by "sweating" - ignoring employee's genuine and real need for work-life balance - is not going to work long term. To the author: please learn how to actually manage. Honestly, you'll do much better. I look forward to reading your article about how you learned to manage problems instead of sweating your team into the ground.
- rdtsc 2y ago> Trying to solve that prioritisation problem by "sweating" - ignoring employee's genuine and real need for work-life balance - is not going to work long term. Indeed. Even the words "sweating the problem" just sounds callous and inconsiderate. It can even work for Google, where can pull in different team members at different times. But in a smaller company that means everyone's work-life balance is permanently stressed.
- munchbunny 2y agoIf thing A is important enough to declare a "code yellow" in order to ignore things B, C, and D to focus on A, then were B, C, and D really that important? Could you have focused your team on A from the start, making a "code yellow" unnecessary? (Hint: yes, you could have, so the question should be why you didn't, and whether you could have seen it coming.) I've seen this happen a lot with mediocre leaders. "Code Yellow" equivalents happen because they weren't able to understand that A was really the most important thing, typically because B, C, and D were important for optics or politics, but not genuinely important to the customer or the problem at hand. A "Code Yellow" is a useful political tool to move an organization to focus a bit more on problem solving by saying "I don't care about your politics, your org charts, whatever, just solve the damn problem." In that sense, it really does work.
- charles_f 2y agoThe worst in the story is that thing B, C and D were probably decided, and commendeered by the same exec who, 3 months later will decide that thing A doesn't move enough and "they need to act on that". It's a practical tool to correct their indecisiveness and errors, and who's paying the price for it...
- SpicyLemonZest 2y agoPolitical constraints are real constraints! It's very possible to end up in a state where declaring an emergency is genuinely better than any of the options to avoid it. Consider: * A is clearly important and will have to be "code yellowed" if it's not addressed in the next few months. But you've got to drop one of B, C, or D in order to avoid the code yellow. * B is one of your best engineers' passion project. She's not a big fan of C or D, and she might run away to Google if you tell her they're more important. * C is a promise you made to a customer to unblock some big deal. They might not sign if you tell them you changed your mind and would prefer to work on some projects B and D they don't care at all about. * D is a demand from your VP to address some company priority, and he doesn't think the effort-reward ratio from B or C are very high. If you try to drop D he will probably get mad and tell you that you're not allowed to.
- aftbit 2y agoWow what a toxic attitude. The worst part is acting like this constant crunch time is a normal and reasonable expectation. I'll also call out the derision towards "keeping the lights on" because, of course, keeping the damn lights on is a prerequisite to any kind of growth. Code Yellow doesn't tell people "hey it's not important that the site stay up, just focus on growth instead". It tells people "what you are doing 9-5 to keep the site up is not enough, spend your 5-9 working on this other initiative as well". Evil.
- pavel_lishin 2y agoIt doesn't feel like it means the same thing to OP as it does to the rest of us: > The biggest constraint I have seen teams imposing unconsciously is the ‘keeping the lights on’ fallacy. This manifests as the project team giving the project/goal 25-50% of their attention (although saying it is their main priority) because they feel that they need to continue ‘keeping the lights on’ with other work activities or projects. It sounds like "keeping the lights on" means ... working on other things as well? It doesn't specify that it's to make sure those other things keep working. Maybe it's implied? I have no idea.
- dagss 2y agoI take keeping the lights on to mean, e.g., customer support and fixing bugs an incidents. Not necessarily uptime alone.
- gregors 2y agoThis guy is bad at his job.
- pavel_lishin 2y agoWell, he may be bad at his job, but at least it also sounds like he's bad at being a father.
- MathMonkeyMan 2y agohe did show up
- pavel_lishin 2y agoI shall start engraving the "Bare Minimum" award.
- nasseri 2y agoIs this satire?
- BeefWellington 2y agoI've met people like this and they seem to always have a disaster on their hands. Probably because their companies spend so much time deciding which task should be prioritized and having meetings about it that nobody can get actual work done.
- turbojet1321 2y agoIf everything is an emergency, nothing is an emergency. I don't understand why so many managers fail to grasp this. It's the same with prioritization. I've literally had conversations that go: Manager: I need you to drop everything and do X right now, it's top priority Me: Ok, well I'm currently doing Y, which was top priority this morning. Which is more important, X or Y? Manager: Well, they're both equally important! Me: OK, sure. I can't work on both, which one would you like me to do first? Manager: [uncomfortable thinking noises] Manager: Are you sure you can't do both at once? Me: Yes Manager: [Pause] Keep going with Y. I'll see if someone else can do X sigh
- nine_zeros 2y agoThis dilbert-style management has become very normalized in tech - to the detriment of everyone. It fundamentally comes from a place where there are too many layers of managers, none of who want to make crucial decisions that can bite them back. Aka leadership only in org chart but not in setting direction, leading from the front, or accepting tradeoffs. Ultimately it all boils down to org chart leadership. If managers could be voted out of the org chart by people below them in the chart, they'd have no choice but to show more spine and help the people below. But this is not going to happen in corporations. So the best thing to do is quiet quitting. There is a reason this exists.
- jcgrillo 2y agoIME this is much more prevalent in organizations that hire "nontechnical managers". I personally will never work in such companies again, the experience of having your boss have literally no clue what it is you do for a job is not one I'll sign up for again.
- ern 2y agoI think the problem is a bit harder than that. I've seen a lot of managers who were once technical, but whose last hands-on experience was years, or decades ago. They have a vague idea of what they need to know/do but lack the skills to find things out themselves. and they turn into a major drag on productivity.
- nine_zeros 2y ago> strongly encourages the team to sacrifice the ‘L’ and ‘B’ from Work-Life-Balance. And what is the incentive for the rank and file to do this? Execs get millions of dollars worth of stock every 3 months. The rank and file isn't getting this. Why would the sacrifice their L and B? To keep their jobs? Naw. For this, they will cut corners and lie. Management practices have become so terrible in tech. It is no wonder that the best people keep switching jobs every few years. There is zero incentive to work under such poor management practices.
- readthenotes1 2y ago"During a code-yellow, a leader can escalate a project/situation to a war room situation, pulling people out of their day-to-day work to focus entirely on the problem at hand. " In one of those books management gurus right at the end of their workspan, one of them wrote that he did not believe a project was worth doing if it was not worth having a war room in which to focus people on the problem at hand. Lowering Work In Progress is one of the easy ways to gain success...
- jacknews 2y ago"we will build a culture that makes Sweating The Problem our default." What a douche. So it's always code-yellow, and now they'll need to invent a code-red.
- avidiax 2y agoIf you are really sweating the problem, you are sweating only one problem. But an organization usually has more than one problem that needs to be addressed concurrently, unless it wants to constantly be "fire fighting" problems that could have received partial attention earlier but instead have become emergencies. So this guy is going to want to sweat more than one problem, which really just means doing a normal job of prioritizing things, but with more sweat. Just remove the L & B from WLB, permanently. Sort of like this boss imagines he does by writing emails at his family's events, but without the extreme upside potential afforded to executives.
- __turbobrew__ 2y agoYou exit from the president of Instacart and then spend Saturdays at your son’s tae kwon do practice coercing subordinates to give up their life?
- lizzas 2y ago24/7 oncall to ... be yanked onto something the boss fancies. No thanks. What about... plannning?
- habosa 2y agoThe apostrophe placement was almost as bad as the content. No thanks.
- CalRobert 2y ago“ found myself sitting at my kids Tae Kwon Do competition (which was running 3.5 hours late!) on a Saturday writing an email to the team ” Maybe if you work hard enough you won’t need to stay at such a bad job
- steinuil 2y agoI don't understand which part of this requires the team to "sacrifice the ‘L’ and ‘B’ from Work-Life-Balance". If you want to encourage your team to re-evaluate their decisions and tackle big problems maybe you should build a culture around that rather than forcing everyone to be in emergency mode? I think that's what this post is saying, but all the posturing about forcing people to working harder to build "grit" is making it hard to read it as genuine. Clearly this is not a management issue, it's the team's fault for not having enough grit to work hard all the time!
- singleshot_ 2y ago"2019: Building our fourth senior executive team in six years to lead us to the next plateau of scale" Valuable lessons from a competent person. Great article!
- snapcaster 2y agoYeah this seems insane to me, I don't understand how 4 executive teams in 6 years is a sign of anything but chaos and dysfunction
- AIorNot 2y agojust like layoffs, this just part of the tech culture, as long as we reward managers with promotions for riding their teams (and no doubt I'm sure those middle managers are working their ass of too) this culture doesn't change. in many cases deep financial bets have already been made before the code is even written or the idea properly sounded out. culture is culture.. human desire to appease the social order is stronger than rational decision making
- rhapsodic 2y ago[dead]
- closeparen 2y ago>Under the surface constraints, the ones which are imposed almost unconsciously, are the most pernicious. This is true and important. For example, no matter how critical a project may be, it is understood that the first duty of any white collar worker is to show up to every meeting on their Google Calendar. When we are are allocating, prioritizing, and estimating resources for projects, what we are really talking about is the time left over after discussing ~anything that ~anyone may want to pull them into. Only an executive whose calendar is kept by an EA subjects meetings to the kind of cost-benefit tests and prioritization that we use for IC project bandwidth.
- kittikitti 2y agoCorporate mind games exposed!