4 ms·

> Okay. Well, I can keep butting my head against this wall. I had conversations with that manager where I was told in literally so many words, ‘You’re too ideal

> Okay. Well, I can keep butting my head against this wall. I had conversations with that manager where I was told in literally so many words, ‘You’re too idealistic. You don’t care enough about the bottom line. You should change your values.’ And I was like, no, nope, that’s not how this is going to work, man. Uh, outside I did the politic. To me that's the most interesting quote of the podcast and that's the impression I had gotten before reading this quote. Sounds like they received valuable feedback along the way and ignored it on purpose. My career lesson is that being right isn't the hard part of being a senior staff engineer. Being right isn't enough. It's building alignment across the entire org towards the right solution. That's the hard valuable problem that I'm paid to solve. I worked on the facebook.com rewrite to React in 2019. I find this story particularly interesting.

atomicnumber3 3y ago
I mean, yes and no. I find staff engineers, and especially the senior-staff-engineer-type positions, to be the ones most sensitive to the exact specifics of the org they've joined. I've seen a senior staff engineer (at a popular unicorn you've all heard of) who was a very smart, but also friendly and reasonable person (how often are those traits in opposition of each other!). They were very good at the staff engineer game. They were vouching for us to upgrade a framework that we pumped 50M of spend through per year. From v2 to v3. It wasn't a huge change, not like python 2 to 3 at all. Positively trivial compared to py2 to 3. But management just were not up for it at all. Even with research done showing we could expect literally a baseline of 10% improvement in performance (so you do the math on that vs the 50M of spend), they just did not want to "waste time on version upgrades." The staff engineer was told they were being unreasonable in pushing so hard. Their manager tried to ditch them, but again, they're good at the staff eng game, so they switched teams due to being in the good graces (due to an excellent track record) with a middle manager above them. They decided, hey fuck you, I'm going to one man army this effort. They did, they had preview versions ready in less than a month and had migrated some of the bigger-win jobs within 2, saving their annual comp (which was prodigious) several times over just in that. Suddenly everyone was keen on moving their stuff over now that the startup cost (both political and eng-time) was paid. Anyway fast forward a year, the rollout is complete and nearly the entire fucking management chain above him has turned over, a 50/50ish mix of firings vs quittings. But he's still there and the upgrade migration is complete. So sure, sometimes staff engineers have a stick up their bum and can't dislodge it enough to get work done on the bottom line. But sometimes they're the only sane ones in a crazy world, and you can only do so much to try to turn a balky mule around.
- slimsag 3y ago
  One way to read your story is that upper management applied heavy downward pressure, to the extent that relatively trivial changes in the organization were discussed as 'costly' - in an effort to get engineers like the senior staff engineer in your example to pull heroic feats of engineering in a few weeks all by himself, instead of using a small group of engineers / resources. I think your argument is that 'it worked for him' because he is still there while the managers are not, but reality may be the managers were there to have that effect in the first place and the 50/50 turnover was because they were in the thick of it. That'd be my cynical take, anyway.
  - ytx 3y ago
    Especially if he wasn't rewarded commensurately for that feat.
  - atomicnumber3 3y ago
    "upper management applied heavy downward pressure" Correct "I'm an effort to get engineers like the senior staff engineer to pull heroic feats" I think this gives them too much credit. The reality, imo, is that the org had a pathological emphasis on "impact" and ran performance reviews so tight that everyone, managers included, were so scared of being PIPd and losing their grants that they were extraordinarily risk averse in planning and picking projects to bet on. And leadership in general undervalued "infra-y" changes as mostly make-work. The turnover was partly because of this general effect - people scared, not betting big or smart enough, and optimizing for holding on / not getting fired. And to their credit, most of these people vested 2-3 years of their 4-year grant before it all fell over, so they made out decently lucratively, so maybe they're the real winners in all this after all.
    - slimsag 3y ago
      That's an interesting take, thanks for writing that. That gives me something to think about.
- fmbb 3y ago
  I’m not American so I don’t know what a staff engineer does, and googling the term is not helping much. I have also not worked in any software company that has “engineering teams” which would maybe have helped me understand the Google results. What you are saying is that sometimes a senior engineer manages to make good things happen? Did this person get rewarded for this huge effort they put in? You say they already have prodigious yearly compensation, was it at least doubled given that they saved it several times over?
skeeter2020 3y ago
you gotta a do what you gotta do, but if you have the luxury, and conceptual integrity, there's going to be sacrifices in the name of "alignment" that you're just not prepared to make. I agree with your positioning on lots of "right" solutions, IME it's recognizing and understanding and still not accepting when it violates your foundational values. I've been there and had the ability to not participate, but this isn't always the case.
- bgirard 3y ago
  You're exactly right and to me that's part of finding the 'right' solution for the organization. Sometimes you compromise for 'alignment', and heck sometimes you discover that those compromises were a better solution all along. In other cases you chose to 'pick a battle' and find the best way to navigate and get buy in. Because again being right -and getting buy in- is the important part. Not later giving everyone an 'I told you so'.
- choppaface 3y ago
  Separate Church and Business. Get paid while making a meaningful contribution. Use the time outside of work to live the values you do not otherwise monetize. Critically: Have Time Outside of Work.
samstave 3y ago
Have you ever worked with a BOFH? BOFHs suck. Your career lesson is correct.
- wizzwizz4 3y ago
  I have never worked with a BOFH. You can tell this because I am still alive, have not been convicted for somebody else's fraud, and have never been defenestrated. Doing the right thing isn't a known characteristic of a BOFH.
  - samstave 3y ago
    Windows LOVE YOU. -- linus, not so much.
chriskrycho 3y ago
There's an element of truth to this! I alluded to this (and Adam totally reasonably cut even more of it for time) but I definitely was not as effective organizationally as I could have been; that is one of the major things I learned from my time at LinkedIn. The biggest challenge for DX teams remains figuring out how to (a) communicate but more importantly (b) align their work with the key priorities of the business. I got to be okay at (a) but never particularly succeeded at (b) in my time at LinkedIn. Some of that is on me! Some of it… is on LinkedIn. That said, in this case, the person telling me “You’re too idealistic” literally meant it in the sense of “You should not care about anything unless it directly moves the bottom line,” and I reject that to my bones. The bottom line matters. So, though, do things like user experience, developer experience, and for that matter just basic ethics about what we build (though happily the latter wasn’t in view here).
- bgirard 3y ago
  > “You should not care about anything unless it directly moves the bottom line,” and I reject that to my bones I have a lot of empathy for that feeling. I've come across this often in my career and I have not enjoyed it. I enjoy building software that I'm proud off. Thank you for sharing your story!
ejb999 3y ago
>>My career lesson is that being right isn't the hard part of being a senior staff engineer. Not only that, but your own definition of 'right' may or may not align with what is 'right' for the people paying your salary - pretending otherwise is just silly. In any organization, you advocate for what you believe is 'right' the best you can, someone else (or a consensus of others) will likely decide if they agree - you get to decide if you are willing to live with it/compromise, or move on - I have done both in my career.
choppaface 3y ago
A great choice quote, and add to that LinkedIn as an engineering culture already heavily skews towards “craftsmanship” (as they call it) and refinement versus the larger industry. The speaker appears ridiculously unaware of how outlier his position is. Do you want to wax the wheels of a title-winning race car or of a 1990 Toyota Camry that happens to be in mint condition? A lot of LinkedIn engineers I’ve met are the latter, and Microsoft is trying to push them into being the former. If you are The One True Master Wheel Polisher then sure maybe you get to choose your car, but don’t soapbox it. The audience is smarter than that (I hope).
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
wlll 3y ago
> To me that's the most interesting quote of the podcast and that's the impression I had gotten before reading this quote. Sounds like they received valuable feedback along the way and ignored it on purpose. We don't really have enough context to make a judgement here IMO. I've been in places large enough to have fairly complex politics where people would manouver themselves into better jobs as much as they could, including performing a hostile takeover of another department. They might not have had the best ideas or the best plans, but they did have the right connections, go to lunch with the right people and say the right words. I've seen bad decisions made because more persuasive and better connected people were attempting to climb the ladder and managed to talk management into it. "You’re too idealistic. You don’t care enough about the bottom line. You should change your values." could just be something someone says when they're trying to smear someone in the way, especially if that's how they've sold themselves and their ideas to upper management. Personally I've seen and done a number of large scale changes/upgrades (mostly Ruby/Rails/Postgres things) in orgs smaller than Facebook but still with hundreds of engineers and large codebases, both stop the world, and where the work was streamed seamlessly into regular feature etc. work and the methodology outlined by Chris seems incredibly sensible and aligns with what I have found to be the most successful way of doing these things. As the host says, we ony have one side of the story here, but at least a lot of the things Chris is saying seem to make sense. > My career lesson is that being right isn't the hard part of being a senior staff engineer. Being right isn't enough. It's building alignment across the entire org towards the right solution. That's the hard valuable problem that I'm paid to solve. I agree, leadership positions require trust and respect to be effective. You still have to be right for that to be useful of course. Progress in the wrong direction isn't really progress.

atomicnumber3 3y ago

I mean, yes and no. I find staff engineers, and especially the senior-staff-engineer-type positions, to be the ones most sensitive to the exact specifics of the org they've joined. I've seen a senior staff engineer (at a popular unicorn you've all heard of) who was a very smart, but also friendly and reasonable person (how often are those traits in opposition of each other!). They were very good at the staff engineer game. They were vouching for us to upgrade a framework that we pumped 50M of spend through per year. From v2 to v3. It wasn't a huge change, not like python 2 to 3 at all. Positively trivial compared to py2 to 3. But management just were not up for it at all. Even with research done showing we could expect literally a baseline of 10% improvement in performance (so you do the math on that vs the 50M of spend), they just did not want to "waste time on version upgrades." The staff engineer was told they were being unreasonable in pushing so hard. Their manager tried to ditch them, but again, they're good at the staff eng game, so they switched teams due to being in the good graces (due to an excellent track record) with a middle manager above them. They decided, hey fuck you, I'm going to one man army this effort. They did, they had preview versions ready in less than a month and had migrated some of the bigger-win jobs within 2, saving their annual comp (which was prodigious) several times over just in that. Suddenly everyone was keen on moving their stuff over now that the startup cost (both political and eng-time) was paid. Anyway fast forward a year, the rollout is complete and nearly the entire fucking management chain above him has turned over, a 50/50ish mix of firings vs quittings. But he's still there and the upgrade migration is complete. So sure, sometimes staff engineers have a stick up their bum and can't dislodge it enough to get work done on the bottom line. But sometimes they're the only sane ones in a crazy world, and you can only do so much to try to turn a balky mule around.

slimsag 3y ago
One way to read your story is that upper management applied heavy downward pressure, to the extent that relatively trivial changes in the organization were discussed as 'costly' - in an effort to get engineers like the senior staff engineer in your example to pull heroic feats of engineering in a few weeks all by himself, instead of using a small group of engineers / resources. I think your argument is that 'it worked for him' because he is still there while the managers are not, but reality may be the managers were there to have that effect in the first place and the 50/50 turnover was because they were in the thick of it. That'd be my cynical take, anyway.
- ytx 3y ago
  Especially if he wasn't rewarded commensurately for that feat.
- atomicnumber3 3y ago
  "upper management applied heavy downward pressure" Correct "I'm an effort to get engineers like the senior staff engineer to pull heroic feats" I think this gives them too much credit. The reality, imo, is that the org had a pathological emphasis on "impact" and ran performance reviews so tight that everyone, managers included, were so scared of being PIPd and losing their grants that they were extraordinarily risk averse in planning and picking projects to bet on. And leadership in general undervalued "infra-y" changes as mostly make-work. The turnover was partly because of this general effect - people scared, not betting big or smart enough, and optimizing for holding on / not getting fired. And to their credit, most of these people vested 2-3 years of their 4-year grant before it all fell over, so they made out decently lucratively, so maybe they're the real winners in all this after all.
  - slimsag 3y ago
    That's an interesting take, thanks for writing that. That gives me something to think about.
fmbb 3y ago
I’m not American so I don’t know what a staff engineer does, and googling the term is not helping much. I have also not worked in any software company that has “engineering teams” which would maybe have helped me understand the Google results. What you are saying is that sometimes a senior engineer manages to make good things happen? Did this person get rewarded for this huge effort they put in? You say they already have prodigious yearly compensation, was it at least doubled given that they saved it several times over?

skeeter2020 3y ago

you gotta a do what you gotta do, but if you have the luxury, and conceptual integrity, there's going to be sacrifices in the name of "alignment" that you're just not prepared to make. I agree with your positioning on lots of "right" solutions, IME it's recognizing and understanding and still not accepting when it violates your foundational values. I've been there and had the ability to not participate, but this isn't always the case.

bgirard 3y ago
You're exactly right and to me that's part of finding the 'right' solution for the organization. Sometimes you compromise for 'alignment', and heck sometimes you discover that those compromises were a better solution all along. In other cases you chose to 'pick a battle' and find the best way to navigate and get buy in. Because again being right -and getting buy in- is the important part. Not later giving everyone an 'I told you so'.
choppaface 3y ago
Separate Church and Business. Get paid while making a meaningful contribution. Use the time outside of work to live the values you do not otherwise monetize. Critically: Have Time Outside of Work.

samstave 3y ago

Have you ever worked with a BOFH? BOFHs suck. Your career lesson is correct.

wizzwizz4 3y ago
I have never worked with a BOFH. You can tell this because I am still alive, have not been convicted for somebody else's fraud, and have never been defenestrated. Doing the right thing isn't a known characteristic of a BOFH.
- samstave 3y ago
  Windows LOVE YOU. -- linus, not so much.

chriskrycho 3y ago

There's an element of truth to this! I alluded to this (and Adam totally reasonably cut even more of it for time) but I definitely was not as effective organizationally as I could have been; that is one of the major things I learned from my time at LinkedIn. The biggest challenge for DX teams remains figuring out how to (a) communicate but more importantly (b) align their work with the key priorities of the business. I got to be okay at (a) but never particularly succeeded at (b) in my time at LinkedIn. Some of that is on me! Some of it… is on LinkedIn. That said, in this case, the person telling me “You’re too idealistic” literally meant it in the sense of “You should not care about anything unless it directly moves the bottom line,” and I reject that to my bones. The bottom line matters. So, though, do things like user experience, developer experience, and for that matter just basic ethics about what we build (though happily the latter wasn’t in view here).

bgirard 3y ago
> “You should not care about anything unless it directly moves the bottom line,” and I reject that to my bones I have a lot of empathy for that feeling. I've come across this often in my career and I have not enjoyed it. I enjoy building software that I'm proud off. Thank you for sharing your story!

ejb999 3y ago

>>My career lesson is that being right isn't the hard part of being a senior staff engineer. Not only that, but your own definition of 'right' may or may not align with what is 'right' for the people paying your salary - pretending otherwise is just silly. In any organization, you advocate for what you believe is 'right' the best you can, someone else (or a consensus of others) will likely decide if they agree - you get to decide if you are willing to live with it/compromise, or move on - I have done both in my career.

choppaface 3y ago

A great choice quote, and add to that LinkedIn as an engineering culture already heavily skews towards “craftsmanship” (as they call it) and refinement versus the larger industry. The speaker appears ridiculously unaware of how outlier his position is. Do you want to wax the wheels of a title-winning race car or of a 1990 Toyota Camry that happens to be in mint condition? A lot of LinkedIn engineers I’ve met are the latter, and Microsoft is trying to push them into being the former. If you are The One True Master Wheel Polisher then sure maybe you get to choose your car, but don’t soapbox it. The audience is smarter than that (I hope).

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]

wlll 3y ago

> To me that's the most interesting quote of the podcast and that's the impression I had gotten before reading this quote. Sounds like they received valuable feedback along the way and ignored it on purpose. We don't really have enough context to make a judgement here IMO. I've been in places large enough to have fairly complex politics where people would manouver themselves into better jobs as much as they could, including performing a hostile takeover of another department. They might not have had the best ideas or the best plans, but they did have the right connections, go to lunch with the right people and say the right words. I've seen bad decisions made because more persuasive and better connected people were attempting to climb the ladder and managed to talk management into it. "You’re too idealistic. You don’t care enough about the bottom line. You should change your values." could just be something someone says when they're trying to smear someone in the way, especially if that's how they've sold themselves and their ideas to upper management. Personally I've seen and done a number of large scale changes/upgrades (mostly Ruby/Rails/Postgres things) in orgs smaller than Facebook but still with hundreds of engineers and large codebases, both stop the world, and where the work was streamed seamlessly into regular feature etc. work and the methodology outlined by Chris seems incredibly sensible and aligns with what I have found to be the most successful way of doing these things. As the host says, we ony have one side of the story here, but at least a lot of the things Chris is saying seem to make sense. > My career lesson is that being right isn't the hard part of being a senior staff engineer. Being right isn't enough. It's building alignment across the entire org towards the right solution. That's the hard valuable problem that I'm paid to solve. I agree, leadership positions require trust and respect to be effective. You still have to be right for that to be useful of course. Progress in the wrong direction isn't really progress.