10 ms·
How I operated as a staff engineer at Heroku
- etimberg 6y agoWow! As someone who has one of these nebulous titles ("Software Architect"), this is one of the most accurate descriptions I've seen of what the job entails. My employer is nowhere near the size of Salesforce, so a lot of things scale down, but the basics are the same. I would say the biggest challenge with roles like this is the loneliness and what seems like constant conflict. One needs to work hard to keep up personal relationships with product/sales/project management/etc to ensure that the level of trust is there to be able to have the hard conversations. It is really easy to forget that, and a lot of those other folks start to feel like you're always just shooting down their ideas.
- rarecoil 6y ago> One needs to work hard to keep up personal relationships with product/sales/project management/etc to ensure that the level of trust is there to be able to have the hard conversations. In the heat of the moment, on deadline with a bunch of bugs and yet multiple compliance hurdles to clear, with a short-staffed team and conflicting management priorities, it's pretty easy to lose sight of the common goal. Microsoft has a nice cliche you can use when you catch yourself on the adversarial end of this type of conversation: "assume positive intent". It's written on most of the Redmond campus whiteboards. If we are all assuming positive intent, it can give us the right empathetic mindset with which to build this trust even when it seems difficult. I've found working with other teams through the hard conversations with compromise instead of "no" leads to better results. They're assuming positive intent as am I and it's OK. If I'm feeling something is adversarial I need to understand why.
- jaggederest 6y agoI like to frame it as "not me versus you, us versus the problem" when things get too heated. Otherwise things degrade into ad hominem "Those goddamn X people", instead of "We're having a lot of problems getting our project to work with X"
- rarecoil 6y agoI like this one, too. The moment the people are your adversary in a technical discussion, you have probably lost sight of the actual problem. Much of the time I find it's misaligned incentives or, as Amy wrote here, sometimes it's people working with obsolete context or old grudges. Reframing yourself first, and then the discussion, is very much the way to succeed.
- sdbrown 6y agoMy partner is a physician, and this is _exactly_ how they manage disagreements about treatment plans, almost by instinct. It is amazing how well this works at clearing things up.
- koolba 6y agoI do not know this person at all but after reading this I think I’d enjoy being her coworker.
- amerine 6y agoAs someone who worked with Amy for years at Heroku, I can confirm that this is very accurate. Amazing write up, Amy. Damn.
- vishvananda 6y agoLess years, but I concur.
- kenhwang 6y agoSo glad the term "hopes and dreams" was used. That's exactly how I describe half my staff level role. I move mountains to enable and materialize "hopes and dreams", or I destroy them without a sliver of doubt, in equal parts. That means doing at least all the things the post describes. But the other half was only lightly touched on. Staff is as much spearhead as it is rearguard. So that means caring for legacy codebases, aging tech, and anything else that's falling behind. It's search and rescue and postmortems after things have gone catastrophically wrong. Great great write up!
- jrott 6y agoThis is really good. It's one of the best descriptions I've seen of what being a staff engineer is like. When I came into the role I was stunned at the amount of time that I ended up talking to people and just trying to figure out what is going on.
- georgeoliver 6y ago>Everybody wants to hire senior engineers; nobody wants to make them. Gold.
- humanlion87 6y agoI have been struggling with this right now. I am not good enough nor experienced enough to figure out things on my own (to get to senior engineer). I feel I just need some guidance or push to get to the next level. But it's so discouraging to get no support from my management. And other folks around me also keep saying that depending on management to support me in my career growth is not a good idea. I feel it's ridiculous and distorts my image of the role of management. I just have to accept that good managers/management is hard to find. Anyway, it's kinda good to know that it's not just my company/team that seems to have this issue.
- scollet 6y agoSounds like you need some meat (analogy aside of dietary preference) to sink your teeth into. It's not a push because a push comes from behind. It's you grabbing onto something elevating and interesting because it's juicy. A reasonable senior will recognize your talent, but a stagnant organization will simply extract your current value without pulling you up with that bite. I know the analogy is bit inhumane, but the human part comes from those very human people who just want to solve problems. Whether they be lower in pay/'rank' or higher, find the people who appreciate your strengths immediately. Don't ever be afraid to express your ideals in your capabilities and learning process. In any organization from 10 to 10,000 you will find those folks who can take a genuine appreciation for your skillset and apply it. The failure state should always be: "I'm departing to company Y, I wish you the best." The success is an expression of your ideals. The 'meat' of my argument is - at your level - don't look for the push, find the pull. You've got this. Edit: an -> and (Para. 5)
- oxfordmale 6y agoIn my experience, management will generally not care about your career development and will prefer to hire a senior engineer from outside to avoid internal conflicts. The best way to progress your career is to train yourself up externally. For example, is there an open source project you would like to contribute to ? It is also worth to attend tech group meetings in your area of expertise. As everything is done remotely now, this has become even easier.
- orliesaurus 6y agoDamn really nice read! I wonder: is this what enterprise software engineering really feels like up there in the high ranks? I don't know if I would ever enjoy doing any of what she listed!
- cmwelsh 6y agoI had a similar reaction but there are a few archetypes that are very similar to the duties of an IC. She mentions “Deep Diver” and that’s similar to my current role. I enjoy having systems thrown onto my lap and becoming an expert in them as quickly as possible. Then the organization uses my analysis to decide what to do next: keep it or pivot to a different strategic solution in the long-term. Ideally my deep dives result in useful documentation, code improvements, or process automation to allow the future deep diver to come up to speed quicker than I did.
- spdionis 6y agoAs a staff engineer, you have multiple responsibilities you need to cover, and you can delegate some of them. The reality is that in most teams you can delegate the "Deep Diver" responsibility most of the time. "Context sharing" and "Coaching" not so much. So yes, as some comments in other threads mention, your coding/hard tech contributions diminish with time.
- daxfohl 6y agoI think it is. In my younger days I wanted to rise up so that I could work on really complex and challenging code. But now I'm there and I hardly code at all. Most of it is working across teams to figure out dependencies, blockers, security concerns in launching a new project. About a year in, I still don't like it much, nor do I feel like I'm good at it, even though I consider myself a good coder. I asked my manager for a transfer to a more code forward role and apparently they really don't exist at the high levels. There are some exceptions if you are a known expert in some field, but for a generalist in optimizing queries or caching patterns, there's usually someone cheaper and younger they can get to do the work.
- moron4hire 6y ago... What is a staff engineer?
- kenhwang 6y agoThe level above lead/senior.
- noir_lord 6y agoNot every org has them though depending on size, I’m a lead who does a lot of what she describes and there is no level beyond mine that isn’t pure management (which I don’t want if I’m honest). When it comes down to it my role is a lot of ad-hoc dealing with bun fights before they come actual fires.
- aprdm 6y agoThis is truly great! Work in an organization with ~200 sw engineers and some of it is pretty accurate to my role as well.
- spenczar5 6y agoWhat a great writeup. A lot of it sounds very familiar to my experience as a principal engineer at Amazon. There’s a certain point at which you realize the hardest software engineering problems for a large company are social problems, and that’s what you devote most of your energy to as a PE (or, I guess, a Staff Engineer). Personally, I really didn’t have the stomach for the work, eventually; I have gone back to active programming, this time for scientific projects, and I am much happier.
- beams 6y agoGreat questions and answers. Admittedly, I can’t help but feel there aren’t very many pain points mentioned, which makes me ask what’s being left out. Additionally, how often was coding actually involved in this role? I hear this claim often but in my experience it’s rare. One-line fixes don’t count. Specifically, what projects were you a code contributor to?
- bradlys 6y agoI'm going to guess quite infrequently based on this first sentence of how she spends her time. > While I do write code from time-to-time, it’s only after I’ve delivered on my obligations on these four functions.
- kenhwang 6y agoFor me, it's either the core or foundational part of the most important projects to the company, or, one liners followed by a paragraph of comments in some tricky/obscure part of a codebase no one has looked at in a while.
- User23 6y ago>> How do you keep in touch with how things really work as you spend less time on hands-on development? > As someone who still does write code occasionally, this is something I struggle with. Our codebase shifts underneath me constantly, in ways I often don’t see. It has been my goal for the year to make sure I say “I’m not sure, let me check” even when I feel confident. This is not at all consistent with Staff Software Engineer at any of the several Big Tech Companies You've Definitely Heard Of I've worked at. It would be more of a Distinguished Engineer level sinecure[1]. Staff is usually expected to spend at least 50% of their time actually writing software. [1] Not pejorative. If I were the relevant hiring manager I'd be happy to let someone like, say, Leslie Lamport, to pick a name out of the hat, spend as much or as little time as he felt like writing software just to have him around.
- seanmcdirmid 6y agoIsn’t Leslie Lamport a distinguished researcher, not a distinguished engineer? Research tracks are fairly different from engineering ones.
- spenczar5 6y agoDefinitely extremely consistent with Amazon’s Principal Engineers. Internal surveys on time spent coding support it.
- YorickPeterse 6y agoThis is also how we do Staff Engineering at GitLab: you're expected to spend most of your time writing code. What changes is how you go about doing that, how you communicate things, what responsibilities you have, the expectations of you, etc.
- fizixer 6y agoSolver/Deep diver FTW All the way.
- selfinvariant 6y agoThis article is incredibly well written and rings true to me and my experience being in a similar role / working with people in such roles. I like the notion of articulating a career path in tech as of: 1) Learning the ropes 2) Being a producer / writing code / driving design decisions 3) Being a multiplier - working across different teams to unblock / amplify other peoples work 3) is pretty much communication centric, while 2) is deep knowledge of tech involved There is a lot of nuance to that of course (which I think this article captures well within specific organization). I'm curious if this is something any one else has the same experience with
- throawayyyy 6y agoSo here's a curious question that is perhaps incredibly naive and seemingly impolite, hence the throwaway, but I really don't know, which is why I'm asking. Why must one be a senior engineer in tech before they can become a staff engineer? To me, this seems much more like a communication role. Moreover, the tech that is needed to know is at a high level. Is it really needed to know the technical finer details of systems when high level software architecture and communication seem to be 90% of the whole role? Or put in a more blunt, albeit unrefined, way: why can't someone who specializes in communication become a staff engineer? I mean people like consultants. They get so much experience with communication and high level tech (some of them at least) that this seems to be what they'd be good at.
- bcherny 6y agoIt takes a lot of experience to build the intuition to make the right technical tradeoffs, choose the right architecture, escalate staffing gaps, forecast timelines, rope in the right folks, etc. It’s not something you can learn in school, and there aren’t any shortcuts.
- deleted 6y ago[deleted]
- endymi0n 6y agoYour job isn‘t just to coordinate people, but help them reach consensus. If there‘s a tie, that decision is yours to make. So: 1. You have to keep writing code to remain an effective tiebreaker 2. You still need to be a badass engineer in order to hire and keep exceptional talent who‘s looking up to you for mentorship. Respect and excellence are non-negotiable ingredients in a working meritocracy. Sure you can also work outside of a meritocracy, but results and engineering happiness/productivity is usually subpar. So that‘s exactl why the hunt is on for those unicorn engineers who can also socialize: It‘s proven to work.
- necubi 6y agoThere is a role like you're describing at some companies, often called a technical program/project manager. They're responsible for managing schedules, communication, etc. The difference with a staff engineer is that TPMs are not expected to be technical experts, while a staff/principal engineer is. So they're not just routing information between stakeholders, they're also synthesizing and evaluating, partly responsible for the project going well and empowered to make changes they think necessary. All of that takes a lot of experience in the trenches, having seen many projects succeed and fail. It also requires a lot of trust and respect, which takes experience to cultivate.
- delete_more 6y agoA thought I had recently is that staff/principal engineers who got their position by long tenure (not hired recently) should delete more code. Deleting code is actually one of the most difficult things to do in a project. To do it properly and without risking breaking things, requires not only vast and detailed knowledge of the system, but also knowing the context and decision around when the feature was added. A long tenure engineer is the perfect fit for this. They know how to surgically remove the useless code. They remember all the hopes they had for the feature and when it failed to produce those results. They know well that code is not free, it's an ongoing cost to maintenance and complexity. This is not easy work to give to a junior. It's not even for senior engineers because they often lack the context and broad system knowledge. Only the long tenure staff engineer can confidently say, that feature failed, and I can remove it without breaking anything. If someone brings up sunken cost fallacy and how some users use the feature, only the long tenure staff has the background, respect, and authority to say, no this feature failed, it has 1% of the results we wanted, it's not worth the ongoing maintenance burden. A staff engineer is supposed to a team multiplier through code. People often think of designs, refactoring, helpful tools, architecture discussions and documents. And it's true, these are all code related things that can multiply a team's efforts when done well. But what better way to multiply a team's productivity than to reduce the size of the system? Think of how green field projects start. They feel so fast and progress is so quick. Then the features come, more and more, slowing down development with expanding system complexity and maintenance of code. How much faster and quicker can the team move after the staff engineer has reduced the system by 50%? It's like turning back the clock on an aging system. Suddenly it feels years younger, able to move quickly again, but this time with the right features in place from the experience of past years. And the team is happier, less useless code to wade through and understand, less strange bugs caused from code that supports a useless feature.
- kenhwang 6y agoIt's never actually useless code, it's the code that is _almost_ there. Then there's the whole political dance where one team still uses it because the replacement isn't quite the same and now you're solving some obscure use case and prioritizing the work gets done. Plus, it's likely the original author is still present so you have to placate them too. I definitely take on the grim reaper role and make deletions happen. But it does take all the skills in the post.
- deleted 6y ago[deleted]
- unixhero 6y agoI really liked the perspectives on the four obligations every day. Planning Context sharing Coaching & Culture
- sabujp 6y agoWhat is the salary incl bonus for a "staff engineer" at heroku? I find that titles are meaningless and in the end it also boils down to show me the money.
- scollet 6y agoTrade C's like lunchroom fruit snacks before the VCs get here.
- nfriedly 6y agoHeroku is now part of Salesforce. They don't appear have an official Staff level, but per [1] the levels above Senior are Lead at ~$287k (total comp), Principal at ~$378k, and Architect at ~$557k. [1]: https://www.levels.fyi/?compare=Salesforce&track=Software%20Engineer https://www.levels.fyi/?compare=Salesforce&track=Software%20...
- secondcoming 6y agoUS salaries are insane
- nfriedly 6y agoYes, although to be fair, that's on the high-end even for San Francisco. Total compensation in the range of $100-200k is much more common.
- yibg 6y agoThat's not right. Base compensation of 100k - 200k for engineers is more common, and even that is usually for up to "senior" engineers or so.
- sabujp 6y agounfortunately those are low, bay area FAANG (incl bonus + stock): junior: 200-250 mid junior : 250 - 300 senior: 300-350 staff: 450-500 principal: 600-700 director: 800 - million+ vp: millions
- paganel 6y ago> If we made the wrong bet, some blame is with product, but it’s also on you, and the manager probably won’t look great. Why would any manager in his/her right mind choose to take any major risks while working for a big company? Maybe I'm looking at it wrong, but "looking not great" because you failed makes sure that next time you'll have to make the choice you'll always going to pick the safe bet. And safe bets aren't good for any technological company long-term.
- shankhs 6y agoI am with you on this. There is some positive incentive if the technical manager gets the product right maybe a strongly exceeds for a year. But if the product fails TLM gets equal share of negative press and a low rating which means greatly low bonus and bad reputation which will make hiring even more difficult. I don't see any reason why a TLM will take any major risks while working in a large company.
- spdionis 6y agoReally good article. Context sharing is really powerful. Your job as a staff engineer involves going into a cross-team meeting and explaining in technical detail a given piece of work, thanks to which your/another team re-estimates a given ticket from 5 points to 1 point. Those 4 points of difference are your hard work, done over the course of a 30-60 minute meeting. This is probably what people mean by "force multiplier".
- ChrisMarshallNY 6y agoGreat write-up. Not sure I'd want the job. Sounds like she was a good fit for it. I was a manager for most of my career, and spent most of that time pining for the fio- er, my days as a coder. So, after I left my career position, I reverted back to a coder. It's been nice.
- sciurus 6y agoIf you enjoyed this, check out https://staffeng.com/ https://staffeng.com/ It's composed of interviews with Staff-plus engineers as well as guides for reaching and succeeding at Staff-plus roles
- nojvek 6y agoI love this site. Thanks for sharing.
- pier25 6y agoFor the past years it seems Heroku has kinda stagnated. When was the last time they announced a new feature? It seems all the talent is focused on Salesforce.
- sethammons 6y ago> You may be furthering existing misconceptions or old grudges that really should die off. Actively invest in meeting new folks as a counterbalance. This one hits home. I've been at a rapidly growing org for nearly a decade and have grown a massive amount in turn. Things that I thought were the case sometimes ceased being so, literally, years ago.