5 ms·
I've worked at 8 different companies and none of them could even hold a candle to Google's pace of work. Everything just works. The quota system works, the iden
by jeffbee 6y ago
I've worked at 8 different companies and none of them could even hold a candle to Google's pace of work. Everything just works. The quota system works, the identity and authorization system works, observability works, the build and release system works. You can be a productive engineer at Google on day 2 if you read the codelabs on day 1. Nobody at Google has ever needed to have their manager email the Jira admin to get them added to the right group to edit tickets. Nobody at Google has a need to raise a ticket with some ops group in Bengaluru to partition a Kafka topic, renew a certificate, bridge two VPCs, or any of that type of thing. It's almost frictionless. I don't get people who say that Google's pace is slow. I've definitely worked in startups where some kid thought they were ultra-productive with their late-night merges of 20000 lines of untested code, but on longer time scales those startups inevitably ground to a complete standstill under the weight of that debt.
- gresrun 6y ago5+ yrs @ Google, Google is my 5th company. Google has all the building blocks for great backend services and front-end development and, if you know where to look and have some experience with them, you can build a rock-solid product in <6mos, also assuming you have a team that can execute and the political will to ship it. Politics/consensus building is where the real roadblocks lie in Google, and presumably other large companies. Trying to make high-level product & technical decisions when you have 10 stakeholders with 3 VPs, all in different orgs, is serious exercise in patience; months of emails & meetings await you.
- cobookman 6y agoConsensus Building really sums up the issue. There's no clear decision maker at Google. There is no Tim Cook or Jeff Bezos. Instead it is a collection of teams in a department. It is like a democracy, but differs in that you need every single team leader onboard to get anything done vs say 51% of the "vote".
- joshuamorton 6y agoThis is far less true than you make it seem, I think. I can think of executives who are very clear decision makers in particular contexts. But for most engineers, most of the time, you're working well below the level of those executives, and especially if you're engaging with shared infrastructure, the executive who is the final decision maker is Sundar. For example, I am involved in an issue where 3 ICs whose levels are between 4 and 6 (really this is a simplification), are engaged in dealing with solving a problem. These three ICs are in 3 different PAs, reporting indirectly to 3 different SVPs, who join up at Sundar. It isn't worth it to have the CEO spend time refereeing this decision. Ultimately this was resolved at the director level, by consensus building, because it would reflect badly on every one of those directors if they failed to resolve it and had to escalate to SVPs or CEOs about something that is, on the company scale, trivial (to be clear this is still a thing that is multiple engineer-years of work, but it's still Google-trivial). I expect the same is true at Amazon or Apple. Cook and Bezos aren't making every decision. VPs and Directors deal with small potatoes, and most things are small potatoes. The difference may be organizationally that those companies are more siloed and so leaves from different trees interact less often. But this friction also is often intentional and has value (SRE explicitly not reporting up through normal product eng ladders, for example).
- cobookman 6y agoThe executives are still beholden to supporting teams. Want to launch a new feature that depends on GFE. Looks like the current GFE is end-of-life, but the the new one still isn't ready yet. Let's connect with GFE team on if they'll support the older GFE and accept our CL to launch...<GFE has power to delay your launch right here> Next up is the documentation. That requires the doc team's approval. Oh they require IL8n, lets go to that team and see where in their queue we are <Doc team has power to delay your launch>. This same flow occurs across all supporting teams. And it can get complex with Service A depends on Service B, which depends on Service C...and Service C can reject the quota increase delaying your launch...etc. > I expect the same is true at Amazon or Apple At amazon you would connect to who everyone reports up to, or whoever has clear decision making authority. You would then provide a written document going over the facts and suggested decisions, and ask they make the call. After that its "disagree and commit".
- brandon 6y ago> Nobody at Google has a need to raise a ticket with some ops group in Bengaluru to partition a Kafka topic, renew a certificate, bridge two VPCs, or any of that type of thing. Except when your team wanted to initially onboard with GOOPS and your request sat in Buganizer for 2 weeks waiting for someone to triage. Uh oh — we're turning down this service next quarter, you will need to go start this onboarding process again with its replacement. Or when you needed quota in a cell where your product area didn't have Flex. Maybe you can set up a VC with your PARM? Does next week work for your launch plan? Hopefully they can do something for you! Or when your logs access request sat in GUTS for a month because both of the approvers were on vacation and no, there's not an escalation path. Or when you needed to change a firewall rule for a project your team inherited which for some reason runs on GCE. Make sure you bring your Ariane link when you open your request. Have ISE reviewed your code? No? ISE currently have a quarter-long backlog, so we're not sure we can grant your firewall exception. None of these examples are contrived; the weight of the operational bureaucracy is staggering. It may well be that this stuff is felt more on the SRE/Security side around production launches than on the SWE side for experimentation or iterative development, but I struggle with the idea that Google is nimble.
- throwaway3699 6y agoSure, things get hairy when you go off the beaten path, but day-to-day infrastructure is not the issue. As a user of Google products I don't care as much about developer velocity as I do them shipping swiss cheese products security-wise. If I have to wait a few months more for some new feature, I'll take that trade-off.
- jeffbee 6y agoRight, the slothful approval process for log retention and access is a feature, not a bug. It's part of the reason why Google's technical privacy story is incomparable.
- jeffbee 6y agoThat was some T7-9 whining right there. Do you think it's easier to get unplanned compute capacity at some other company?
- xmprt 6y agoAll the reasons you've given for fast pace are tech reasons and I don't think anyone is arguing with that. The author mostly discussed how people reasons is why the pace is slowed down and that's something that's a lot more prevalent at big corporations than at startups.
- osipov 6y agoThis is due to Google's over-reliance on peer reviews during perf. As the result, programs avoid poor customer satisfaction at all costs. This also means that they are incredibly risk averse and are no longer innovative. Innovation means repeated failure and failure isn't treated well come perf time.
- pid_0 6y agoWow and yet, they can't really build a single profitable product! Emphasis on single product as they tend to have 5 apps that do the same thing until they kill off the popular ones.
- tdeck 6y agoThere's plenty of red tape and broken processes at Google if you know where to look. I've waited months for a small log schema change to be approved for a yet-unused log topic. CI presubmit runs for tools I worked on routinely took several hours and needed to be manually restarted due to flakes, whereas at Square people would complain if a CI run took more than 10 minutes. The tool for releasing Android Studio SDKs was a broken mess of Python that nobody understood, so they spent 2 years writing a replacement that never came rather than fixing it. I could go on. These things definitely affect your happiness and productivity while working and the pace sure didn't seem fast.