9 ms·
I recently left Google having worked on a number of projects with various YouTube teams. I think I can explain why it's being handled this way by YouTube. This
by Mg6yDfjp5U 3mo ago
I recently left Google having worked on a number of projects with various YouTube teams. I think I can explain why it's being handled this way by YouTube.
This is a fairly nuanced/involved issue, so the task of classifying the bug likely made it's way to one of the engineers responsible for the implementation of this feature.
That engineer has already launched this project, and filed it away under their GRAD (performance) artifacts for when promo/annual review talks roll around. There's no motivation for this engineer to waste time fixing this bug because it won't benefit their promo packet, and they are already being put under pressure to launch other projects which _will_ benefit their promo packet.
So they do what they can to sweep it under the rug because that's what the promo/annual review framework (GRAD) incentivizes and rewards.
- ronbenton 3mo agoGlad to hear this is a universal big tech experience. The promo process is entirely antithetical to shipping good products
- citizenpaul 3mo agoWhat do you mean? Youtube is unquestionably one of the most successful projects ever launched? Seems like the process works astoundingly well.
- OtomotO 3mo agoGood != Successful. I assume that's why they wrote good and not successful. It's an average software product with incredible scaling behind it and a lot of elbow grease to keep it chumming along, but it's not great software by the definition of "bugs actually get dealt with"
- jascha_eng 3mo agoIt's great software in the sense that it makes a shit ton of money though. In the end software that doesn't get used and doesn't make any money but has no bugs is not valuable either. Not saying that this is the trade off you have to make but if you have a working mode in place that achieves usage and money somewhat consistently i can understand being hesitant about changing it to optimize for less bugs instead.
- estaroc 3mo agoThe only people for whom it makes sense to define "great" as "makes money" are the people who produce and sell said product. Similarly, most people don't put much stock in the salesmen of a product describing their own product as great. Stop debasing all of quality to profitability.
- OtomotO 3mo agoThat's just two different scales. Weapons are a great product for weapon dealers and manufacturers as well, just not so much for the people killed by them (or their families, or survivors) So sure, if making a shitload of money is the metric, YouTube is a great product. That wasn't the point of the person you answered to though.
- ori_b 3mo agoSurely the Therac would have made more money if they had covered up the deaths instead of fixing the bugs and owning up to them. Why do you think they would compromise how good their software is merely to save lives?
- strictnein 3mo agoYoutube wasn't launched by Google, it was purchased.
- UnlockedSecrets 3mo agoYoutube launched 1 year and 8 months before being acquired by google.... It's largely semantics to say that what Youtube is today, isn't a direct result of Google's ownership for nearly 20 years now....
- strictnein 3mo agoHuh? It's not semantics to point out that a project wasn't launched by Google, when the point was about a successful project launch from Google.
- ismailmaj 3mo agoFrom talking to someone that worked at YouTube for 15 years, they still had a lot of core Python code in 2016 that was legacy from the OG company/team and that code needed to be transitioned to follow the Google way of doing things in C++/Go. I don't think it was distinct enough from the Google culture like Android was at the start of the acquisition but it seems they had leeway to do their own thing.
- grg0 3mo agoGoogle had Google Video and couldn't hold up, that's why they bought Youtube.
- sdevonoes 3mo agoWhat YT is now: - ads every now and then - addictive shorts no one needs - suggested videos nobody asked for - geo ban of videos
- dizhn 3mo agoNo concept of language in a user facing way. No filter by language no search by language. On the contrary searches are translated before running and return all languages, videos are dubbed even when you speak the original language, same but with titles being translated etc. Search being shit is kind of on par with being a Google product though. I wonder if they had any language preferences before Google bought them. I don't remember that far back.
- deleted 3mo ago[deleted]
- mid-kid 3mo agoYoutube survives on google's massive repertoire of products being vastly more profitable, not because it's the best of its kind.
- thx67 3mo agoAnd free bandwdith. Free bandwidth is nice.
- BetterThanSober 3mo agoGoogle definitely doesn't have free egress
- thx67 3mo agoGoogle owns the backbone. They definitely have free egress.
- BetterThanSober 3mo ago(1) There is no single entity called "the backbone." (2) Yes, they do direct peering with ISPs whenever possible, but maintaining on-premise cache AND the "backbone" is not a trivial matter, nor free They're paying a marginal cost compared to us plebs, yeah, but definitely not "free", especially when YT is allegedly responsible for 1/6th of global internet traffic
- deleted 3mo ago[deleted]
- ghurtado 3mo agoAnd you honestly believe the main factor in YouTube success was the quality of the code? That's a thought that doesn't even deserve further comment.
- dooglius 3mo agoDid the promo process exist at YouTube's creation?
- tiahura 3mo agoSweep it under the rug is not limited to any paticular industry.
- Aunche 3mo agoI don't think it's the promo process itself. If the bug was something that actually affects Google's bottom line, I guarantee that Google would find a way such that the engineer would be incentivized to fix it.
- gguncth 3mo agoShipping great products is about the details that almost nobody will notice A good promo process needs to notice the invisible Apple did it for decades
- tgma 3mo agoHaha, almost everything at Apple that is not seen by the user is full of crap (and increasingly you see user facing bugs and schedule slippage too). Possibly still somewhat true in hardware, but in software, they have far worse engineering practices than Google: remember "goto fail"? Back in Snow Leopard days you could instantly crash the kernel with a fuzzer. In fact I managed to do that accidentally by hand. NT kernel had much more systemic hardening than XNU. Apple also treats employees capriciously and in non-standard ways. Your experience is almost entirely dependent on who your manager is. I have never heard any of big tech make so many false promises to employees. I have friends who negotiated for an immediate green card application upfront, but they later found out they were bait and switched, etc.
- a34729t 3mo agoIt depends heavily on your manager and skip. My boss values operations and getting things done (including both doing things right from the beginning, and fixing things when we have to cut corners to launch quickly due to exogenous pressure), and that means people get promoted for being good engineers. Of course this falls apart for higher levels where it is entirely politics, but that is beyond my boss' influence.
- ghurtado 3mo agoOf all the fucked up things in this comment, giving a single Engineer lifetime responsibility for all bugs in code they wrote is probably the dumbest. And it's slowly becoming the norm. The last place I worked at, a large and well known Tech company, didn't even roll with QA's. That just wasn't a role anywhere in the division. You are fully responsible for all the bugs in all the code you ever wrote Cute at first. Unsustainable in the long term
- vlovich123 3mo agoOk. So QA finds a bug. Who’s responsible for fixing it? The only value of QA is to try to make sure you become aware of issues before customers find them
- episteme 3mo agoThe company, not the individual
- ShrootBuck 3mo agoAnd who in the company do you propose should fix it
- jareklupinski 3mo agosomeone hired by the company to understand the application and fix the bug ive inherited a lot of code
- SoftTalker 3mo agoFixing bugs is a great activity for new hires. Gets them familiar with the codebase.
- ProjectMRI 3mo ago[flagged]
- mlmonkey 3mo agoThis is what you get when the MBAs are in charge. They just go with P&L, Spreadsheets, etc. and care only about the current quarter and meeting the goals.
- wahnfrieden 3mo agoGoogle leadership has been from research/engineering and product backgrounds. This is how hierarchical businesses operate
- lesuorac 3mo agoExcept leadership is largely not from employees moving up the rank Sundar (CEO) is from Mcksinsley. Ruth (President) is from Morgan Stanley. TK (Cloud CEO) is from Oracle. Mohan (YouTube CEO) is from DoubleClick which is Google at this point (~15 years). --- Largely the story of the past several decades is that "doing your time" is a bad strategy. Always move to another company to go upwards.
- magicalist 3mo agoWait, but Sundar Pichai was there pre google IPO as a Chrome PM, and Neal Mohan was there for 18 years. How are they examples of "doing your time" being a bad strategy?
- foltik 3mo agoNot really, in such large companies there's enormous selection pressure favoring career politicians. Maybe some of the survivors did some engineering at one point, but expertise fades fast when you stop getting your hands dirty. Most are empty suits.
- throwrioawfo 3mo agoI feel like things have become so much more cynical in the last 5 years, in this regard. I feel like part of it is the "over-systemization" of promos. I see the logic behind it to some extent - if there's a system, it's "fairer"/"more democratic". But, then we end up with ridiculous gamified promo systems.
- wahnfrieden 3mo agoIt’s not about fairness or democracy (maybe you meant meritocracy?) at all although it’s sold that way to participants - it’s primarily about ownership’s ability to cascade management duties, including mitigating latent negotiation powers by individual workers and groups of workers
- jambalaya8 3mo agoEh, clearcut promo paths used to be a bigger thing in the 90s and they did work for a little while, they just didn't handle exceptions well, and then the whole developed world up and thought they were also exceptions. Certifications used to matter more, now they are so cheapened that you cannot do much without them.
- campbel 3mo agoobjective systems become gamified subjective systems become politicized pick your poison
- BadBadJellyBean 3mo agoWhy not both?
- lacunary 3mo agoit is both because the "objective" system is also rife with subjective judgements
- ismailmaj 3mo agoI'll pick small company, thank you.
- varispeed 3mo ago> This is a fairly nuanced/involved issue Is it though?
- Mg6yDfjp5U 3mo agoDefinitely. The front line support agents handle only the most basic requests. Anything even remotely complicated, such as this, would be internally kicked around until they found someone familiar with the project to give input. Which most likely is someone who worked on the original implementation.
- jskeicjwkxjwkd 3mo agoIt isn’t though. Just fix the goddamned thing. Fuck promo packages—fix your shit. What’s the point of saying you “work at Google” if all you ever do is work on half-baked, unfinished, unpolished slop? Fix your shit.
- esrauch 3mo agoIn 2026 things have changed, there's literally whatever tens of thousands of "security" reports that are almost all bogus as a raging crap river. I think theres very little chance this particular report made it to any engineer who works on product at all, because if they did they would be completely overwhelmed by reports, the filter which has to handle the many thousands of reports based on a playbook almost definitely filtered it out before it made it that far.
- cdbdbspt 3mo agoI also used to work at Google and what you have described is not the way the VRP works at all. 1. The engineers on the VRP teams set the severity of the bug based on impact. The engineering team responsible for the fix can argue the severity but only if they can show there is some other mitigating factor that the VRP team wasn't aware of. 2. Google has a great security culture and while it may be true that maintaining existing code may not be as sexy as building new features, fixing vulnerabilities does look good on GRAD (performance) because the impact is already well documented. 3. Believe it or not, the VRP team does like to give away rewards. However, to do this, they have to follow a rubric to keep all of the payouts consistent and fair. 4. Constructive and polite discourse is welcome and a researcher may reply to their bug asking for more details or to make their case in the event that they think the VRP team did not understand the severity. The team is made up of humans who are open to the idea that they missed something in the initial report. They, like all other bug bounty programs, are also struggling to keep up with the huge influx of AI generated slop so mistakes can happen.
- jonahx 3mo agoMy first thought when reading the article was: "The generous interpretation here is that whoever is fielding reports gets so many false positives that they miss true positives (like this report), especially if there's any gray area." I'm not saying that excuses it, but it is one likely explanation for how it happened. When looking at just one report, the response seems negligent. When looking at a pile of 1000 nonsense reports, with a handful like this, I understand the difficulty.
- sscaryterry 3mo agoThe rot is deep.
- newtonianrules 3mo ago[dead]
- dfxm12 3mo agoIt's ultimately Google's responsibility to ship bug free products. I don't care who implements a fix, but Google management should make sure someone fixes it.
- carl_dr 3mo agoNo, it’s really not, it’s none of our jobs to do that. It’s our job to make our employer (even if you are your own employer) money. It’s incredibly rare you have the luxury of even trying to deliver bug free code, let alone achieve it.
- dfxm12 3mo agoPeople eventually stop using, and paying for, buggy code.
- ZiiS 3mo agoROFL this has not been my experience. Many more people stop paying because of some featuritis request you snubed to keep the bugs under control.
- deathanatos 3mo agoBecause big tech companies are oligopolies, and there isn't enough competition in the market. If you're dissatisfied with the 2 choices out there, you cannot vote with your wallet.
- nxc18 3mo agoAnd this attitude is why we have the software we have in 2026. The profession used to recognize value beyond next quarter’s dividend (jk, we only do stock buybacks now for tax reasons).
- thi2 3mo ago> It’s incredibly rare you have the luxury of even trying to deliver bug free code, let alone achieve it. What? Every company I worked for wished for bug free code. Mistakes happen but there was no acceptance for yolo-ship features.
- NamTaf 3mo ago[flagged]
- richardfey 3mo agoI remember hearing this perspective when I first started in the software industry, and I agreed with it for quite some time. But frankly, we’ve never been further from it.
- fathermarz 3mo agoI think there is a fine line. YouTube is not critical software and no one’s life depends on the safety (putting mental health aside) of the code running. Some software engineers do however write code that is critical, but to your point, I don’t think they are ever considered liable. I went through an acquisition as a Canadian software developer getting acquired by an American company. They wanted us to be called engineers like the rest of their SWEs but in Canada it’s a protected namespace. It’s illegal to call yourself an engineer without having the ring and the papers. Which personally I can appreciate.
- m00x 3mo agoYoutube should consider their engineers responsible for the software they write. Big companies these days are just bureaucracy tricks and politics. There's a small handful of real talent, but they're quickly moving to new startups. Also, I'm Canadian as well, and almost everyone calls themselves "software engineer" these days. You just can't say P.eng. in your title. You could be forced to remove it from linkedin/etc if you're called out, but it rarely happens.
- eldaisfish 3mo agoYour latter point is legally incorrect. The protected term in Canada is “engineer”. If someone calls themselves an engineer without a P.Eng, that’s an offence.
- fathermarz 3mo ago
- alfiedotwtf 3mo agoHoly shit working at Google sounds depressing AF
- sieabahlpark 3mo ago[dead]
- BrenBarn 3mo agoThose are many words to say "no one feels an obligation to do the right thing".
- lordie 3mo agoIt's easy to cynically generalize and attribute to the broken promo process when it is more likely either a non-engineer reviewing the report or someone else not really understanding the nuances of prompt injection. I work at YouTube, and I've escalated it to the appropriate TLs and TnS leads to take a look. Bugs in existing projects and a sense of ownership and leadership are absolutely a part of GRAD, having been in several calibrations and promo committees myself. So while this understanding has a grain of truth, it is far from what's evaluated, at least in my VP's org. I can't speak to Cloud or any other PAs.
- lnrd 3mo agoIn my company, user reported bugs get triaged and once they are confirmed they end up in the board of the team owning that part of the system (often they also built it, but is not necessarily the same people). Then there's a team bug threshold and if it gets reached then the whole team can't merge anything else until X amount of bugs are fixed and the number goes down. It's very annoying system and in cases of emergency it can of course temporarily be lifted by someone above, but honestly is very effective in making sure that bugs are fixed in a reasonable amount of time. This makes fixing bugs "part of maintenance work that is a given and expected to be done between new initiatives".
- tgma 3mo agoObviously what you said is not some unheard-of secret or deep analysis. It is a running joke inside the Google system. But... many Googlers who have a tendency of repeating these things have (1) not seen how shittier things are on the outside (2) are not in management and do not know how much manager lies to them (3) have unrealistic expectations of how well any process applied to 100-200k people can work. If you see a place that has a better overall promo system, you'll almost certainly find that it is a much smaller shop and things are decided more ad-hoc at the top with higher information flow. Specifically, for (2) the manager and their adjacent group can clearly flag the slipping under the rug behavior and ding one's promo. However, sometimes when they message it back they would lie about it to the employee and blame some other management or requirement or complexity, etc. Other times, the manager is a "people manager" moron and non-technical, and can't really evaluate (in which case it's not the process that's at fault, but useless management.) It's also not clear that the optimal quality is achieved by spending more time "perfecting" things. At Google, people already work much less than other companies. Perhaps the answer is in fact the opposite: pushing to ship more milestones per unit of time and driving harder to then perfect it. My bet is if the promo packet was accepted without a full "launch"[1] they would have still shipped the same half-baked crap at a later point in time. [1]: many years ago, they wanted to reduced half-baked "launches" and said we want "landings" not "launches" and wrote some documents explaining the difference and self-congratulated themselves. Net result: s/launch/landing in promo packets.