38 ms·
Google's infamous internal 2010 “I just want to serve 5TB” video now public
- opinion-is-bad 5y agoThe multiple repetitions of “This is Google” hit home for me. I never worked as a software engineer so much of the rest is out of scope to my experience, but the constant idolization of Google, and by proxy each other for working at such a place, eventually changed from feeling coy to cultish.
- martini333 5y agoGoogle: The Sunk Cost Fallacy
- jbverschoor 5y agoomg so toxic
- devnull3 5y agoAt 2:05 the green dude asks if you think your users are scum and do you hate them. The funny thing is Google as an org ends up hating their users "accidently" anyway because of their history of pulling the rug under the services/APIs etc.
- munk-a 5y agoIf the users had properly set up a PCR notification about the change and registered it to a bigdata instance then they would never be surprised about service discontinuations. The moral of the story is that you can't fix stupid users. /s
- nunez 5y agoeven more ironic given that google+ came out four years later
- mseepgood 5y agoThis monotonous speech synthesis is annoying to listen to. The delivery of the jokes is awful. Who can sit through a 3 min video like that?
- drannex 5y agoThat's what makes this even more funny.
- zaphar 5y agoI think it mostly works best when you've lived it. Which I did. And the resurfacing of that video brought back a lot of memories.
- pas 5y agohttps://m.youtube.com/watch?v=b2F-DItXtZs https://m.youtube.com/watch?v=b2F-DItXtZs
- zucked 5y agoThis was an output of a free (now defunct) service that used to accept transcripts and pump out these videos with TTS audio. It led to some hilarious results, usually within niche communities. Around ~2010 these things were everywhere.
- deleted 5y ago[deleted]
- didip 5y agoWhat is "Borgmon readability" and why was it important. I think that's one of the punch line of the video.
- yeputons 5y agoIf you change source files in language X, someone proficient in language X (aka "has X readability") should approve that it corresponds to Google Code Style in language X. You start without it and may obtain once you've written a bunch of code in language X. I'm not sure if there is really a Borgmon readability. But if there is, it seems like Borgmon configuration files are both common (so that there is a readability requirement) and uncommon (so that there are very few people with readability).
- GeneralMayhem 5y agoThere isn't Borgmon readability anymore because of this video (the video is about 10 years old). Borgmon is a monitoring system for Borg (https://sre.google/sre-book/practical-alerting/#the-rise-of-borgmon https://sre.google/sre-book/practical-alerting/#the-rise-of-...), and it has its own language for configuring monitoring rules. The Borgmon language is infamously obtuse, and the idea of "clean" Borgmon code has basically always been a punchline. To be fair, the problem domain that it's in (declarative computation and reduction over high-dimensional datasets) is tricky to make clear, and if you use it exactly as intended, it can do some really cool stuff. But the upshot is that it has a high learning curve, a high propensity to degenerate into wallpaper, and a very small number of people who are sufficiently familiar with it to be readability reviewers. It also doesn't help that "just copy-paste someone else's" actually does cover 90% of use cases, which both reinforced the idea that it was a chore and actually made it harder to gain readability, since being granted "readability" status requires writing a meaningful amount of non-trivial, non-copy-paste code.
- raldi 5y agoBackground: https://rachelbythebay.com/w/2021/10/30/5tb/ https://rachelbythebay.com/w/2021/10/30/5tb/ This video was hugely influential on changing the way Google does internal tools and operations.
- justicezyx 5y agoI worked at TI, Planet and later Borg, I did not feel much influence of this video other a chuckle. Or I might be too low level to perceive.
- jeffbee 5y agoI think it was a very common perception among application-level SWEs and SREs that TI, platforms, and Borg did not themselves use the stack enough to perceive its flaws.
- quelltext 5y agoHow did things change?
- compiler-guy 5y agoThe most obvious change that came from this video is that Google abandoned the Borgmon readability requirement. At Google, every change needs approval from someone who has passed a detailed style-guide review process in the given language. Now over multiple changes, it used to require one fairly big one. It's still a pain in the languages that require it--which is all the main ones, but very few of the niche ones. Many other things changed as well. Much of what the video complains about got automated and better documented. But the company has grown so much, and the product lines have diversified so dramatically, that there are still plenty of places to complain about the overhead.
- dekhn 5y agoContext on borgmon: https://sre.google/sre-book/practical-alerting/ https://sre.google/sre-book/practical-alerting/ borgmon was a truly weird system.
- mikewarot 5y agoI didn't realize this was made by Google in the first place when I saw it a few days ago. I hope things are simpler now, but I doubt it.
- xiphias2 5y agoI wish I would have had this video before 2010. I got paged at night every time there was a PCR failover, and I didn’t know what to do with it. This video is better than all the extensive documentation that we had.
- deleted 5y ago[deleted]
- Imnimo 5y agoWhat I don't get is why they wouldn't just use MongoDB. MongoDB is web-scale.
- nostrademons 5y agoThat was a major impetus for this video, IIRC. The "MongoDB is web-scale" video went around Google about a month before Broccoli Man and some enterprising Googler figured they could use the same software to make a satire of Google's internal tools.
- hedgehog 5y agoLink for the Mongo video: https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs And bonus lean startup video: https://www.youtube.com/watch?v=3J9KhpgYVB0 https://www.youtube.com/watch?v=3J9KhpgYVB0
- fragmede 5y agoMongoDB is web-scale: https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs NSFWish; it gets a bit personal around 3:11
- mlindner 5y agoI miss 2010.
- vinay_ys 5y agoAh, 2010 - when web scale and its secret sauce – sharding was all the rage.
- alexjplant 5y agoI had a similar conversation with a heavily-intoxicated MongoDB sales guy in a diner at 1AM after the second day of KubeCon 2019. My concerns were primarily around data consistency issues during denormalization and lack of schema. H pitch was essentially "Who cares?! I'm getting [three-letter agency] to move _everything_ to Mongo because it's so cheap and easy! It's all just JSON! Why does it need a schema?!" He probably made more than I did that year so maybe he has a point ¯\_(ツ)_/¯
- sergiotapia 5y agoMuch more infamous. "Mongo DB Is Web Scale" - https://www.youtube.com/watch?v=b2F-DItXtZs https://www.youtube.com/watch?v=b2F-DItXtZs
- keymone 5y agos/in//
- shirleyquirk 5y agofamous ternal?
- tusharsadhwani 5y agothat would be s/in//g :)
- birken 5y agoHey... those of us that worked on Google's internal Bigtable service worked very hard so you didn't have a file to a ticket to set up replication between your Bigtable cells. The rest does seem about accurate though.
- cletus 5y agoAh, this takes me back (disclaimer: Xooger, 2010-2017). It's painful and funny because it's true. Or was true. Rumour had it that the Borgmon readability requirement was removed when Sergey saw this video. I don't know if this is true but that's what I heard.
- leg 5y agoIt is true that Borgmon readability went away due to this video. It wasn't Sergey, it was an eng director.
- ikiris 5y ago"no one has borgmon readability". years later and i still die laughing.
- DaiPlusPlus 5y agoPray tell, what is/was Borgmon?
- dekhn 5y agoprometheus 0.1
- jensensbutton 5y agoThis is the correct answer.
- knorker 5y agoTo elaborate on this answer: Prometheus was written by ex-Googlers who had no shame in copying Borgmon pretty much exactly, syntax-compatible and all.
- twinge 5y agoA system for alerting based on time-series data, with its own rule language. The language (along with many others) required authors have demonstrated they can adhere to the style guide by going through a process to obtain readability. https://sre.google/sre-book/practical-alerting/ https://sre.google/sre-book/practical-alerting/
- taldo 5y agoAh, the laughs (Xoogler since 2020). It was a lot easier, at least last year: you'd use "flex" quota from your PA pool (product area) for Spanner and Borg, write some code for your server, a few configs here and there, and you'd be ready and serving.
- dilyevsky 5y agoThat video came out a few years before flex appeared I think at a time they were having a sort of “resource crunch” on the heels of growth spur following the GFC.
- the-rc 5y agoIt wasn't that there was a crunch — that had always existed. There just wasn't all the tooling to implement anything like flex. At least this video was made after "buying Borg quota" was a normal thing. Before it, you had to "buy" regular machines and donate/assimilate them into Borg. Then after X days you'd receive your quota, minus a Borg "tax" of 10% to cover borglet and system daemons' overhead.
- exikyut 5y agoSo before Borgmon existed, the quota unit was basically an entire machine? No virtualization? Oh wait, this was probably 2008, VMWare had only just figured out JITed software virtualization a couple years prior. Makes a bit more sense now. And now the containerization thing makes a tad more sense: Google basically skipped over the "use QEMU" (or more recently, Firecracker) phase everyone else is now going through.
- the-rc 5y agoBorgmon and Borg are different things, but yes, before the latter, machines were owned by teams. Brian Grant talks a bit about the predecessors, Babysitter and GWQ: https://softwareengineeringdaily.com/2018/04/27/google-cluster-evolution-with-brian-grant/ https://softwareengineeringdaily.com/2018/04/27/google-clust... Borg was started in 2003 and by 2006-2007 was already the default way to run things, even though isolation wasn't perfect then. I think it started with chroot jails, then fake NUMA, then cgroups, which were written for it. It took years before all of web search moved to dedicated Borg machines (their quota belonged to you and nobody else could run on them) and, eventually, shared ones.
- jedberg 5y agoThe same conversation at Netflix 10 years ago: I want to serve 5TB of data. Ok, spin up an instance in AWS and put it there. I want it production ready. Ok, replicate it to a second instance. If it breaks we'll page you to fix it. The funny thing is, for important stuff, we ended up doing similar things to what you see in this video, but for unimportant things, we didn't. I think it was a better system, and it was amusing when we hired people from Google who were confused by the lack of process and approvals.
- ignoramous 5y ago> I want to serve 5TB of data. Ok, spin up an instance in AWS and put it there... it was amusing when we hired people from Google who were confused by the lack of process and approvals. Quoting from Velocity in Software Engineering https://queue.acm.org/detail.cfm?id=3352692 https://queue.acm.org/detail.cfm?id=3352692: In 2003, at a time in Amazon's history when we were particularly frustrated by our speed of software engineering, we turned to Matt Round, an engineering leader who was a most interesting squeaky wheel in that his team appeared to get more done than any other, yet he remained deeply impatient and complained loudly and with great clarity about how hard it was to get anything done. He wrote a six-pager that had a great hook in the first paragraph: "To many of us Amazon feels more like a tectonic plate than an F-16." Matt's paper had many recommendations... including the maximization of autonomy for teams and for the services operated by those teams by the adoption of REST-style interfaces, platform standardization, removal of roadblocks or gatekeepers (high-friction bureaucracy), and continuous deployment of isolated components. He also called... for an enduring performance indicator based on the percentage of their time that software engineers spent building rather than doing other tasks. Builders want to build, and Matt's timely recommendations influenced the forging of Amazon's technology brand as "the best place where builders can build." ...leading up to the creation of AWS.
- GauntletWizard 5y agoHoly crap! I've been asking for this for forever[1]. Thank you to the leaker! [1] https://news.ycombinator.com/item?id=21786729 https://news.ycombinator.com/item?id=21786729
- benley 5y agoYou're welcome <3 Here's hoping Google doesn't get mad about it - though after 12(ish) years there's really nothing secret in that video.
- jazzyjackson 5y agowhat am I looking at here EDIT: it has been explained to me: https://rachelbythebay.com/w/2021/10/30/5tb/ https://rachelbythebay.com/w/2021/10/30/5tb/
- w0mbat 5y agoWhen I first started at Google I got things done a lot faster because I didn't know all those rules existed and nobody stopped me. My service was still plenty fast & reliable. Eventually it all got rewritten by other people to do things properly like the video says.
- dekhn 5y agoI managed to deploy a whole system at google that had the ability take down all of google globally by DoS'ing the network, and ran it casually (IE, starting and stopping it when I felt like it, at the capacity I felt like, with the binary versions I wanted) for 3 years. In retrospect, this was absolutely crazy! The actual visible outcomes were: 1 cluster drained due to heat rising so fast the alerting thought there was a fire, 1 page to an engineer in the middle of the night (sorry discovery-service) and a whole bunch of complaints about CPU stealing that weren't my fault. Those were the good old days.
- robocat 5y agoRachel said something similar “My own ‘solution’ to it after far too much thrashing was just to say ‘we cannot get all N types of quota in the same place so we are at the mercy of whatever happens to be available, and if that dries up, we stop running’. Granted, this was for some internal stuff that was seven or eight levels removed from anything that anyone on the outside might ever see, but still, it was stupid and made me feel so dirty. I'm sure my non-solution probably bit someone later. Sorry, whoever.” — https://rachelbythebay.com/w/2021/10/30/5tb/ https://rachelbythebay.com/w/2021/10/30/5tb/
- hiepph 5y agoIsn't that things are supposed to be? Prototype and then polish later. I think teams should work in layer like this, but I guess with big company they're just too lazy and assume you're smart enough and have time to take care of everything.
- w0mbat 5y agoRight. Time was of the essence with this service, and it was still very reliable and scalable in its original simple form, because it leaned on a lot of existing Google infrastructure. Later on it got upgraded to follow the rules better.
- dpryden 5y agoNon-Googler: What do all those words mean? Noogler: Haha, this video is so funny! L4 SWE: (Crying because the video is so true) L5 SWE: Haha, this video is so funny! I should show it to my interns, this will be a good training for them. L6+ SWE: Why do people think this is funny? This Broccoli Man guy makes some really good points...
- belter 5y agoL7+ SWE My life is a waste but the money is too good...
- throw1234651234 5y agoNon-Googler: What do all those words mean? Exactly. This wasn't too relatable, even though I have the GCP Certified Architect cert.
- throwaway20371 5y agoThese kind of organizational problems happen everywhere, that doesn't bug me. What bugs me is when leadership knows about it and doesn't care. After low-level engineers stick their professional neck out to complain in internal town halls and through feedback forms, and leadership gives some bullshit answer that doesn't address or even acknowledge the problem. It would be less infuriating if they just said "I don't give a shit." It's the weasel words and pretending the problem doesn't exist that infuriates me. A lot of the time it doesn't even take much work at all to begin addressing the issue, like a working group for continuous improvement of highly-painful high-value processes. You don't even have to solve it. Just attempt to address it.
- calmlynarczyk 5y agoI work at a global corporation with 50,000 employees. Even though I've never been at Google I felt every pain point this video was getting at because our company is trying to implement all of this stuff right now. "Oh you want to go to production? Here's a list from A-XX stating what you need to accomplish that." Thing is I thought they actually handled this gracefully when I started because lots of requirements were tiered with various criteria you had to meet to move up (mostly for brownie points). But then one day the Tech Execs lose their minds and decide "everything needs to meet all criteria for every single process." You want to create an S3 bucket to store data? That will be a week of submitting paperwork and another month of meetings and approvals from various teams you've never heard of. Plus you have to register your schema, implement data quality checks, unit tests, regression tests, get a PR and CO approved for your central config change, remediate any CVEs in the tooling that you used, and build all of this using our in-house CI/CD platform we created because we're just soooo special. Now you're allowed to launch. Oh wait, NO because we've put the entire corporation on hold from launching new systems for the last calendar year because we're still trying to agree on the final process everyone needs to follow to go to production. It's surreal how universally so many orgs makes the same mistake of trying to throw more and more process at problems.
- acdha 5y ago> It's surreal how universally so many orgs makes the same mistake of trying to throw more and more process at problems. Followed by the inevitable ranting about “shadow IT”, AKA the requirements gathering they really should have done.
- Spivak 5y agoI’m so confused, isn’t this just like basic highly available infrastructure mixed with a toxic SRE culture? I want to serve 5TB! Okay grab two instances in different patching zones, create a bucket in our replicated RADOS storage that can hold your data or create a table/db in our Postgres cluster, write your app with tests, add an entry in to the load balancer, add an entry in our big ole distributed job scheduler if you need cron, and submit a PR against the infra repo to add Prometheus metrics and alerts. And when your done with that set up CI/CD because you shouldn’t assume that instances are reliable and if you don’t give us the code to do a deploy we can’t recreate your app when the VM goes belly up and we’ll have to page you. Are people not used to what it really takes to “just run some code?”
- gliese1337 5y agoI am used to it, but 1. It is rare for the details of how to actually accomplish each of those steps to be both documented and the documentation made accessible. 2. If you can describe it that succinctly, it really ought to be automated. If it can't be automated... then you left something out of your instructions, which goes back to point (1).
- Spivak 5y agoLike the steps to do all of this are automated, but we can't read your mind. All of this is basically boils down to submit a PR against some repo that says "there shall be two instances in these regions, there shall be a database in this cluster, there shall be a bucket with this name, etc etc" that the SRE team reviews and merges, which triggers an infra deploy.
- q3k 5y agoPeople with HA production experience can easily vibe with points made by Broccoli Man. Yes, these things make a lot of sense if you actually want to get code running reliably, especially at scale (organizational and userbase). But we must not forget how this can look from the point of view of someone who hasn't had to deal with a page due to an entire datacenter going offline, who's not aware of all the hundreds of small things that can go wrong by doing the 'obvious' thing. I think the video is more of a way to poke fun at the optics of this (and some of the overly arcane stuff involved), rather than at the idea of high availability being useless. At least that's how I've always felt about it, a way to remind SREs to respect their internal users (simplify! automate! document!) and that what makes sense to them might look ridiculous to others.
- mlindner 5y agoWow I saw this somewhere a long time ago. But I don't remember where and in what context.
- jamestimmins 5y agoAs an external user who has found Google's services to be incomprehensible, it's nice to know it is (was) equally as painful internally.
- anshumankmr 5y agoWhat is this exactly?
- ts4z 5y agoXtranormal was a video service that would animate scripts with some stock characters. Someone made a bit of internal-only snark, and "I just want to serve 5TB" became an in-joke for turning easy problems into exercises in frustration. Some of these things have, actually, been addressed.
- treebog 5y agoWhat does “serve 5TB” refer to? They expect 5TB of network bandwidth over some time period (a month?)? Or their database takes up 5TB on disk?
- drjasonharrison 5y agoIf you watch the video, it doesn't matter. It's just something they want to serve.
- raldi 5y agoIt's a joke that's sort of open to interpretation. The most straightforward is, "I just want do this incredibly simple thing; why is it so hard?" But there's also the level of, "Googlers are so engineeringly pampered that they think serving 5 terabytes is the equivalent of Hello World." And then there's another level of, "Well, isn't it? After all, this is Google and this is $YEAR."
- metalliqaz 5y agoi think it just means to put 5TB of data online
- rachelbythebay 5y agoImagine it as "I want to have a http://foo/~me/ http://foo/~me/ type path where I can park 5 TB of stuff and other people can fetch from it when they feel like it". 5 TB of data made available, not 5 TB of transfer/bandwidth/etc.
- packetslave 5y agoIIRC, the impetus for Jon Orwant creating the video was him wanting to make 5TB of data publicly available (the US patent dataset? it was before my time) and all the hassles that were involved.
- jamestimmins 5y agoAs a friend of mine explained why she left Google a year or so ago, "I got tired of emailing 30 people to try to figure out who owned a single variable."
- raldi 5y agoThe best and worst parts of being a Google engineer: Impossible things are merely very hard, but on the other hand, easy things are also very hard.
- nostrademons 5y agoA TL I worked with once had a simple but effective strategy for that: "Remove it and see who complains." I did that (with the impenetrably named "PrefetchExperiment", last touched by a branch that lost previous file history in 2007). Turned out it was the source data for Google's DNS to figure out how to route queries to the lowest-latency datacenter, based on their geographic location. In about a month, it would've taken down all Google services. Oops. It was a very effective way of figuring out who owned the variable and writing a big long comment explaining what it's there for and which team to contact before changing it, though.
- joshuamorton 5y agoScream tests are always fun. ("break it and see who screams")
- ikiris 5y agoThe hilarious thing is I know exactly what file you're talking about here.
- AlexanderTheGr8 5y agoLMAO, isn't this very similar to FaceBook's recent DNS problem? Also I love the idea of removing it and seeing who complains.
- _3u10 5y ago
- bluefox 5y agoBitTorrent existed since 2001. Get on with the times.
- gcampos 5y agoWhat exactly are these "peer bonuses"? Is it real? Is it what I think it is? Do people actually use them as bargain chips?
- q3k 5y ago> What exactly are these "peer bonuses"? Is it real? Is it what I think it is? Each month, you can nominate another employee for a small bonus. This is designed to be given to coworkers who have gone above and beyond what was expected from them. > Do people actually use them as bargain chips? From my experience it's so over-the-top absurd that it would be difficult to have someone interpret such an offer as anything other than a joke or a meta-joke. https://blog.bonus.ly/a-look-at-googles-peer-to-peer-bonus-system/ https://blog.bonus.ly/a-look-at-googles-peer-to-peer-bonus-s...
- tazjin 5y agoIt's not each month. You can send a lot of them. There's a theoretical limit and a bunch of restrictions but in practice they're unenforced.
- advisedwang 5y agoYes they are real, it's in the low hundreds of bucks range. They must be approved by the recipients manager. Its also limited how many a employee can send (but the limit is fairly high). There is also "kudos" which comes with no money, but has no limits or approvals required. They are intended to be used for going above and beyond, not for stuff that falls within the scope of ones job. Using them as bargaining chip is explicitly against policy.
- SilasX 5y agoWow, kind of funny that Xtranormal now lives on in the few viral videos that were made with it. Here's where the company is now (the original domain is used for something else now): https://en.wikipedia.org/wiki/Nawmal https://en.wikipedia.org/wiki/Nawmal
- shoeshoeshoey 5y agoFacebook had its own meme: "Pusher I need a hotfix"
- dang 5y agoRecent and related: I don’t know how to count that low - https://news.ycombinator.com/item?id=28988281 https://news.ycombinator.com/item?id=28988281 - Oct 2021 (259 comments) especially this comment: https://news.ycombinator.com/item?id=29032656 https://news.ycombinator.com/item?id=29032656
- nunez 5y agostill hilarious
- nunez 5y agothis is making me miss memegen google had its downs, but wasting hours on memegen was not one of them
- loxias 5y agoAnd now, in 2021, Google has inflicted their "clarity" on the rest of the world. I miss jobs from the 2000s, the jobs where you were paid to write software for a living. You know, engineering! Given a task, or set of requirements, develop software on your computer, software which eventually runs on the customer's computer, where it's used to solve the customer's problem. My most recent full time employment a year ago was at a great company. Healthy culture, some of the most talented coworkers I've ever had the pleasure to work along side. Over the year I lasted there, I used for the first time: Docker, Golang, Kubernetes, Terraform, Gitlab, Saltstack, Prometheus, (and probably other middleware that my brain has GCd to free space). I was barely able to get anything done. At least, it always felt that way. Maybe I'm just an idiot, I don't know. I'd accept it if true! What I do know is that I used to be able to build things for people, be compensated well for it, and get satisfaction from a customer liking what I've built. It was simple. In this brave new world, with containers, pods and this and that and the other thing, where it can take months before one even understands enough primitives to do a "hello world".... how can anything ever get done?? How can anything inventive, creative, or experimental emerge from our industry when the develop/test/improve cycle has gone from minutes to weeks or months? I don't know what the future looks like, but the present strikes me as unsustainable in the long run. (edit: Wow! I expected this to be downvoted to oblivion, not my highest rated comment on the site...) <tiny>(Forgive this shameless self promotion: if, dear reader-who-is-a-hiring-manager, you have a paid role for a lowly but experienced systems engineer who doesn't know anything about "web" or "apps" or "social" but is quite adroit with C/C++ (and a few others), most "sciencey/mathy" type problems, signal processing, firmware, network protocols, automation/scripting, and more, ... email is welcome!)</>
- ethbr0 5y agoThe current landscape (optimizing for hyperscale, at the cost of complexity) seems like a natural extension of relatively few giant corporations funding the majority of programmers. To such an organization, efficiency & time to market are more important than simplicity.
- iamstupidsimple 5y agoBut at least in the FAANG example, time to market is much slower because of said complexity.
- Zababa 5y agoThere's almost some kind of irony on uploading that to youtube, a feeling of "why can't I deploy my service as easily as people can upload videos to youtube?".
- frakkingcylons 5y agoFor anyone else who'd rather read than watch this video, here's the transcript (from YT's auto-generated captions): https://pastebin.com/8UrFftM6 https://pastebin.com/8UrFftM6
- dekhn 5y agoThank you, whomever did this! I asked for it in a comment recently. This video basically is making fun of a common situation of Google at the time, where a person wants to serve up some data for analytics, but the sysadmins expect the person to follow a process intended for much more complex and high availability services run by teams of skilled engineers. It parodies SRE as a BOFH sysadmin, even though in general SRE are quite easygoing and helpful. It helped poked fun at a number of overly stuffy processes and also helped push people to make hosting modest datasets (like this 5TB one) easier.
- Wonnk13 5y agoOne of the few things I miss about my time there... Never did get Java readability :(
- slac 5y agoI have the t-shirt!
- sbpayne 5y agoI'm so glad I can see this again. I forgot how much I missed this.
- jboggan 5y agoEagerly awaiting the "GoogFellas" video leak.
- smartician 5y agoIt's not much different today. Nowadays you'll also need privacy review, accessibility review, security review, and diversity & inclusion review.
- ntaylor 5y agodiversity & inclusion review
- pangolinplayer 5y agoBased
- sayhar 5y agoHello, I wasn't aware we were on /r/politicalcompassmemes
- thatguy0900 5y agoAre you trying to claim based is from some random subreddit
- paoda 5y agoI think it's rather clear that using "based" in this context is more common in certain communities than others.
- thatguy0900 5y agoThis is how the whole internet uses based
- X6S1x6Okd1st 5y agoNot HN
- cromwellian 5y agoAs a Googler, it's often easier for me to setup a GCP consumer account, AWS, or Heroku account to demo something, compared to using anything internal. I remember the most annoying situation was like 10 years ago when me and other engineers ported Quake 2 to run in Chrome, we were in a time crunch to demo it multiplayer, and I ended up setting up an AWS account to serve it. But then I left it running and forgot and ended up getting a few hundred dollars billed to me because the Quake2 server was chewing CPU.
- bamboozled 5y agoI could imagine you're violating some pretty strict policies doing this? You're taking proprietary code and running it on a competitors platform? I like to think I'm pretty open minded about stuff like this, and I've actually done something similar, but I'd be surprised if you didn't get your ass handed to you for that type of thing?
- mabbo 5y agoWhy? Do you think there's a risk of Amazon stealing code from a customer? No matter what code they took, the cost would never be worth it for them.
- bamboozled 5y agoIf I took my companies code and hosted it anywhere execept where I was authorized to do so, I'd expect flack for it. I'm less worried about Amazing stealing it, but it seems like a silly place to put it nevertheless.
- cromwellian 5y agoJust to reiterate, that’s not what happened, no proprietary code was exposed externally, as the OP is based off of Java clone of the Quake2 code both of which were released as open source. I don’t know what the policy of Google is with respect to hosting stuff that’s proprietary or sensitive on AWS but I imagine it requires approval just for lawyercat reasons, especially if any PII is collected given all of the regulations these days.
- silentsea90 5y agoActively working towards being a Xoogler so I don't have to live in this dystopia.
- sunyc 5y agoI actually have Borgmon readability! Peer bonus pls.
- ineedasername 5y agoI think I'm missing something. Does readability have a special meaning in this context or is it really as basic as it sounds, and therefore all the more ridiculous for being such a bottleneck?
- goleary 5y agoAt Google an employee must have "readability" in a language in order to sign of on a code reviews in that language. You must pass a test in order to get "readability".
- nojvek 5y agoThe borgman readability approvers makes me chuckle. At Stripe, there were language approvers. Only those blessed could approve PRs. Even XML had a set of approvers. I had fun time getting hold of an XML approver.
- kgin 5y agoOnly if you think your users are scum. Do you think your users are scum? Do you? Why do you hate your users?
- metanonsense 5y agoMove fast and break things! And while you are at it, please, don't break anything.
- romwell 5y agoAh, as a xoogler, I can finally enjoy this video without tears :)
- Lammy 5y agoPersonally I'm still waiting for a copy of "Pusher, I have a hotfix."
- rachelbythebay 5y agoMaaaaaaybe ID 10150136890464739. Maybe.
- Lammy 5y agoDidn’t work logged-out, but I’ll try again if I ever reactivate my account. Thanks!
- CephalopodMD 5y agoBroccoli man is incredible. I hope someone leaks the rest...
- b20000 5y agomeanwhile google docs is still a slow piece of shit.
- zzt123 5y agoWas this video ever used by Google for anything outside of internal before now? I swear I’ve seen this before (it’s still hilarious!) but I never worked at Google.
- codewiz 5y ago10 years ago, on my first week at Google, my manager suggested I watch this video to get a fairly accurate idea of what the team does :-)
- ChuckMcM 5y agoWow, way to trigger my Google PTSD. :-) I think the only thing I would have added would be using some service that recently was changed so that someone could get promoted but does the same thing the old one did but with different bugs.
- Reschivon 5y ago> It was a site where you could type in a script and it would do text-to-speech and actually > animated some goofy characters to lip-sync the words for you Does anyone still have the code for generating "Fox girl and Broccoli man" videos? I'm thinking of starting a small revival of this meme.
- adminscoffee 5y agoit's funny how you have to have a dictionary in order to understand how to navigate google's infrastructure