7 ms·
How we scaled Google Meet during Covid-19
- jtwaleson 6y agoTo Googlers working on this: thank you! My work life shifted from 5% Meet to 80% Meet and it only got better during the crisis. Never noticed deteriorated performance.
- ramshanker 6y ago30x growth only? My wild gutted is that at least 1 billion Gmail App installations must have been updated to integrated Google Meet versions. So I expected much higher jump. Just yesterday, we were asking people in our school WhatsApp Group to install Teams app. As for Google Meet, just update!!!
- tpmx 6y ago> 30x growth only srsly?
- m3kw9 6y agoTo be fair 1 to 30 is 30x growth. It really depend on the original size.
- jeffbee 6y agoIt says they ended up at 100 million daily active users. The fact that they could deploy that capacity in 60 days should terrify their competitors.
- stingraycharles 6y agoI don’t think so, as their competitors already know Google is Google and has virtually unlimited resources. As far as I understand from the article it was mostly a matter of “hey can we have extra capacity”, which shouldn’t surprise any competitors.
- jsnell 6y agoThe competitors scaled by similar factors from similar baselines. Here’s a comment I wrote in April, collecting the then-current info from the three big ones: https://news.ycombinator.com/item?id=23029151 https://news.ycombinator.com/item?id=23029151
- oblio 6y agoGoogle Search had 1.2 billion users back in 2013, now it probably has 3 billion. Youtube has 2 billion users. Gmail has 1.5 billion users. Android/Play Services has 2.5 billion users (well Android has 2.5, Play Services probably has at least 1.5 billion users). For Google a service with 100 million users is a drop in the bucket :-)
- jeffbee 6y ago1.5 billion monthly active users sending short text messages is a really different network capacity problem compared to 100 million daily active users sending high-definition video streams to each other.
- londons_explore 6y agoBut it's still small compared to YouTube video content served. Also, the WebRTC stuff all browser based videoconferencing apps are built on supports peer to peer Comms in most cases, so worst case they could have moved all the users over to that in an overload situation.
- jtwaleson 6y agoNot sure about the comparison with YouTube. YouTube is one to many, and can be heavily cached. Meet is many to many with no caching.
- jeffbee 6y agoRight, and YouTube doesn't need high priority traffic because it has a deep buffer on the receiver, whereas videoconferencing will be disrupted by even a single dropped frame.
- ocdtrekkie 6y agoLiterally adding Google Meet URLs to Zoom meeting invites in Google Calendar, so half of a meeting's members went to the wrong room is kinda a bad way to scale your product: https://mspoweruser.com/googles-latest-audacious-growth-hack-sending-google-meet-links-along-with-zoom-invites/ https://mspoweruser.com/googles-latest-audacious-growth-hack...
- izacus 6y agoThe dates on the shown graph significantly predate this new feature (or GMail tab).
- gav 6y agoThere's a setting you can turn on called "Automatically add Google Meet video conferences to events I create"[1]. I tested on a new account and the default was on, which you might argue is bad, but when I tried to create my first calendar event there was a popup that told me that it was on and gave me the option to turn it off. Plus when you create a event the option to include a Google Meet is right there and it's one click to remove. [1] https://support.google.com/a/answer/9898950?hl=en https://support.google.com/a/answer/9898950?hl=en
- ocdtrekkie 6y agoQuestion is, what's it doing when you use client software that doesn't support the spammy popups? It appears that for a third party client user, it auto-injected URLs the sender didn't expect or even see. So someone creating a Zoom invite had no idea their coworkers might see a Meet link. Also, Google could/should recognize that a meeting invite with a competitor's room URL in it is somewhere it shouldn't inject its own. Especially since it opens up yet another avenue for an open and shut anticompetitive case against them.
- londons_explore 6y agoBlacklisting some competitor URL's is discriminatory against smaller competitors or people's on-site solutions at private URLs
- deleted 6y ago[deleted]
- jeffbee 6y agoI'd love to hear the war stories about the network capacity plan violations that were involved here. There's no way they had 30x headroom in their 90-day capacity forecast. The organizational ability to reach in and shuffle up the network capacity allocations may be even more interesting than the technical items discussed here.
- justicezyx 6y agoNah, this is considerably minor scaling problem for Google. Youtube bandwidth is there. Google meet is just a rounding error. Of course (most likely) there will be dedicated planning going forward, but these type of traffic ramping up is daily operation at Google. At Google, application developers rarely pay much attention to scalability, that's how the TI org tries to achieve, and they are probably the best AFAIK.
- jeffbee 6y agoIt's clear that the Google where you worked existed in a parallel universe from mine. On my home planet, google netcap required a forecast per-product, per region pair, years in advance, and the product group got charged back for the capacity requested, not for the usage, so nobody was requesting extra.
- londons_explore 6y agoHah - except there were overestimates at every level, like where teams were confident of there 50% growth per quarter for the next 2 years, and that never materialised. Know the right people, and that unused capacity can be reallocated to you in a few hours. Capacity planning done like that is mostly a waste of time...
- justicezyx 6y agoYes, planning is always there. I was saying that this sudden increase in the traffic should not cause major issues. Most likely several SREs have emergency meetings, and get quotas across places. Then it's done. If you read my post above, you'll notice that I did not mention SREs. I was saying app developers do not care planning. Of course, some app developers thought they were doing planning, as in some activity to estimate demands, etc. That, in Google infrastructure engineering, is not considered capacity planning. The typical capacity planning is done by SRE with more procedures that certainly border the most curious developers.
- CubsFan1060 6y agoI used Google Meet once, and immediately ended up back at Zoom. I was not particularly impressed.
- deleted 6y ago[deleted]
- exclusiv 6y agoIt's gotten better but the video quality is definitely lacking.
- Aachen 6y agoCan't say I notice a difference, most platforms just take your webcam data and forward it so I don't get why people report different qualities? Never had to use zoom luckily but between Wire, Jitsi, Hangouts, BigBlueButton, and two others whose names I don't remember right now, it always depended on people's mics, cams, and WiFis (and sometimes available CPU if my crappy work laptop had too much to do in the background to en/decrypt the streams).
- kevindong 6y agoI found it to be perfectly adequate. Just like Zoom. Neither stood out and I'm perfectly fine with that. Some of my fellow employees asked for a company wide Zoom license at the beginning of WFH so out of curiosity I looked to see how much Zoom costs. Turns out it costs about the same as all of G Suite which honestly surprised me (in how expensive Zoom is). https://zoom.us/pricing https://zoom.us/pricing https://gsuite.google.com/pricing.html https://gsuite.google.com/pricing.html
- parhamn 6y agoIt's crazy how little Google's marketcap benefits from things like this. Zoom's here sitting on 75B (~1/15th Googs) as we speak and Google Meet probably isn't even legitimately used in pricing Google. Wonder what Google's market cap would be if each individual service got the VC future growth pricing.
- bhauer 6y agoNot only that, but modern Google Meet is significantly more useful to my day-to-day remote conferencing needs than Zoom. Modern Google Meet's advantages: * It's nearly instantaneous to use. You can just bookmark any prior meeting ID you created and reuse it indefinitely. You just click the meeting on your bookmark toolbar and join the meeting. Creating a new meeting is also nearly effortless. * Works right in your web browser. It doesn't default to pushing you to use a native executable. * IMO, the video and audio both are slightly better than Zoom. Google Meet or Hangouts from years ago was quite bad, but it has improved nicely.
- MisterPea 6y agoAlso the transcription. Fastest, most accurate transcription I've ever seen.
- hitekker 6y ago> IMO, the video and audio both are slightly better than Zoom. Firm disagree. Three days ago, I presented a video in a google meet stream for 50+ people. For three minutes the stream dropped frames , displayed visual artifacts and lagged behind my narration. I was on gigabit internet and so were my colleagues. I never had this problem with zoom, and I would never use google meets for a streaming setup again.
- ariwilson 6y agoDid you use the newish "Present a Chrome tab" feature to do this? https://gsuiteupdates.googleblog.com/2020/04/high-quality-video-audio-meet.html https://gsuiteupdates.googleblog.com/2020/04/high-quality-vi... Normal Google Meet Present is optimized for slides/docs which is a very different problem than streaming video.
- j4mie 6y agoEvery web-based video chat I’ve used (including Google Meet and Slack) makes the fans on my decently-specced 2018 MacBook Pro spin up like it’s about to take off. I’ve tried Firefox, Chrome and Safari. Zoom (via the native Mac app) doesn’t do this. Does anyone know why browser-based video is so resource intensive?
- kenhwang 6y agoJust tested a Google Meet meeting while on a native Zoom video call. Google Meet used 3x more GPU resources while having 1/5th the video participants.
- giovannibajo1 6y agoUse Safari. Turn on the developer settings, and find the setting to disable VP9 in WebRTC. That will force Meet to stop using VP9 and switch to H264 for which MacBooks (and Intel CPUs in general) have a hardware accelerated decoder and encoder.
- tasssko 6y agoWhen did Google meet get better? I used it actively in April had a some really bad customer meetings on it (it seemed to be bandwidth heavy though that is anecdotal). I decided on Zoom and haven’t really looked back. I also was irritated at the Google Meets browser compatibility issues as I mostly use Firefox.
- guillemsola 6y agoI think the point of this article is how the Meet team handled the increase usage, not whether if this product is better of worst than competitors. I bookmarked the article that can be seen as an engineering approach to solve a problem like this.