6 ms·
Disclaimer: I work at Google but not on YouTube, also speaking just personally. This isn’t practical at all in a latency sensitive environment. Feature detecti
by mattnewton 7y ago
Disclaimer: I work at Google but not on YouTube, also speaking just personally.
This isn’t practical at all in a latency sensitive environment. Feature detection works once JavaScript has loaded on the page, which means we either have to serve you a giant bundle of stuff you may not be able to use, or we have to degrade the experience of the latest browsers by detecting features and fetching more HTML/CSS/JavaScript once we know they will be able to handle it.
Feature detection absolutely is used for some things, but it’s unreasonable from a latency perspective to serve the bundle of all possible sites and make all these decisions on the client. We aren’t talking about polyfills here- A lot of these features mean we are shipping a totally differently lay out because this webkit uses flex box 2009, or mobile safari 8 has a layout bug, or IE has a quirk around event bubbling in video elements, etc.
Feature detection stops being better when a) your supported browser list stretches into the early 2000s around the world and b) you are in a latency sensitive environment where it is unacceptable to degrade performance of the latest browsers. If you aren’t bound by those constraints, or the detection has a cheap fallback so it isn’t meaningfully impacting latency, I agree with feature detection.
- kchoudhu 7y agoIt's Youtube, not high frequency trading.
- clarry 7y agoAnd subjectively, the latency on most google services (except for the search page) is pretty bad. Maybe Firefox Nightly isn't modern enough?
- danShumway 7y agoThe logic is apparently that we can't do progressive enhancement because it would force us to make a second network request, but we can rely on unstandardized APIs that force us to ship polyfills that make our site slow on every other browser that's not Chrome. Every decision, from progressive enhancement to supporting bleeding-edge features comes with tradeoffs. Youtube is still choosing tradeoffs, it's just choosing them based on what will work best in Chrome specifically.
- mattnewton 7y agoI agree there are just a bunch of trade offs, and this shows you can’t please everyone. If you ship large bundles of polyfills people are sad they get some slow JavaScript, and if you give them fallback html and JavaScript that renders quick and supports their feature set, they are sad they didn’t get the same feature set as chrome latest. You have to minimize the fallback sadness as well as keep pushing better and better cutting edge experiences. This demanding environment is one thing I do like about the job even if it makes me feel constantly unqualified and doomed to fail :D Speaking personally from my time before working at Google, the second network request is definitely the worse of the two options if you can’t do anything until it completes for most people- the round trip time kills you in the median US case I am familiar with. At Apple for my small project we didn’t have these constraints so we just shipped a giant JavaScript bundle with feature detection and browser-side re-rendering that worked alright.
- stonogo 7y agoNo, this shows you're not interested in pleasing anyone. There is no universe in which instructing the user to change browser software is preferable to adding milliseconds of latency at pageload. This attitude is so wrong it's almost sinister.
- mattnewton 7y agoI am not defending YouTube redirecting a UA it hadn’t seen to download chrome when that UA explicitly went to /new for the new experience. I don’t know what happened there, and if I did I probably couldn’t say. I am defending using the UA to tier responses based on the principle that it lowers the latency before the page is useful to the user, and I don’t see how that is a sinister practice.
- deleted 7y ago[deleted]
- bduerst 7y agoYou're right, Youtube is the largest content streaming website in history. HFT'ers would not even be able to host it.
- serioussecurity 7y agoMost of the big tech companies (amazon, google, facebook most notably) have discovered an extremely strong correlation between latency and $$$. Every saved millisecond of average load time tends to generate an incredible amount of money (like, 10s of millions of dollars a year per millisecond).
- la_barba 7y agoI think disabling the all the user snooping would help reduce the latency too. Just saying :)
- asdff 7y agoNearly 15 years of youtube and the only new feature I've noticed that isn't directly copied from the unusually innovative explicit video sites are the ads polluting every video, 4 minutes apart. Brilliant engineering, really, the best talent working on this. As an aside: is it possible to go back to a 2005 version of the site? I bet that site absolutely flies on modern hardware, and all I really want is the search bar and player anyway.
- la_barba 7y agoWell, I do think that delivering all the YT features to users on a global scale is an engineering challenge. Its just that I would much rather prefer they didn't actually succeed at doing that, especially when it comes to a few of those so called "features". >As an aside: is it possible to go back to a 2005 version of the site? I bet that site absolutely flies on modern hardware, and all I really want is the search bar and player anyway. On one hand, I dislike the SV venture capital thing about throwing money at companies and products that contineu to rack up giant losses simply because they're growing like crazy. On the other hand, YT would probably never have been a global phenomenon if it didn't run at a loss during the initial years and if google hand't acquired it. It is expected that Google is looking for a return on investment of their 6 or whatever billion dollars they paid. Its just too bad that Google is an advertising company and all they can do is shove ads and crapify the experience. Aside: I find it hard to criticize Google without sounding like I'm personally antagonistic towards the engineers who work there.
- danShumway 7y agoThe modern version of Gmail has a loading bar that takes about 3 seconds to complete on every single page refresh. This is subjective, and it's a bit unfair of me to bring up, but my experience is that whatever Google is optimizing for, it doesn't seem to be latency. At least not on Firefox for any of the products I use.
- EpicEng 7y ago"Google" encompasses many different projects and teams. What does Gmail loading time have to do with what the GP said?
- jrochkind1 7y agoBecause the GP said "I work at Google but not on YouTube", and was speaking of Google and the sorts of technical challenges Google has in general, not any specific product. So danShumway pointed out that in one flagship product, they didn't seem worried about latency, so it could be argued that avoiding user-agent detection is more important than latency on youtube too. But yeah, there are lots of competing things going on, I found both comments helpful.
- kinkrtyavimoodh 7y ago> Because the GP said "I work at Google but not on YouTube" That was a disclosure, not a credential. My reading of it was that GP was speaking of the tradeoffs in their capacity as an engineer, and said they are a Googler for fair disclosure and nothing more.
- jrochkind1 7y agoThey weren't explicit that they weren't talking about YouTube specifically but generally about the kinds of challenges and tradeoffs Google faces, so that's why the reply was also talking about products other than YouTube in the context of the kinds of tradeoffs Google has been known to make. That's what it has to do with what original GP said, to answer your question directly. Anyway, I found both original GP and the reply useful, and do not find this exchange to be, so I'll stop!
- spaceribs 7y agoIt's unreasonable if you feature detect on every load. why not store feature detected capabilities within a cookie that's passed on additional requests?
- mattnewton 7y agoIf you think about it, that’s kinda what is happening, except instead of a punishing first load that appears broken and slow every time the user loses the cookie or we change our feature detection code, we do that ahead of time and the “cookie” is in a map where the key is the UA
- spaceribs 7y agoInteresting, so to be clear, the sequence looks like this: 1. User goes to website, UA communicates previously known features for this browser. 2. Browser does feature detection after load 3. Browser communicates any new browser features up to a backend service, crowd analysis determines if new features have been implemented for this UA. If that's the case, that's a pretty awesome way of keying features, but I'd have to think about the implications in practice.
- mattnewton 7y agoNo, I haven’t seen anything that fancy. The “user” in step 1 is internal QA testing the site on the browser, maybe after seeing it a lot in logs. Exposing users to novel UA strings to a bunch of feature detection JavaScript is an interesting idea but it’s probably better to log them and then do this in a QA setting.
- finchisko 7y agolike this proposal
- faxnlogic 7y agoThe detection only needs to be done once on a given browser; only an extremely incompetent engineer would suggest performing that detection on every single page. As such, I'm unsurprised to learn that a Google engineer thinks that the only possible way to solve this is to perform feature detection, repeatedly, on every single page. The reality is that Google hires mediocre and lazy engineers, and that they also have a vested interest in pushing people into their proprietary browser. The result is that they do the thing that's best for Google as a business (pushing their invasive browser), and their incompetent staff line up to publicly defend this selfish and immoral decision.
- filoleg 7y ago>This isn’t practical at all in a latency sensitive environment. Feature detection works once JavaScript has loaded on the page, which means we either have to serve you a giant bundle of stuff you may not be able to use, or we have to degrade the experience of the latest browsers by detecting features and fetching more HTML/CSS/JavaScript once we know they will be able to handle it. It might sound naive, but why would fetching more JS later degrade the experience? I thought that bundle splitting was an accepted good practice. For example, you have your main core bundle, then some polyfill bundles (that get downloaded if the feature detector says it is needed), and then you have separate bundles for obscure and not-that-often-used stuff that gets loaded on demand. It is a genuine question, and I would like to learn what's wrong with this approach, because that's something I work with on a day-to-day basis. P.S. Please don't go into the negatives of bundle splitting every little thing into its own bundle that gets loaded on demand, I am aware of downsides of that. I was talking about more sensible approach to bundle splitting (which I described in a somewhat oversimplified way).
- mattnewton 7y ago>> It might sound naive, but why would fetching more JS later degrade the experience? I thought that bundle splitting was an accepted good practice. For example, you have your main core bundle, then some polyfill bundles (that get downloaded if the feature detector says it is needed), and then you have separate bundles for obscure and not-that-often-used stuff that gets loaded on demand. It is a genuine question, and I would like to learn what's wrong with this approach, because that's something I work with on a day-to-day basis. There isn’t anything wrong with this approach at all - keep doing this! It just doesn’t solve the initial load problem. This kind of code splitting _also_ happens on all google properties I am aware of. The problem here is what gets into the initial bundle, and is it enough to make the page useful. If the initial bundle isn’t enough to make the page useful, then the user has to wait for two requests to complete serially before they can perform their task.
- jlokier 7y ago> If the initial bundle isn’t enough to make the page useful, then the user has to wait for two requests to complete serially before they can perform their task. Since the alternative is just broken, I'm sure users with non-Google browsers would prefer a working page with one round-trip of extra latency. Meanwhile, it causes Chrome users no harm, they are unaffected. But anyway, there's no need for extra delay. You can put a coarse-grain "shall we load part 2" test inline in the HTML, and/or have the server predict from User-Agent (as a performance heuristic - and this can be automated). The above can provide the same, optimal latency to all different browsers, even when the site is beautifully well-tuned, 0RTT-optimised, TLS/3, QUIC service, with all the PUSH trimmings. (For even better performance you can select from multiple builds using the same heuristics, optimising out compatibility branches for other browsers on all browsers, not just the favourite. But this is not necessary, just to prevent the delay being discussed. It's an extra boost.) There is no latency excuse, just engineering deciding not to bother. Something Google should be criticised for, since they can afford to do better, have a monopoly, and claim to be a thought leader on open web standards.
- plorkyeran 7y agoyoutube.com takes a bit over 3 seconds to load for me in Chrome with a warm cache. Minimizing latency is clearly not a top priority.
- taf2 7y agoIt's also true that many browsers appear to have features but they are actually broken features so sometimes in order to get the application to work 100% you have to do browser user agent sniffing... think Indexeddb and safari
- manigandham 7y agoEvery single Google consumer app has gotten more bloated and added seconds in load time. What exactly are the metrics being used to measure UX and latency? What Google service loads faster than HN?
- mattnewton 7y agoI mean, HN isn’t an apples to apples comparison at all. I bet Google search (where I work) loads much faster when we serve 10 blue links to an older phone or IE6, and that page is at least an order of magnitude more complex to assemble. When we serve you a live stock ticker, video results and carousel of relevant tweets though it’ll take a few more milliseconds to get that data. My point isn’t that latency is the only thing worth optimizing for, it’s that you can spend your latency budget on something like feature detection, or, adding new features and better UX for newer browsers.
- manigandham 7y agoI should've been more specific. Other than the search page (which obviously is meant to be fast), the actual consumer apps are all becoming slower: Gmail, Maps, Youtube, Play, etc. This is a well-known complaint across many users with all sorts of devices and connections. I understand there are budgets and tradeoffs but the general sentiment is that Google is making the wrong decisions here.
- ha99ja 7y agoAll of Google can't come up with a simple way of caching supported features by browser? ``` doesBrowserSupportFeature? = (userAgentString, feature) => boolean ```
- BinaryIdiot 7y ago> This isn’t practical at all in a latency sensitive environment. Other than Google Streaming / Stadia and some AMP sites I have never seen a Google site / app load in less than 2 seconds or so (which is kinda slow, all things considered). So I'm a little confused about this latency sensitive environment statement. Sure, what you say is correct in that using agent strings can provide an optimized bundle to the browser the faster, but isn't there far better optimizations that Google could make first? I've seen and worked on very fast, sub second web apps before so this argument feels more speculative than any good reasoning. Besides, you shouldn't be gating _so hard_ with agent strings anyway. The non-Chromium version of Edge is probably fine on YouTube as well. You likely want to gate away the obvious offenders, assume the best with the newer browsers but allow a fallback option that detects a few things on load so you can adjust and set cookies for the next time.
- mattnewton 7y agoI wish you worked with me so I could pour over the metrics with you and try some experiments. I am not being flippant - I am genuinely interested in working with people who want to think through these hard problems. If you are at all interested in coffee and a potential referral, let me know.
- jlokier 7y agoI'm not the parent poster to your comment, but I'm genuinely curious: What is it about metrics that makes breaking a browser like Edge unavoidable?
- BinaryIdiot 7y agoYeah, performance is a hard issue especially with well established applications. Experimentation is fun. Good luck, wish I could help you more :)
- OrgNet 7y ago> This isn’t practical at all in a latency sensitive environment. Google doesn't really care that much about latency, they only care about it so that they can hide from you that they are loading all that extra javascript that spies on you. And btw, watching videos is not really latency sensitive, it is more bandwidth sensitive... (I don't really care if I have to wait 1 second to start watching a 2hour video, but I will be mad if it is buffering every 30 seconds)
- runarberg 7y ago> Feature detection stops being better when a) your supported browser list stretches into the early 2000s around the world and b) you are in a latency sensitive environment where it is unacceptable to degrade performance of the latest browsers. So whitelist “safe browsers” with server side user-agent sniffing and send the rest a bigger bundle with feature detection. There is no reason not to serve users of modern browsers the latest feature just because you fail to detect them in on the server.
- XMPPwocky 7y agoThis works great until browser vendors that aren't whitelisted but support the features of a whitelisted browser (e.g. Chromium derivatives) realize that they're paying extra costs they don't have to, and change their user agents to a whitelisted browser... Posted from Chrome- > Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36
- runarberg 7y agoIt is better to get the feature along with a bigger bundle then not get the feature at all even though your browser supports it. Note the parent is claiming that feature detection is not feasible for a site like YouTube because of the extra cost. Here I show how you can still bundle optimally for whitelisted (i.e. QA-ed) user-agents while still not blocking non whitelisted user-agents from accessing these features, and still not brake the page for user-agents that don’t support it.
- forgottenpass 7y ago>and b) you are in a latency sensitive environment where it is unacceptable to degrade performance of the latest browsers As you don't work at YouTube specifically, you might not know they don't have this constraint. They are perfectly happy to let YouTube run like complete dogshit if the user is on Firefox.
- 013a 7y agoThat's all fine, but at the same time these conclusions seriously sound like an excuse that leads to cherry-picked metrics optimization. In other words: "we consider low latency across browsers we support an SLO, and to maximize that we just don't support that many browsers."