10 ms·
YouTube's road to HTTPS
- iptables 10y ago> You watch YouTube videos on everything from flip phones to smart TVs what year is this?!
- pki 10y agoI am less curious about the year and more curious as to how the hell flip phones are rendering video. Like 64p 3GPP or something realtime transcoded?
- euyyn 10y agoASCII art.
- dredmorbius 10y agoaalib and aaxine, baby! https://m.youtube.com/watch?v=mGphynqDkS4 https://m.youtube.com/watch?v=mGphynqDkS4
- coob 10y ago> We found that HTTPS improved quality of experience on most clients: by ensuring content integrity, we virtually eliminated many types of streaming errors Wow, what were carriers doing to the streams?!
- JBiserkov 10y agoAttempting to inject adds? Reduce the size by applying JPEG compression? :)
- TD-Linux 10y agoWhen I had Sprint, they did in fact scale down and recompress all images in lower quality JPEG. Some carriers even attempt to realtime transcode video into a lower bitrate. The implementations of this are universally poor and produce rather broken files.
- dublinben 10y agoThe more modern implementation of this (I'm looking at you, T-Mobile) is just to throttle all connections to content providers. They rely on the content providers' own adaptive streaming technology to deliver the appropriate 480p video when presented with a constrained connection.
- ianlevesque 10y agoNot ideal but that's a lot less intrusive.
- mhurron 10y ago> The more modern implementation of this (I'm looking at you, T-Mobile) is just to throttle all connections to content providers. If YouTube can't handle traffic shaping, Youtube is broken.
- dublinben 10y agoIf your ISP applies artificial traffic shaping, your ISP is broken.
- kbenson 10y agoIt's long been known that some carriers purposefully degraded quality of youtube to keep from saturating their interconnections. That's the reason[1] behind the Google Video Quality Report[2]. Youtube does (did?) redirect you to a comparison of how well your ISP streamed content compared to others when there were streaming problems in some cases to combat this. 1: Well, I'm assuming, but it seems likely this was the main reason. 2: https://www.google.com/get/videoqualityreport/ https://www.google.com/get/videoqualityreport/
- lqdc13 10y agoDegrading video quality is not a bad idea if you have limited bandwidth. This would help in cases such as airplane flights. One person watching HD cat videos is going to consume more bandwidth than 20 people doing work. Now, assuming bandwidth increase is difficult to achieve, they would be forced to keep increasing the price instead. Prioritizing non-video content is challenging if all content is encrypted.
- icebraining 10y agoWhy not limit the bandwidth per user instead? Youtube itself will auto-adjust quality based on connection speed.
- dyladan 10y agoNot sure about other services but at least GoGo was limited to about 500kbps per user last I heard.
- acdha 10y agoExactly – this is both easier to implement and has the advantage of not incentivizing people to use something like a VPN to get more bandwidth at the expense of everyone else.
- mdani 10y agoPer user limits are not easy to implement, maybe easier to visualize - consider that there are a bunch of packet gateways sitting behind a load balancer and each HTTP session may end up on a different server. There is no entity that counts the live bandwidth usage on a per user basis, let alone control it. Billing and metering is done on a session basis through logs. So from T-Mo's point of view it is much easier to detect a HTTP session as video and just throttle that session.
- mholt 10y agoI grew up in rural Iowa with a poor Internet connection (satellite) that dropped packets if there were thick clouds, any snow or rain, even wind sometimes. Integrity checks would have helped a lot.
- mikeash 10y agoTCP already covers that sort of thing. What HTTPS adds is integrity checks against an intelligent attacker, not just random natural interference.
- jasonuhl 10y agoTCP only gives you 16 bits of checksum, which is really not enough. I used to work for a CDN which served downloads over HTTP, and when the object size grew to a few gigabytes a non-negligible percentage of users ended up with corruption. The SSL MAC is valuable even in the absence of enemy action.
- mikeash 10y agoMakes sense. Thanks for the informed correction.
- DanielDent 10y agoThis is a really interesting 'real world' anecdote. Do you know of any related data available publicly? i.e. "on our network, X bits in Y terabytes end up with undetected corruption, meaning that approximately Z% of downloads of a 2GB file will have at least one bit error".
- Dolores12 10y agoFrom http://noahdavids.org/self_published/CRC_and_checksum.html http://noahdavids.org/self_published/CRC_and_checksum.html In "Performance of Checksums and CRCs over Real Data" Stone and Partridge estimated that between 1 in 16 million and 1 in 10 billion TCP segments will have corrupt data and a correct TCP checksum. This estimate is based on their analysis of TCP segments with invalid checksums taken from several very different types of networks. The wide range of the estimate reflects the wide range of traffic patterns and hardware in those networks. One in 10 billion sounds like a lot until you realize that 10 billion maximum length Ethernet frames (1526 bytes including Ethernet Preamble) can be sent in a little over 33.91 hours on a gigabit network (10 * 10^9 * 1526 * 8 / 10^9 / 60 / 60 = 33.91 hours), or about 26 days over a T3.
- drzaiusapelord 10y agoI imagine this was a snarky comment about services like T-mobile's binge. They limit video 720p or 480i at a moderate bitrate for mobile. Its supposed to be opt-in but I wouldn't be surprised if many mobile providers peppered the video stream with errors to knock down costly high-bitrate 1080p streams.
- dawnerd 10y agoOpt in? It was opt out unless it changed very recently. I had a call with their corp office to plea why my plex server should also qualify but they said it wouldn't.
- Thaxll 10y agoRemind me the madness that mobile carrier are doing to the TCP stack.
- mh- 10y agoany links with technical details about this?
- Animats 10y agoIt's turning out that HTTPS is more important for content integrity than for security. It's not surveillance that's the big problem for static content. It's middle boxes designed with the delusion they're entitled to mess with the content.
- abalone 10y ago> It's not surveillance that's the big problem for static content. Why would you say this? Surveilling what URLs someone is accessing / content they are watching / books they're checking out of the library has been a major security issue, historically.
- Animats 10y agoHTTPS doesn't conceal the domain, or the length, or the order of requests. It's less of an issue for Google, because so much stuff comes from one domain. For small sites, figuring out who read what isn't a hard problem. In practice, you can probably buy that info from an ad tracking service.
- abalone 10y agoYou didn't qualify your claim as only pertaining to small sites. You simply said surveillance is not the big problem for static content. Given that most static content is served by large sites and the article is about Youtube, you haven't really supported that claim.
- dredmorbius 10y agoI'd temper that statement significantly. Content integrity is the visible consequence of improved HTTPS compliance and use. The biggest factor about surveillance is that you are rarely aware you're being surveilled. Direct evidence is rarely present. Case in point, Michael Lewis's Flyboys. HFT trades intercepts weren't being overtly signalled, but were only evident when trades were structured such that they bypassed the opportunity to intercept intentions at the first market.
- JoshTriplett 10y ago> In short, some devices do not fully support modern HTTPS. I'd love to know which devices, and what "modern HTTPS" features they don't support.
- coob 10y agoAndroid < 4.1 doesn't properly support TLS 1.2. PayPal were originally going to stop supporting non TLS 1.2 (TLS 1.1, 1.0, SSL 2, SSL3) connections in June - they've now pushed this to next year: https://devblog.paypal.com/upcoming-security-changes-notice/ https://devblog.paypal.com/upcoming-security-changes-notice/
- benologist 10y agoMy NAS has security settings for modern, intermediate and old browsers that says it's to do with cipher suites they support - https://imgur.com/a/PmaUE https://imgur.com/a/PmaUE
- schoen 10y agoThose settings are likely to be derived from Mozilla's recommendations with the same names: https://wiki.mozilla.org/Security/Server_Side_TLS https://wiki.mozilla.org/Security/Server_Side_TLS You can read the details, rationale, and information about supported clients there.
- joshmn 10y agoLove, love love love love LOVE my Synology!
- ashmud 10y agoOn the desktop side, IE9, the last version on Vista, which is in extended support until April 11, 2017, only supports up to TLS 1.0.
- drzaiusapelord 10y agoIts only fairly recently that Android got TLS 1.2 support with 4.1 in 2012. The problem is that for years budget Chinese android phones were shipping with 2.3, which was very light on resources and good for low powered phones with minimal ram. I believe this changed with an effort to use less ram and cpu with 4.4 or 5.0, but by then millions of these phones were sold and are still out there. Heck, up until a year or two ago, these were on shelves in the US at budget places like Cricket. I think almost 10% of Androids in the wild still don't support 1.2. Maybe more considering Google has limited snooping abilities in China and may not fully know the extent of these installs. IE7/IE8/IE9 have a combined global marketshare of about 8% also.
- dchuk 10y agoWhat a strange thing to automatically (I assume) add to the end of these blogposts: > Sean Watson, Software Engineer, recently watched "GoPro: Fire Vortex Cannon with the Backyard Scientist." > Jon Levine, Product Manager, recently watched "Sega Saturn CD - Cracked after 20 years."
- TD-Linux 10y agoAt least they didn't watch anything embarassing. My Youtube recommendations page is a dumpster fire.
- munificent 10y agoTD-Linux: Hacker News Commenter - Last watched "Relaxing Background Videos Volume IIV: 4 Hours of a Peaceful Dumpster Fire (with sound)".
- dredmorbius 10y agoClearing YT history often helps. I'd vastly prefer channel blocks.
- softawre 10y agoI highly doubt that is automatically linking videos. Click the Sega one, it's linked to a specific spot in the video.
- jplevine 10y agoNot automatically included :) I came across the Sega Saturn video a couple weeks ago from this thread: https://news.ycombinator.com/item?id=12074096 https://news.ycombinator.com/item?id=12074096
- niftich 10y agoIs there an 'Unencrypted user traffic by device type' [1] chart just for YouTube traffic, as opposed to all Google services? Specifically, are non-HTTPS connections coming from desktops, mobile/device outside of an official app, or mobile/device through an official app? [1] https://www.google.com/transparencyreport/https?hl=en#devices https://www.google.com/transparencyreport/https?hl=en#device...
- drewg123 10y agoIt would be interesting to know how much overhead HTTPS added for them. Eg, a comparison of CPU usage between HTTP and HTTPS on the server side, whether or not they are using hardware offloads, etc. Netflix has some papers about their experience transitioning to HTTPS: https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf https://people.freebsd.org/~rrs/asiabsd_tls_improved.pdf https://people.freebsd.org/~rrs/asiabsd_tls_improved.pdf However, I expect the YouTube workload would be vastly different from the Netflix workload due to the sheer size of the YouTube catalog.
- mschwaig 10y agoI would also be interested in that. What differnce would you expect from the catalog size? I would guess that aside from maybe the handshake, decoupling storage and encryption would not be worth it in all cases where the storage medium is slower than AES-NI and this would apply even more to youtube than to netflix, exactly because of the bigger catalog.
- bluedino 10y ago>> What differnce would you expect from the catalog size? The Netflix OpenConnect applicances were maxing out the local SSD storage - YouTube isn't keeping as high of a percentage of active content on the servers so I imagine the CPU's aren't being kept as busy, or there's less CPU in their web-facing boxes. NetFlix is using 28-core CPU's in some of their latest machines.
- drewg123 10y agoWith a small catalog, you can pre-encode everything. That means that in the unencrypted case, media files can be sent directly from the kernel's page cache, as unmapped I/O. Eg, the data is not even touched by the CPU. But in the encrypted case, you're suddenly touching every byte with the CPU, thereby doubling your memory bandwidth requirements. With a large catalog like YouTube's, I would expect that most of the titles are saved in some master format, and are encoded on the fly. That means that the data is already being touched by the CPU to re-encode it on the fly for whatever format is needed by a particular client. My expectation is that if YouTube was re-encoding on the fly due to the catalog size, they would suffer less of a CPU hit than Netflix did, so SSL would be "easier" for them to implement.
- manueltenerife 10y agoThe most interesting piece of information I could take out of this is that some ISPs actually degrade the quality of http video streaming. They do it on purpose?
- richard_mcp 10y agoLast I checked, YouTube still does not support encryption when broadcasting a live stream.
- deleted 10y ago[deleted]
- 0x0 10y agoSo annoying when websites hijack pinch-to-zoom touch events and replace them with "load a random article".
- homero 10y agoHow'd finance decrease
- greenimpala 10y agoThe irony eh!
- minikites 10y ago> Today we added YouTube to Google's HTTPS transparency report. We're proud to announce that in the last two years, we steadily rolled out encryption using HTTPS to 97 percent of YouTube's traffic. What's the point of the report if you only add things that are already in good shape?