4 ms·
On the protocol level everything is working as expected, but I think the flaw is that Google's way of dealing with these unknown fields in the ad data structure
by jeroenhd 2y ago
On the protocol level everything is working as expected, but I think the flaw is that Google's way of dealing with these unknown fields in the ad data structure isn't to throw an error, but to pretend there are no ads to play. After all, Google will definitely release a new version of their app before they modify the protocol to make all the old versions not play ads anymore.
Google could shut down this method of ad blocking instantly by either doing basic certificate pinning or by altering their decoding logic to be less graceful of failures when it comes to extracting ad information. If I were on the YouTube team, I'd consider these flaws.
- wongarsu 2y agoSmoothly handling missing or unexpected fields is half the value proposition of protobuf. May as well switch a a much simpler versioned binary protocol instead of all this schema and field tagging complexity if you want to reject every message that doesn't match the client's schema. But rejecting unknown messages would likely degrade the user experience. Just because Google releases a new version doesn't mean everyone instantly has that new version installed everywhere. Certificate pinning would be a solution, but the world seems to have decided that that's very difficult to get right. Probably easier to get right in an app than in a website, but I understand not using it. They could manually sign the protobuf messages to ensure integrity. Duplicating some of the work TLS would already do, but doing it decoupled from TLS infrastructure may be easier. But unless something like OP's hack becomes mainstream, Google's current approach could be the right one. Sure, it leaves them open to message manipulation, but the potential lost ad revenue from even a tiny failure rate around update time from the other approaches could easily outweigh what they lose from a handful of people running middleware boxes to block ads.
- immibis 2y agoAFAIK ignoring unknown fields is a MUST in the protobuf spec. It's safe to assume all of their tooling is built around this. It wouldn't make a difference anyway. As vitus pointed out, deleting the field isn't much harder than changing it to an unknown field. You can do cert pinning. And the user can modify the app to pin their own cert. And you can lock down the device so the user can't modify the app. And the user can get a different device where they can modifiable apps. And you can add device attestation. And it's not yet feasible to extract an attestation key from a device, but it probably will be in the future. And then you will switch it to a physically uncloneable function. And then someone will figure out how to physically clone it anyway. And so on. The war on ad-blocking is fundamentally the war on general-purpose computing. By the time you achieve unskippable ad blocking, You know, you could also just refuse to send any video segments until the time when the ad is supposed to be over. Then the user may try to download their videos in advance, but most of the time they don't know what they're going to watch that far in advance, so they'll sit through the ad to avoid sitting through a black screen. That seems like a more sane thing to try. And you don't have to destroy the fabric of society to do it.
- wruza 2y agothey'll sit through the ad to avoid sitting through a black screen Bold claim.
- max-privatevoid 2y agoAFAIK Twitch already sends a simple "there's supposed to be an ad here" video stream during ad breaks when viewing through a non-web player. I'd take 30 seconds of silence over 30 seconds of advertisements any day of the week.
- xarope 2y ago+1. Especially those obnoxious, super-loud ads (seems like it's 99% of ads these days)
- alwa 2y agoI’d be thrilled to have a black screen instead of ads. I suspect you assume that my motivation is to escape the interruption. To me, let me tell you—the interruption is a mild annoyance, while the ads themselves are an active assault. Especially now that they’re individually and programmatically targeted to showcase and inflame the neuroses, health concerns, and predilections of each of the specific people in the room around the television. Amazing how the HIV commercials only appear when individuals in risk groups are around. Those outed a friend once—luckily in a supportive environment. And the random miscellaneous cancer drug ads come on when the friend who’s an older cancer survivor comes to visit. And the sports betting ads when friends facing gambling addiction are around. And if I hear one more ad hawking supplies for squealing tiny humans when new parents are around, so help me…
- wruza 2y agoInstead of a black screen, I'd like to watch calm cat/dog sleeping videos when they try to serve ads. Maybe one day uBO will add this opt-in feature, like mute the lie and overlay it with a 5-minute preloaded neutral cute content, for desperate services that force users into it. PS. spent 5 minutes on "a dog sleeping" without stupid music and couldn't find one. Search these days, man...
- dcow 2y agoThe point is that when ignoring unknown fields leads to users defeating your business model, even though the protocol requires it, the business logic of your software almost certainly should not. Protobuf working as intended, youtube client not so much. Agree that cert pinning is not the solution.
- echelon 2y agoProtobuf and the app are behaving the right way. 99.999% of users aren't going to MitM your protocol. How exactly should you deploy client code to the edge, which may not be updated, to handle "unknown" tag number fields? You don't, because that's crazy. Nobody should write software like that because it creates a maintenance hell where you can't upgrade or downgrade because "smart" applications are doing stupid, undeterministic things with the protocol. It's impossible to reason and engineer backward / forward compatibility when you don't treat the wire format and API with respect. Most of the major migration headaches I've had in my career have been the result of engineers trying to be clever in the time and place they wrote the code.
- lokar 2y agoThe next best option is to version the api and support all old versions going back 10+ years I agree, what they are doing with proto is much better
- dcow 2y agoUh… how about display an error message if the user isn’t subscribed to premium and the server doesn't send any ads?
- deleted 2y ago[deleted]
- jeroenhd 2y agoYouTube often degrades the user experience when it suspects ads being blocked. I think the idea that Google would care more about following prorobuf's best practices than rejecting ad blockers is strange from a business perspective. Cert pinning for YouTube would actually be quite easy, as Google runs its own CA. They can just hard pin their root CA and update the app in twenty years or so when that expires.
- wat10000 2y agoIf you want to prevent MitM modification attacks, the way to do it is to sign the data. Trying to do it by making the serialization format less forgiving isn’t the right approach at all. It still has to be pretty flexible. It’s going to be pretty hard to come up with a format that cannot possibly be altered to make the client show no ads. Something like certificate pinning is way easier.
- fastball 2y agoI'm very surprised Google isn't already certificate pinning.
- kbolino 2y agoThe current YouTube certificate has a lifetime of 84 days. The next one will live that long or even shorter. Certificates are rotated too quickly nowadays for pinning to be viable. They'd be better off pinning their CA than their certificate, but that might cause issues for users in certain scenarios.
- dcow 2y agoCert pinning is not a solution and defeats legitimate traffic inspection cases. It also just moves the goalposts as the author discusses it can be defeated by a modified app binary. The industry has moved on. Sign your data if modification in-flight is a threat. I’m surprised google isn't doing that.
- dambi0 2y agoAlternatively perhaps the cost of implementing / performing the overhead of signing and verification isn’t worth the cost given the relatively few people who are likely to do this
- JoshTriplett 2y ago> Sign your data if modification in-flight is a threat. Which effectively pins a different key in the binary. Might as well use certificate pinning, which provides both signing and encryption without a second layer of crypto.
- dcow 2y agoOnly naively. You can do a key exchange.
- gowld 2y agoA legitimiate traffic inspection case, from the content publisher's perspective, would have an alternate certificate backed by a trusted authority who agreed to protect the ads.
- deleted 2y ago[deleted]
- gowld 2y agoFunny you mention that, because unnecessary "security" controls on streaming data is how Google broke Chromecasts lacked week, and still hasn't figured out how to fix.
- jeroenhd 2y agoI haven't had any trouble on my dumb Chromecast so I think they must've pushed an update to fix the issue already. The only app I remember breaking was Jellyfin, ironically enough. Security controls weren't the reason the devices broke, which is why applications that didn't care about things like DRM still played just fine. The internal CA for the Chromecast hardware certification expired, which requires updating all Chromecasts or temporarily ignoring the expiry date in client apps. It seems like apps are doing the latter while Google is figuring out how to update the Chromecast certificate infra on short notice.
- smadge 2y agoThoughts my own, smooth, uninterrupted video playback is much higher priority than recovering a small amount of ad revenue.
- smileybarry 2y agoIronically, Protobuf does have field options for "required" and "optional", but over a decade ago (I guess shortly after it was first introduced), Google added a huge "do not use required! validate required fields in your own logic!" warning to the docs, because a missing "required" field throws out the whole packet. (And when it's time to deprecate a required field -- it's way more effort) We originally used "required" for fields that must be present and it screwed us over later due to that. Switched everything to "optional" with the code checking for those fields' existence instead.