8 ms·

> instead, I found a flaw in the Protobuf format which allows me to reliably change one byte to obliterate ads. Let me guess, the author changed the field numb

by vitus 2y ago

> instead, I found a flaw in the Protobuf format which allows me to reliably change one byte to obliterate ads. Let me guess, the author changed the field number to a large unused number. > Now, all we have to do is scan the Protobuf bytes for classic ad URL signatures like /pagead/ to bound our field search, then move backward from there until we find the target(s) field tags and thus field keys we would like to denature (e.g. 49399797 –> 49399796). Yeah. This isn't a flaw, this is intended behavior. If you're willing to go through the effort to find the tag, it's really not that much additional effort to then read the (varint) length right next to the tag and... just skip those bytes. Yes, you'd need to copy your buffer to do this, or at least slide your bytes around. But the proof-of-concept script already has to perform a copy because the bytes object returned by mitmproxy's API (`body: bytearray = bytearray(flow.response.get_content(strict=False) or b"")`) is immutable, and even a memoryview isn't going to bypass this limitation.

jeroenhd 2y ago
On the protocol level everything is working as expected, but I think the flaw is that Google's way of dealing with these unknown fields in the ad data structure isn't to throw an error, but to pretend there are no ads to play. After all, Google will definitely release a new version of their app before they modify the protocol to make all the old versions not play ads anymore. Google could shut down this method of ad blocking instantly by either doing basic certificate pinning or by altering their decoding logic to be less graceful of failures when it comes to extracting ad information. If I were on the YouTube team, I'd consider these flaws.
- wongarsu 2y ago
  Smoothly handling missing or unexpected fields is half the value proposition of protobuf. May as well switch a a much simpler versioned binary protocol instead of all this schema and field tagging complexity if you want to reject every message that doesn't match the client's schema. But rejecting unknown messages would likely degrade the user experience. Just because Google releases a new version doesn't mean everyone instantly has that new version installed everywhere. Certificate pinning would be a solution, but the world seems to have decided that that's very difficult to get right. Probably easier to get right in an app than in a website, but I understand not using it. They could manually sign the protobuf messages to ensure integrity. Duplicating some of the work TLS would already do, but doing it decoupled from TLS infrastructure may be easier. But unless something like OP's hack becomes mainstream, Google's current approach could be the right one. Sure, it leaves them open to message manipulation, but the potential lost ad revenue from even a tiny failure rate around update time from the other approaches could easily outweigh what they lose from a handful of people running middleware boxes to block ads.
  - immibis 2y ago
    AFAIK ignoring unknown fields is a MUST in the protobuf spec. It's safe to assume all of their tooling is built around this. It wouldn't make a difference anyway. As vitus pointed out, deleting the field isn't much harder than changing it to an unknown field. You can do cert pinning. And the user can modify the app to pin their own cert. And you can lock down the device so the user can't modify the app. And the user can get a different device where they can modifiable apps. And you can add device attestation. And it's not yet feasible to extract an attestation key from a device, but it probably will be in the future. And then you will switch it to a physically uncloneable function. And then someone will figure out how to physically clone it anyway. And so on. The war on ad-blocking is fundamentally the war on general-purpose computing. By the time you achieve unskippable ad blocking, You know, you could also just refuse to send any video segments until the time when the ad is supposed to be over. Then the user may try to download their videos in advance, but most of the time they don't know what they're going to watch that far in advance, so they'll sit through the ad to avoid sitting through a black screen. That seems like a more sane thing to try. And you don't have to destroy the fabric of society to do it.
    - wruza 2y ago
      they'll sit through the ad to avoid sitting through a black screen Bold claim.
      - max-privatevoid 2y ago
        AFAIK Twitch already sends a simple "there's supposed to be an ad here" video stream during ad breaks when viewing through a non-web player. I'd take 30 seconds of silence over 30 seconds of advertisements any day of the week.
        xarope 2y ago
        +1. Especially those obnoxious, super-loud ads (seems like it's 99% of ads these days)
    - 2y ago
- wat10000 2y ago
  If you want to prevent MitM modification attacks, the way to do it is to sign the data. Trying to do it by making the serialization format less forgiving isn’t the right approach at all. It still has to be pretty flexible. It’s going to be pretty hard to come up with a format that cannot possibly be altered to make the client show no ads. Something like certificate pinning is way easier.
- fastball 2y ago
  I'm very surprised Google isn't already certificate pinning.
  - kbolino 2y ago
    The current YouTube certificate has a lifetime of 84 days. The next one will live that long or even shorter. Certificates are rotated too quickly nowadays for pinning to be viable. They'd be better off pinning their CA than their certificate, but that might cause issues for users in certain scenarios.
  - dcow 2y ago
    Cert pinning is not a solution and defeats legitimate traffic inspection cases. It also just moves the goalposts as the author discusses it can be defeated by a modified app binary. The industry has moved on. Sign your data if modification in-flight is a threat. I’m surprised google isn't doing that.
    - dambi0 2y ago
      Alternatively perhaps the cost of implementing / performing the overhead of signing and verification isn’t worth the cost given the relatively few people who are likely to do this
    - JoshTriplett 2y ago
      > Sign your data if modification in-flight is a threat. Which effectively pins a different key in the binary. Might as well use certificate pinning, which provides both signing and encryption without a second layer of crypto.
      - dcow 2y ago
        Only naively. You can do a key exchange.
    - gowld 2y ago
      A legitimiate traffic inspection case, from the content publisher's perspective, would have an alternate certificate backed by a trusted authority who agreed to protect the ads.
- deleted 2y ago
  [deleted]
- gowld 2y ago
  Funny you mention that, because unnecessary "security" controls on streaming data is how Google broke Chromecasts lacked week, and still hasn't figured out how to fix.
  - jeroenhd 2y ago
    I haven't had any trouble on my dumb Chromecast so I think they must've pushed an update to fix the issue already. The only app I remember breaking was Jellyfin, ironically enough. Security controls weren't the reason the devices broke, which is why applications that didn't care about things like DRM still played just fine. The internal CA for the Chromecast hardware certification expired, which requires updating all Chromecasts or temporarily ignoring the expiry date in client apps. It seems like apps are doing the latter while Google is figuring out how to update the Chromecast certificate infra on short notice.
- smadge 2y ago
  Thoughts my own, smooth, uninterrupted video playback is much higher priority than recovering a small amount of ad revenue.
- smileybarry 2y ago
  Ironically, Protobuf does have field options for "required" and "optional", but over a decade ago (I guess shortly after it was first introduced), Google added a huge "do not use required! validate required fields in your own logic!" warning to the docs, because a missing "required" field throws out the whole packet. (And when it's time to deprecate a required field -- it's way more effort) We originally used "required" for fields that must be present and it screwed us over later due to that. Switched everything to "optional" with the code checking for those fields' existence instead.
sgarland 2y ago
> the bytes object returned by mitmproxy's API (`body: bytearray = bytearray(flow.response.get_content(strict=False) or b"")`) is immutable Byte objects are immutable, but bytearray objects are not.
- vitus 2y ago
  Yes, that's my point, although I can see why you interpreted it otherwise. mitmproxy's API is flow.response.get_content(strict=False), which returns a bytes object; the proof-of-concept script then copies it into the bytearray using the code that I've cited.
samstave 2y ago
[dead]

by vitus 2y ago

jeroenhd 2y ago

On the protocol level everything is working as expected, but I think the flaw is that Google's way of dealing with these unknown fields in the ad data structure isn't to throw an error, but to pretend there are no ads to play. After all, Google will definitely release a new version of their app before they modify the protocol to make all the old versions not play ads anymore. Google could shut down this method of ad blocking instantly by either doing basic certificate pinning or by altering their decoding logic to be less graceful of failures when it comes to extracting ad information. If I were on the YouTube team, I'd consider these flaws.

wongarsu 2y ago
Smoothly handling missing or unexpected fields is half the value proposition of protobuf. May as well switch a a much simpler versioned binary protocol instead of all this schema and field tagging complexity if you want to reject every message that doesn't match the client's schema. But rejecting unknown messages would likely degrade the user experience. Just because Google releases a new version doesn't mean everyone instantly has that new version installed everywhere. Certificate pinning would be a solution, but the world seems to have decided that that's very difficult to get right. Probably easier to get right in an app than in a website, but I understand not using it. They could manually sign the protobuf messages to ensure integrity. Duplicating some of the work TLS would already do, but doing it decoupled from TLS infrastructure may be easier. But unless something like OP's hack becomes mainstream, Google's current approach could be the right one. Sure, it leaves them open to message manipulation, but the potential lost ad revenue from even a tiny failure rate around update time from the other approaches could easily outweigh what they lose from a handful of people running middleware boxes to block ads.
- immibis 2y ago
  AFAIK ignoring unknown fields is a MUST in the protobuf spec. It's safe to assume all of their tooling is built around this. It wouldn't make a difference anyway. As vitus pointed out, deleting the field isn't much harder than changing it to an unknown field. You can do cert pinning. And the user can modify the app to pin their own cert. And you can lock down the device so the user can't modify the app. And the user can get a different device where they can modifiable apps. And you can add device attestation. And it's not yet feasible to extract an attestation key from a device, but it probably will be in the future. And then you will switch it to a physically uncloneable function. And then someone will figure out how to physically clone it anyway. And so on. The war on ad-blocking is fundamentally the war on general-purpose computing. By the time you achieve unskippable ad blocking, You know, you could also just refuse to send any video segments until the time when the ad is supposed to be over. Then the user may try to download their videos in advance, but most of the time they don't know what they're going to watch that far in advance, so they'll sit through the ad to avoid sitting through a black screen. That seems like a more sane thing to try. And you don't have to destroy the fabric of society to do it.
  - wruza 2y ago
    they'll sit through the ad to avoid sitting through a black screen Bold claim.
    - max-privatevoid 2y ago
      AFAIK Twitch already sends a simple "there's supposed to be an ad here" video stream during ad breaks when viewing through a non-web player. I'd take 30 seconds of silence over 30 seconds of advertisements any day of the week.
      - xarope 2y ago
        +1. Especially those obnoxious, super-loud ads (seems like it's 99% of ads these days)
  - 2y ago
wat10000 2y ago
If you want to prevent MitM modification attacks, the way to do it is to sign the data. Trying to do it by making the serialization format less forgiving isn’t the right approach at all. It still has to be pretty flexible. It’s going to be pretty hard to come up with a format that cannot possibly be altered to make the client show no ads. Something like certificate pinning is way easier.
fastball 2y ago
I'm very surprised Google isn't already certificate pinning.
- kbolino 2y ago
  The current YouTube certificate has a lifetime of 84 days. The next one will live that long or even shorter. Certificates are rotated too quickly nowadays for pinning to be viable. They'd be better off pinning their CA than their certificate, but that might cause issues for users in certain scenarios.
- dcow 2y ago
  Cert pinning is not a solution and defeats legitimate traffic inspection cases. It also just moves the goalposts as the author discusses it can be defeated by a modified app binary. The industry has moved on. Sign your data if modification in-flight is a threat. I’m surprised google isn't doing that.
  - dambi0 2y ago
    Alternatively perhaps the cost of implementing / performing the overhead of signing and verification isn’t worth the cost given the relatively few people who are likely to do this
  - JoshTriplett 2y ago
    > Sign your data if modification in-flight is a threat. Which effectively pins a different key in the binary. Might as well use certificate pinning, which provides both signing and encryption without a second layer of crypto.
    - dcow 2y ago
      Only naively. You can do a key exchange.
  - gowld 2y ago
    A legitimiate traffic inspection case, from the content publisher's perspective, would have an alternate certificate backed by a trusted authority who agreed to protect the ads.
deleted 2y ago
[deleted]
gowld 2y ago
Funny you mention that, because unnecessary "security" controls on streaming data is how Google broke Chromecasts lacked week, and still hasn't figured out how to fix.
- jeroenhd 2y ago
  I haven't had any trouble on my dumb Chromecast so I think they must've pushed an update to fix the issue already. The only app I remember breaking was Jellyfin, ironically enough. Security controls weren't the reason the devices broke, which is why applications that didn't care about things like DRM still played just fine. The internal CA for the Chromecast hardware certification expired, which requires updating all Chromecasts or temporarily ignoring the expiry date in client apps. It seems like apps are doing the latter while Google is figuring out how to update the Chromecast certificate infra on short notice.
smadge 2y ago
Thoughts my own, smooth, uninterrupted video playback is much higher priority than recovering a small amount of ad revenue.
smileybarry 2y ago
Ironically, Protobuf does have field options for "required" and "optional", but over a decade ago (I guess shortly after it was first introduced), Google added a huge "do not use required! validate required fields in your own logic!" warning to the docs, because a missing "required" field throws out the whole packet. (And when it's time to deprecate a required field -- it's way more effort) We originally used "required" for fields that must be present and it screwed us over later due to that. Switched everything to "optional" with the code checking for those fields' existence instead.

sgarland 2y ago

> the bytes object returned by mitmproxy's API (`body: bytearray = bytearray(flow.response.get_content(strict=False) or b"")`) is immutable Byte objects are immutable, but bytearray objects are not.

vitus 2y ago
Yes, that's my point, although I can see why you interpreted it otherwise. mitmproxy's API is flow.response.get_content(strict=False), which returns a bytes object; the proof-of-concept script then copies it into the bytearray using the code that I've cited.

samstave 2y ago

[dead]