5 ms·

How long before the ads are realtime encoded into the video streams such that even youtube-dl can't bypass them without a premium login? I've been surprised th

How long before the ads are realtime encoded into the video streams such that even youtube-dl can't bypass them without a premium login? I've been surprised this wasn't already the case, but assumed it was just an encoding overhead issue vs. just serving pre-encoded videos for both the content and ads with necessarily well-defined stream boundaries separating them.

martin-adams 6y ago
I suspect doing personalised ads obliterates any caching method on cheaper hardware than transcoding servers. Interesting problem to solve though.
- amelius 6y ago
  > Interesting problem to solve though. Ah, smart people and ads ...
- jwandborg 6y ago
  The ads are not part of the encoded video AFAICT, they are probably served as a separate stream which the client requests alongside the regular video stream, this means that videos and ads can be cached using traditional techniques.
garblegarble 6y ago
You wouldn't even need to do real-time encoding for that, you can simply mux them in at any GOP boundary (other services already do real-time ad insertion in MPEG-DASH manifests) Example: https://www.youtube.com/watch?v=LFHEko3vC98 https://www.youtube.com/watch?v=LFHEko3vC98
- elithrar 6y ago
  Right, using DAI means you don’t have to actually touch the original video (good!) but doesn’t stop a smart enough client (youtube-dl) from pattern matching and ignoring those segments when stitching the final video together. I am not, however, suggesting that encoding ads into the final stream is appropriate or scalable, though!
  - kevincox 6y ago
    The client doesn't even have to know that there is an ad playing if they really want to thwart ad blockers. If you are talking about pattern-matching the actual video stream ad-blockers could do that today and just seek forwards but none do yet.
callamdelaney 6y ago
Then you could just skip the ad in the video, unless the player has some meta-data around when the ad is; in which case youtube-dl can chop it out.
- pengaru 6y ago
  Not if you tightly control the streaming rate to not get far ahead of a realtime playback, just mete out the video stream at a rate appropriate for watching, not as fast as the pipe can suck it down.
  - londons_explore 6y ago
    I'm kinda surprised Google doesn't do this... They would need to keep track of user seeks and stuff, but it still seems do-able. One simple model is for the server to know when ad-breaks should happen, and prevent any more downloading for the duration of the ad. Sure, it would break people who want to watch at 2x realtime, but they seem small-fry compared to those with adblockers.
    - giantrobot 6y ago
      The issue there is scale, MPEG-DASH/HLS let the edge servers for video to be simple. The servers don't need to do much more than serve up bytes via HTTP. This ends up being better for clients, especially mobile clients, since they can choose streams based on their local conditions the server couldn't know about like downgrading from LTE to UMTS. Google would end up having to maintain a lot of extra client state on their edge servers if they wanted to do that all in-band. Right now it's done out of band with their JavaScript player. Chasing down youtube-dl users isn't likely worth that extra cost.
      - londons_explore 6y ago
        The edge server could implement this without much extra complexity. For example each chunk URL could be signed with a "donotdeliverbefore" timestamp. Now the edge server has zero state. Similar things are done to prevent signed in URL's being shared with other users.
IshKebab 6y ago
That sounds like an enormous pain in the arse just to piss off a vocal minority of users.
- whywhywhywhy 6y ago
  A vocal minority who are not bringing in any revenue for the site. Saying that though the day they finally succeed in making ads unskippable will be the time for a competitor to move in.
  - JohnWhigham 6y ago
    Yeah, if YT wanted to insert unskippable ads on the backend, they would have years ago. The tech is not the hard part. They know it'd be a PR disaster for them.
  - corobo 6y ago
    When it's possible to skip baked in ads (SponsorBlock[1]) -- the whack-a-mole will continue no matter what. Even if it means you can't watch videos in realtime but have to wait for them to fully download to rip the ad out, someone will figure it out. At that time everyone starts talking about it and I gotta imagine a bunch of new people become adblocking users. [1] https://news.ycombinator.com/item?id=26886275 https://news.ycombinator.com/item?id=26886275
    - KingMachiavelli 6y ago
      SponsorBlock only works because the sponsored segments are at the same location for every viewer. If Youtube spliced in their own ads they could easily do it at variable intervals preventing any crowd sourced database of ad segment timestamps. To be honest, nothing really stops Youtube from just turning on Widevine encryption for all videos (not just purchased/rented TV & movies) besides breaking compatibility with old devices. Sure widevine can be circumvented but most of the best/working cracks are not public.
      - corobo 6y ago

martin-adams 6y ago

I suspect doing personalised ads obliterates any caching method on cheaper hardware than transcoding servers. Interesting problem to solve though.

amelius 6y ago
> Interesting problem to solve though. Ah, smart people and ads ...
jwandborg 6y ago
The ads are not part of the encoded video AFAICT, they are probably served as a separate stream which the client requests alongside the regular video stream, this means that videos and ads can be cached using traditional techniques.

garblegarble 6y ago

You wouldn't even need to do real-time encoding for that, you can simply mux them in at any GOP boundary (other services already do real-time ad insertion in MPEG-DASH manifests) Example: https://www.youtube.com/watch?v=LFHEko3vC98 https://www.youtube.com/watch?v=LFHEko3vC98

elithrar 6y ago
Right, using DAI means you don’t have to actually touch the original video (good!) but doesn’t stop a smart enough client (youtube-dl) from pattern matching and ignoring those segments when stitching the final video together. I am not, however, suggesting that encoding ads into the final stream is appropriate or scalable, though!
- kevincox 6y ago
  The client doesn't even have to know that there is an ad playing if they really want to thwart ad blockers. If you are talking about pattern-matching the actual video stream ad-blockers could do that today and just seek forwards but none do yet.

callamdelaney 6y ago

Then you could just skip the ad in the video, unless the player has some meta-data around when the ad is; in which case youtube-dl can chop it out.

pengaru 6y ago
Not if you tightly control the streaming rate to not get far ahead of a realtime playback, just mete out the video stream at a rate appropriate for watching, not as fast as the pipe can suck it down.
- londons_explore 6y ago
  I'm kinda surprised Google doesn't do this... They would need to keep track of user seeks and stuff, but it still seems do-able. One simple model is for the server to know when ad-breaks should happen, and prevent any more downloading for the duration of the ad. Sure, it would break people who want to watch at 2x realtime, but they seem small-fry compared to those with adblockers.
  - giantrobot 6y ago
    The issue there is scale, MPEG-DASH/HLS let the edge servers for video to be simple. The servers don't need to do much more than serve up bytes via HTTP. This ends up being better for clients, especially mobile clients, since they can choose streams based on their local conditions the server couldn't know about like downgrading from LTE to UMTS. Google would end up having to maintain a lot of extra client state on their edge servers if they wanted to do that all in-band. Right now it's done out of band with their JavaScript player. Chasing down youtube-dl users isn't likely worth that extra cost.
    - londons_explore 6y ago
      The edge server could implement this without much extra complexity. For example each chunk URL could be signed with a "donotdeliverbefore" timestamp. Now the edge server has zero state. Similar things are done to prevent signed in URL's being shared with other users.

IshKebab 6y ago

That sounds like an enormous pain in the arse just to piss off a vocal minority of users.

whywhywhywhy 6y ago
A vocal minority who are not bringing in any revenue for the site. Saying that though the day they finally succeed in making ads unskippable will be the time for a competitor to move in.
- JohnWhigham 6y ago
  Yeah, if YT wanted to insert unskippable ads on the backend, they would have years ago. The tech is not the hard part. They know it'd be a PR disaster for them.
- corobo 6y ago
  When it's possible to skip baked in ads (SponsorBlock[1]) -- the whack-a-mole will continue no matter what. Even if it means you can't watch videos in realtime but have to wait for them to fully download to rip the ad out, someone will figure it out. At that time everyone starts talking about it and I gotta imagine a bunch of new people become adblocking users. [1] https://news.ycombinator.com/item?id=26886275 https://news.ycombinator.com/item?id=26886275
  - KingMachiavelli 6y ago
    SponsorBlock only works because the sponsored segments are at the same location for every viewer. If Youtube spliced in their own ads they could easily do it at variable intervals preventing any crowd sourced database of ad segment timestamps. To be honest, nothing really stops Youtube from just turning on Widevine encryption for all videos (not just purchased/rented TV & movies) besides breaking compatibility with old devices. Sure widevine can be circumvented but most of the best/working cracks are not public.
    - corobo 6y ago