5 ms·

How Modern Video Players Work

73 points by flavioribeiro 11y ago

eldod 11y ago
A very good article for those who would like to follow the lead of NYT, Youtube or others who recently switched to all HTML5
zzalpha 11y ago
Oh. Well that's disappointing. I was hoping this would be about a modern video/audio decode, synchronize, and display pipeline... not that knowing how video streamed on the web isn't interesting, but it's really about how modern video streaming works.
- smalley 11y ago
  Not that its particularly casually browsable but there are some details in the programmer reference manuals for the Intel integrated graphics encode/decode hardware. If you head to volume 8 the MFx introduction section has some sample flows before it starts getting into command packet details: https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-bdw-vol08-media_vdbox_3.pdf https://01.org/sites/default/files/documentation/intel-gfx-p...
- eldod 11y ago
  It talks about modern streaming indeed, but if you take a closer look, most of the video consumption today is done by streaming: on Desktop you still have a small portions of download then watch, but on mobile, Set Top Boxes like AppleTV and Roku, connected TVs, etc. You watch everything from the web. Besides, the decoding/sync/rendering part didn't really change in the last 10 years, the innovations mostly arrived from the Web streaming use cases, with adaptive bitrate formats, that partly solved the multi-devices streaming problem we started to have with the rise of Mobile and OTT (over-the-top) video streaming.
Nutmog 11y ago
All this fancy software and yet watching videos on the web is just as painful as it was 15 years ago if you don't have a fast enough connection. The download won't keep up with the play speed and they don't accommodate that nearly as well as they could. In the early 2000's with dial-up, you could just press pause then come back a few minutes later to watch what had been buffered. Nowadays that trick often doesn't work - pause also pauses the download. Not only that but the play/pause button sometimes doesn't work at all! If the playback is stopped because it's waiting to download more, the pause button usually does nothing - so you have to rewind to a previously buffered part (if you're lucky enough and it kept the buffer) then let it play, then press pause.
- aaroninsf 11y ago
  This! Related gripe: WTH is there no full-track buffering mode for any web players (looking at You, Tube!) and apps like Pandora or Spotify, for low- or sparse- bandwidth travel?! I don't care if you limit it geographically or only make it work with bandwidth throttling or whatever... ...just make it work. Then your apps would work and I would not be cursing and throwing my phone out the window every road trip. This is costing me a fortune in phones.
  - bagels 11y ago
    The other side of the equation involves at least: 1) Video provider pays for bytes sent to the user that are not used. If the cache policy is "cache the whole thing", you would find yourself paying (a guestimate) 2x for bandwidth. This is because most people do not watch even nearly the entirety of every video that they start loading. 2) Using server bandwidth for bytes that are not used. This increases contention for bandwidth at the origin. Perhaps less of an issue if your site is small and you're using a giant CDN. This does become an issue if you are youtube or you are serving the bytes yourself. 3) Using client bandwidth for bytes that are not used. This slows down any other download that might be happening on the client computer, even when the video is paused. This could be, a different video in a different tab that the user is trying to watch, other assets on the page, other pages, or even other programs on the computer. Of course adding an option for the user to cache the whole thing wouldn't be so bad, as opposed to having this behavior by default for everyone.

73 points by flavioribeiro 11y ago

eldod 11y ago

A very good article for those who would like to follow the lead of NYT, Youtube or others who recently switched to all HTML5

zzalpha 11y ago

Oh. Well that's disappointing. I was hoping this would be about a modern video/audio decode, synchronize, and display pipeline... not that knowing how video streamed on the web isn't interesting, but it's really about how modern video streaming works.

smalley 11y ago
Not that its particularly casually browsable but there are some details in the programmer reference manuals for the Intel integrated graphics encode/decode hardware. If you head to volume 8 the MFx introduction section has some sample flows before it starts getting into command packet details: https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-bdw-vol08-media_vdbox_3.pdf https://01.org/sites/default/files/documentation/intel-gfx-p...
eldod 11y ago
It talks about modern streaming indeed, but if you take a closer look, most of the video consumption today is done by streaming: on Desktop you still have a small portions of download then watch, but on mobile, Set Top Boxes like AppleTV and Roku, connected TVs, etc. You watch everything from the web. Besides, the decoding/sync/rendering part didn't really change in the last 10 years, the innovations mostly arrived from the Web streaming use cases, with adaptive bitrate formats, that partly solved the multi-devices streaming problem we started to have with the rise of Mobile and OTT (over-the-top) video streaming.

Nutmog 11y ago

All this fancy software and yet watching videos on the web is just as painful as it was 15 years ago if you don't have a fast enough connection. The download won't keep up with the play speed and they don't accommodate that nearly as well as they could. In the early 2000's with dial-up, you could just press pause then come back a few minutes later to watch what had been buffered. Nowadays that trick often doesn't work - pause also pauses the download. Not only that but the play/pause button sometimes doesn't work at all! If the playback is stopped because it's waiting to download more, the pause button usually does nothing - so you have to rewind to a previously buffered part (if you're lucky enough and it kept the buffer) then let it play, then press pause.

aaroninsf 11y ago
This! Related gripe: WTH is there no full-track buffering mode for any web players (looking at You, Tube!) and apps like Pandora or Spotify, for low- or sparse- bandwidth travel?! I don't care if you limit it geographically or only make it work with bandwidth throttling or whatever... ...just make it work. Then your apps would work and I would not be cursing and throwing my phone out the window every road trip. This is costing me a fortune in phones.
- bagels 11y ago
  The other side of the equation involves at least: 1) Video provider pays for bytes sent to the user that are not used. If the cache policy is "cache the whole thing", you would find yourself paying (a guestimate) 2x for bandwidth. This is because most people do not watch even nearly the entirety of every video that they start loading. 2) Using server bandwidth for bytes that are not used. This increases contention for bandwidth at the origin. Perhaps less of an issue if your site is small and you're using a giant CDN. This does become an issue if you are youtube or you are serving the bytes yourself. 3) Using client bandwidth for bytes that are not used. This slows down any other download that might be happening on the client computer, even when the video is paused. This could be, a different video in a different tab that the user is trying to watch, other assets on the page, other pages, or even other programs on the computer. Of course adding an option for the user to cache the whole thing wouldn't be so bad, as opposed to having this behavior by default for everyone.