6 ms·

I believe that's standard for Netflix, etc, but is it also true for plain webms and mp4s in a <video> tags? I thought those were downloaded in one request but h

I believe that's standard for Netflix, etc, but is it also true for plain webms and mp4s in a <video> tags? I thought those were downloaded in one request but had enough metadata at the beginning to allow playback to start before the file is completely downloaded.

wewewedxfgdf 2y ago
Yes it is true. Browsers talking to static web servers use HTTP byte ranges requests to get chunks of videos and can use the same mechanism to seek to any point in the file. Streaming that way is fast and simple. No fancy technology required. For MP4 to work that we you need to render it as fragmented MP4.
- jofla_net 2y ago
  Seconded, ive done a userland 'content-range' implementation myself. of course there were a few ffmpeg specific parameters the mp4 needed to work right still
- EE84M3i 2y ago
  Why would the browser send byte range requests for video tags if it expects to play the file back linearly from beginning to end anyway? Wouldn't that be additional overhead/round-trips?
  - koakuma-chan 2y ago
    > Why would the browser send byte range requests for video tags if it expects to play the file back linearly from beginning to end anyway? Probably because byte range is required for seeking, and playing from the beginning is equivalent to seeking at 0. > Wouldn't that be additional overhead/round-trips? No because the range of the initial byte range request is the whole file (`bytes=0-`).
    - EE84M3i 2y ago
      My original comment was about the commenter I replied to saying: > To my knowledge, video stream requests chunks by range and is largely client controlled. It isn't a single, long lived http connection. Wouldn't a byte range request for the whole file fall under the "single, long lived http connection"? Sure it could be terminated early and another request made for seeking, but regardless the video can start before the whole file is downloaded, assuming it's encoded correctly?
      - tedunangst 2y ago
        The client doesn't want to eat the whole file, so it uses a range request for just the beginning of the file, and then the next part as needed.
        koakuma-chan 2y ago
        The client would actually request the whole file and then terminate the request if the file is no longer needed. This is what browsers do at least.
        lxgr 2y ago
        Both are possible, and in fact I could imagine not all servers being too happy with having to trickle data over a persistent HTTP connection through the entire length of the video, with an almost always full TCP send buffer at the OS level.
        koakuma-chan 2y ago
        > Both are possible It is possible if you are in control of the client, but no browser would stream an mp4 file request by request. > with an almost always full TCP send buffer at the OS level This shouldn't be a problem because there is flow control. Also the data would probably be sent to the kernel in small chunks, not the whole file at once.
- wordofx 2y ago
  It’s not true because throwing a video file as a source on video tag has no information about the file being requested until the headers are pushed down. Hell back in 2005 Akamai didn’t even support byte range headers for partial content delivery, which made resuming videos impossible, I believe they pushed out the update across their network in 06 or 07.
  - jonathanlydall 2y ago
    If your HTTP server provides and supports the appropriate headers and you’re serving supported file types, then it absolutely is true. Just putting a url in my Chromium based browser’s address bar to an mp4 file we have hosted on CloudFlare R2 “just works” (I expect a video tag would be the same), supporting skipping ahead in the video without having to download the whole thing. Initially skipping ahead didn’t work until I disabled caching on CloudFlare CDN as that breaks the “accept-range” capability on videos. For now we have negligible amount of viewership of these mp4s, but if it becomes an issue we’ll use CloudFlare’s video serving product.
    - wordofx 2y ago
      > If your HTTP server provides and supports the appropriate headers and you’re serving supported file types, then it absolutely is true. No. When you play a file in the browser with a video tag. It requests the file. It doesn’t ask for a range. It does use the range if you seek it, or you write the JavaScript to fetch based on a range. That’s why if you press play and pause it buffers the whole video. Only if you write the code yourself can you partially buffer a while like YouTube does.
      - bmacho 2y ago
        Nah, it uses complex video specific logic and http range requests as protocol. (At least the normal browsers and servers. You can roll your own dumb client/server of course.) > That’s why if you press play and pause it buffers the whole video. Browsers don't do that.
      - jonathanlydall 2y ago
jofzar 2y ago
The long answer is "it depends on how you do it" unsurprisingly video and voice/audio are probably the most different ways that you can "choose" to do distribution
- bhhaskin 2y ago
  This. You can't just throw it into a folder and have to stream. The web server has to support it and then there is encoding and formats.
AnotherGoodName 2y ago
They can playback as loading as long as they are encoded correctly fwiw (faststart encoded). When you create a video from a device the header is actually at the end of the file. Understandable, it’s where the file pointer was and mp4 allows this so your recording device writes it at the end. You must re-encoded with faststart (puts the moov atom at the start) to make it load reasonably on a webpage though.
- timewizard 2y ago
  > Understandable, it’s where the file pointer was and mp4 allows this so your recording device writes it at the end. Yet formats like WAVE which use a similar "chunked" encoding they just use a fixed length header and use a single seek() to get back to it when finalizing the file. Quicktime and WAVE were released around nearly the same time in the early 90s. MP2 was so much better I cringe every time I have to deal with MP4 in some context.
  - lxgr 2y ago
    At the expense of quite some overhead though, right? MPEG-2 transport streams seem more optimized for a broadcast context, with their small frame structure and everything – as far as I know, framing overhead is at least 2%, and is arguably not needed when delivered over a reliable unicast pipe such as TCP. Still, being able to essentially chop a single, progressively written MPEG TS file into various chunks via HTTP range requests or very simple file copy operations without having to do more than count bytes, and with self-synchronization if things go wrong, is undoubtedly nicer to work with than MP4 objects. I suppose that's why HLS started out with transport streams and only gained fMP4 support later on.
    - timewizard 2y ago
      > and is arguably not needed when delivered over a reliable unicast pipe such as TCP. So much content ended up being delivered this way, but there was a brief moment where we thought multicast UDP would be much more prevalent than it ended up being. In that context it's perfect. > why HLS started out with transport streams and only gained fMP4 support later on. Which I actually think was the motivation to add fMP4 to base MP4 in the first place. In any case I think MPEG also did a better job with DASH technically but borked it all up with patents. They were really stupid with that in the early 2010s.
apitman 2y ago
For MP4s the metadata is at the end annoyingly enough.
- AnotherGoodName 2y ago
  MP4 allows the header at the start or the end. It’s usually written to the end since it’s its not a fixed size and it’s a pain for recording and processing tools to rewrite the whole file on completion just to move the header to the start. You should always re-encode to move the header to the start for web though. It’s something you see too much of online once you know about it but mp4 can absolutely have the header at the start.
- koakuma-chan 2y ago
  You can `-movflags faststart` when encoding to place it at the beginning.
- tomsonj 2y ago
  implementations may request the metadata range at the end in this case, if the content length is known
  - lxgr 2y ago
    For "VOD", that works (and is how very simple <video> tag based players sometimes still do it), but for live streaming, it wouldn't – hence the need for fragmented MP4, MPEG-DASH, HLS etc. It does work for simpler codecs/containers though: Shoutcast/Icecast web radio streams are essentially just endless MP3 downloads, optionally with some non-MP3 metadata interspersed at known intervals.
lxgr 2y ago
The two are essentially the same thing, modulo trading off some unnecessary buffering on both sides of the TCP pipe in the "one big download" streaming model for more TCP connection establishments in the "range request to refill the buffer" one.
bob1029 2y ago
Yea this works for mp4 and HN seems confused about how. The MOOV atom is how range requests are enabled, but the browser has to find it first. That's why it looks like it's going to download the whole file at first. It doesn't know the offset. Once it reads it, the request will be cancelled and targeted range requests will begin.

wewewedxfgdf 2y ago

Yes it is true. Browsers talking to static web servers use HTTP byte ranges requests to get chunks of videos and can use the same mechanism to seek to any point in the file. Streaming that way is fast and simple. No fancy technology required. For MP4 to work that we you need to render it as fragmented MP4.

jofla_net 2y ago
Seconded, ive done a userland 'content-range' implementation myself. of course there were a few ffmpeg specific parameters the mp4 needed to work right still
EE84M3i 2y ago
Why would the browser send byte range requests for video tags if it expects to play the file back linearly from beginning to end anyway? Wouldn't that be additional overhead/round-trips?
- koakuma-chan 2y ago
  > Why would the browser send byte range requests for video tags if it expects to play the file back linearly from beginning to end anyway? Probably because byte range is required for seeking, and playing from the beginning is equivalent to seeking at 0. > Wouldn't that be additional overhead/round-trips? No because the range of the initial byte range request is the whole file (`bytes=0-`).
  - EE84M3i 2y ago
    My original comment was about the commenter I replied to saying: > To my knowledge, video stream requests chunks by range and is largely client controlled. It isn't a single, long lived http connection. Wouldn't a byte range request for the whole file fall under the "single, long lived http connection"? Sure it could be terminated early and another request made for seeking, but regardless the video can start before the whole file is downloaded, assuming it's encoded correctly?
    - tedunangst 2y ago
      The client doesn't want to eat the whole file, so it uses a range request for just the beginning of the file, and then the next part as needed.
      - koakuma-chan 2y ago
        The client would actually request the whole file and then terminate the request if the file is no longer needed. This is what browsers do at least.
        lxgr 2y ago
        Both are possible, and in fact I could imagine not all servers being too happy with having to trickle data over a persistent HTTP connection through the entire length of the video, with an almost always full TCP send buffer at the OS level.
        koakuma-chan 2y ago
        > Both are possible It is possible if you are in control of the client, but no browser would stream an mp4 file request by request. > with an almost always full TCP send buffer at the OS level This shouldn't be a problem because there is flow control. Also the data would probably be sent to the kernel in small chunks, not the whole file at once.
wordofx 2y ago
It’s not true because throwing a video file as a source on video tag has no information about the file being requested until the headers are pushed down. Hell back in 2005 Akamai didn’t even support byte range headers for partial content delivery, which made resuming videos impossible, I believe they pushed out the update across their network in 06 or 07.
- jonathanlydall 2y ago
  If your HTTP server provides and supports the appropriate headers and you’re serving supported file types, then it absolutely is true. Just putting a url in my Chromium based browser’s address bar to an mp4 file we have hosted on CloudFlare R2 “just works” (I expect a video tag would be the same), supporting skipping ahead in the video without having to download the whole thing. Initially skipping ahead didn’t work until I disabled caching on CloudFlare CDN as that breaks the “accept-range” capability on videos. For now we have negligible amount of viewership of these mp4s, but if it becomes an issue we’ll use CloudFlare’s video serving product.
  - wordofx 2y ago
    > If your HTTP server provides and supports the appropriate headers and you’re serving supported file types, then it absolutely is true. No. When you play a file in the browser with a video tag. It requests the file. It doesn’t ask for a range. It does use the range if you seek it, or you write the JavaScript to fetch based on a range. That’s why if you press play and pause it buffers the whole video. Only if you write the code yourself can you partially buffer a while like YouTube does.
    - bmacho 2y ago
      Nah, it uses complex video specific logic and http range requests as protocol. (At least the normal browsers and servers. You can roll your own dumb client/server of course.) > That’s why if you press play and pause it buffers the whole video. Browsers don't do that.
    - jonathanlydall 2y ago

jofzar 2y ago

The long answer is "it depends on how you do it" unsurprisingly video and voice/audio are probably the most different ways that you can "choose" to do distribution

bhhaskin 2y ago
This. You can't just throw it into a folder and have to stream. The web server has to support it and then there is encoding and formats.

AnotherGoodName 2y ago

They can playback as loading as long as they are encoded correctly fwiw (faststart encoded). When you create a video from a device the header is actually at the end of the file. Understandable, it’s where the file pointer was and mp4 allows this so your recording device writes it at the end. You must re-encoded with faststart (puts the moov atom at the start) to make it load reasonably on a webpage though.

timewizard 2y ago
> Understandable, it’s where the file pointer was and mp4 allows this so your recording device writes it at the end. Yet formats like WAVE which use a similar "chunked" encoding they just use a fixed length header and use a single seek() to get back to it when finalizing the file. Quicktime and WAVE were released around nearly the same time in the early 90s. MP2 was so much better I cringe every time I have to deal with MP4 in some context.
- lxgr 2y ago
  At the expense of quite some overhead though, right? MPEG-2 transport streams seem more optimized for a broadcast context, with their small frame structure and everything – as far as I know, framing overhead is at least 2%, and is arguably not needed when delivered over a reliable unicast pipe such as TCP. Still, being able to essentially chop a single, progressively written MPEG TS file into various chunks via HTTP range requests or very simple file copy operations without having to do more than count bytes, and with self-synchronization if things go wrong, is undoubtedly nicer to work with than MP4 objects. I suppose that's why HLS started out with transport streams and only gained fMP4 support later on.
  - timewizard 2y ago
    > and is arguably not needed when delivered over a reliable unicast pipe such as TCP. So much content ended up being delivered this way, but there was a brief moment where we thought multicast UDP would be much more prevalent than it ended up being. In that context it's perfect. > why HLS started out with transport streams and only gained fMP4 support later on. Which I actually think was the motivation to add fMP4 to base MP4 in the first place. In any case I think MPEG also did a better job with DASH technically but borked it all up with patents. They were really stupid with that in the early 2010s.

apitman 2y ago

For MP4s the metadata is at the end annoyingly enough.

AnotherGoodName 2y ago
MP4 allows the header at the start or the end. It’s usually written to the end since it’s its not a fixed size and it’s a pain for recording and processing tools to rewrite the whole file on completion just to move the header to the start. You should always re-encode to move the header to the start for web though. It’s something you see too much of online once you know about it but mp4 can absolutely have the header at the start.
koakuma-chan 2y ago
You can `-movflags faststart` when encoding to place it at the beginning.
tomsonj 2y ago
implementations may request the metadata range at the end in this case, if the content length is known
- lxgr 2y ago
  For "VOD", that works (and is how very simple <video> tag based players sometimes still do it), but for live streaming, it wouldn't – hence the need for fragmented MP4, MPEG-DASH, HLS etc. It does work for simpler codecs/containers though: Shoutcast/Icecast web radio streams are essentially just endless MP3 downloads, optionally with some non-MP3 metadata interspersed at known intervals.

lxgr 2y ago

The two are essentially the same thing, modulo trading off some unnecessary buffering on both sides of the TCP pipe in the "one big download" streaming model for more TCP connection establishments in the "range request to refill the buffer" one.

bob1029 2y ago

Yea this works for mp4 and HN seems confused about how. The MOOV atom is how range requests are enabled, but the browser has to find it first. That's why it looks like it's going to download the whole file at first. It doesn't know the offset. Once it reads it, the request will be cancelled and targeted range requests will begin.