5 ms·

What? Faststart just means the metadata is at the front of the file, which these 500KB padded m4a files do have. They just have excessive paddings between the m

What? Faststart just means the metadata is at the front of the file, which these 500KB padded m4a files do have. They just have excessive paddings between the metadata and the content stream. The author used this flag in FFMPEG because.. well, despite the main goal is to shrink the padding, you still want to have metadata at the beginning.

CharlesW 5y ago
> Faststart just means the metadata is at the front of the file, which these 500KB padded m4a files do have. You're correct in the sense that having metadata at the start of the file means that these files are ready for progressive download/playback, albeit with a chunk of unnecessary data transfer. But the other important part of the "faststart" process is that you get what used to be called a "flattened" file, where the data is contiguous — metadata is followed immediately by compressed media data. (Tools for QuickTime Movies also compressed the 'moov' header, but I'm not sure if that's supported in MPEG-4.)
- thrdbndndn 5y ago
  The article already detailed why Apple (or most of AAC encoders) have this padding, and a reasonable amount of them are useful. They just need to be smaller. The author don't actually need it to be"contiguous"; it's just that FFMPEG stream copy is the easiest way to so. In this sense, Apple don't need to "flatten" their files, they just need to have smaller padding just as what iTunes is already doing when ripping user-provided CDs.
  - CharlesW 5y ago
    I think we're talking about two different things. You're talking about the space reserved to allow for in-place metadata updates on songs that you've ripped. As you've noted, it's not much and doesn't "need" to be smaller. I and the TFA are talking about music files purchased from and delivered by the Apple Music Store, which include 500KB of wasted space. The solution the article suggests (and that I added a bit more background to in my post) is to do what any production workflow should do before distributing MPEG-4/Movie files — fast-start those suckers. 0.5 MB is not much, but multiplied by the billions of tracks Apple sells and streams in a year, pretty soon we're talking about a significant amount of wasted storage and transfer. > In this sense, Apple don't need to "flatten" their files, they just need to have smaller padding just as what iTunes is already doing when ripping user-provided CDs. What's the benefit to shipping any empty KBs with every song purchased or streamed?
    - thrdbndndn 5y ago
      >What's the benefit to shipping any empty KBs with every song purchased or streamed? For (quicker) in-place metadata updates? You just said it. You can do so with the purchased music in iTunes. Not sure about the streaming services, but the article isn't about it at all (nor did it talk about if they actually have these 500 KB padding to begin with).
      - CharlesW 5y ago
        > For (quicker) in-place metadata updates? I just tested this, and it takes about ~140ms for ffmpeg to faststart/remux one song-length .m4a file (M1 Air, internal storage).
        giantrobot 5y ago
        Here's my slightly educated guess (I worked at Apple and for a time iTunes adjacent) the empty space is for file watermarking on edge servers. All iTunes (Apple Music) content is served via CDN. By leaving some empty space in the file the edge server can just write the owner's account ID fingerprint without having to remux the whole file. Apple can just send a virgin copy of a song to the edge servers and they can do the customization inline with serving it. That will be computationally cheaper than remuxing it and require no extra storage since the file is tweaked in flight. Leaving a lot of space provides enough room for some worst case size of watermark data without producing invalid files. But that's only my guess. I haven't looked at any files in question.
        iggldiggl 5y ago
        Good point about the watermarking, although 500 kB still seems rather generous for that: As far as I'm aware, the "watermark" only encompasses a few IDs plus name and e-mail of your Apple ID (compare https://github.com/avibrazil/un-istore https://github.com/avibrazil/un-istore).
        thrdbndndn 5y ago
        You may feel it's very small delay, but if you batch edit your songs often (which I do) it's actually pretty noticeable. Try to select 10 tracks and update the album name and feel the difference. Most of time wasted on remuxing the file comes from re-writing the whole thing on your hard disk, which is more noticable if you save your songs on HDD. Is it really that bad? No. But it's nice to have some padding. Same goes to MP3s and especially FLACs (which is even more noticeable because file size is much larger). Some of the digital distributors for hi-res (hi resolution) music do use "flatten" files, and it's quite annoying to edit them.
- Dylan16807 5y ago
  > But the other important part of the "faststart" process is that you get what used to be called a "flattened" file, where the data is contiguous — metadata is followed immediately by compressed media data. According to who? Do any media players care if there's a couple kilobytes empty there?

CharlesW 5y ago

> Faststart just means the metadata is at the front of the file, which these 500KB padded m4a files do have. You're correct in the sense that having metadata at the start of the file means that these files are ready for progressive download/playback, albeit with a chunk of unnecessary data transfer. But the other important part of the "faststart" process is that you get what used to be called a "flattened" file, where the data is contiguous — metadata is followed immediately by compressed media data. (Tools for QuickTime Movies also compressed the 'moov' header, but I'm not sure if that's supported in MPEG-4.)

thrdbndndn 5y ago
The article already detailed why Apple (or most of AAC encoders) have this padding, and a reasonable amount of them are useful. They just need to be smaller. The author don't actually need it to be"contiguous"; it's just that FFMPEG stream copy is the easiest way to so. In this sense, Apple don't need to "flatten" their files, they just need to have smaller padding just as what iTunes is already doing when ripping user-provided CDs.
- CharlesW 5y ago
  I think we're talking about two different things. You're talking about the space reserved to allow for in-place metadata updates on songs that you've ripped. As you've noted, it's not much and doesn't "need" to be smaller. I and the TFA are talking about music files purchased from and delivered by the Apple Music Store, which include 500KB of wasted space. The solution the article suggests (and that I added a bit more background to in my post) is to do what any production workflow should do before distributing MPEG-4/Movie files — fast-start those suckers. 0.5 MB is not much, but multiplied by the billions of tracks Apple sells and streams in a year, pretty soon we're talking about a significant amount of wasted storage and transfer. > In this sense, Apple don't need to "flatten" their files, they just need to have smaller padding just as what iTunes is already doing when ripping user-provided CDs. What's the benefit to shipping any empty KBs with every song purchased or streamed?
  - thrdbndndn 5y ago
    >What's the benefit to shipping any empty KBs with every song purchased or streamed? For (quicker) in-place metadata updates? You just said it. You can do so with the purchased music in iTunes. Not sure about the streaming services, but the article isn't about it at all (nor did it talk about if they actually have these 500 KB padding to begin with).
    - CharlesW 5y ago
      > For (quicker) in-place metadata updates? I just tested this, and it takes about ~140ms for ffmpeg to faststart/remux one song-length .m4a file (M1 Air, internal storage).
      - giantrobot 5y ago
        Here's my slightly educated guess (I worked at Apple and for a time iTunes adjacent) the empty space is for file watermarking on edge servers. All iTunes (Apple Music) content is served via CDN. By leaving some empty space in the file the edge server can just write the owner's account ID fingerprint without having to remux the whole file. Apple can just send a virgin copy of a song to the edge servers and they can do the customization inline with serving it. That will be computationally cheaper than remuxing it and require no extra storage since the file is tweaked in flight. Leaving a lot of space provides enough room for some worst case size of watermark data without producing invalid files. But that's only my guess. I haven't looked at any files in question.
        iggldiggl 5y ago
        Good point about the watermarking, although 500 kB still seems rather generous for that: As far as I'm aware, the "watermark" only encompasses a few IDs plus name and e-mail of your Apple ID (compare https://github.com/avibrazil/un-istore https://github.com/avibrazil/un-istore).
      - thrdbndndn 5y ago
        You may feel it's very small delay, but if you batch edit your songs often (which I do) it's actually pretty noticeable. Try to select 10 tracks and update the album name and feel the difference. Most of time wasted on remuxing the file comes from re-writing the whole thing on your hard disk, which is more noticable if you save your songs on HDD. Is it really that bad? No. But it's nice to have some padding. Same goes to MP3s and especially FLACs (which is even more noticeable because file size is much larger). Some of the digital distributors for hi-res (hi resolution) music do use "flatten" files, and it's quite annoying to edit them.
Dylan16807 5y ago
> But the other important part of the "faststart" process is that you get what used to be called a "flattened" file, where the data is contiguous — metadata is followed immediately by compressed media data. According to who? Do any media players care if there's a couple kilobytes empty there?