5 ms·

I've pondered recently if with random access being basically free these days if interleaved media formats are still necessary? Clearly for streaming purposes t

• wongarsu 3y ago
  Sequential access is still a lot cheaper on all levels of the stack. Whether that's playing the file from a network drive, reading the file from disk or even reading RAM or L3 cache. Pulling up random benchmarks, top-of-the-line M.2 SSD are 2-3x faster in sequential reads than in random reads. If you do random reads from a single thread the sequential reads are about 100x faster.
  • GauntletWizard 3y ago
    I wonder how much of that is over-optimization for sequential reads. When I was working in filesystems (back in the spinning rust days), I saw some code whose job it was to prefetch the next several hunks of data when you accessed data - It was a scaling algorithm, so if the block you were accessing was already in memory it would read several more hunks, and the next time you accessed from that same sequence it'd read 2x more than that, up to a fairly high limit - It made sense in this context to, shortly after a file was opened, just read the entire file into memory if it seemed likely to be used in short order, because disk queue latency was often the major limiting factor. What this meant was an at least 2x and often 100x read amplification for truly random reads. There were some heuristics to disable that, too, and then drive controllers started doing this directly so some code to managed that interaction, issuing more early in the "pretty sure we want to read more" path but dialing back the scaling. I would not be surprised if this type of logic is implemented into modern SSD controllers directly. When you issue a random read, it actually reads several sectors ahead, just in case those are the next requests. If we disabled that - How much faster would the random reads path be? How much slower would the sequential be? I suspect (very heavily, given the availability of enterprise drives with precisely this kind of tuning available) that there's a lot of tradeoff to be made.
    • zdw 3y ago
      Even "random" reads from "Random Access Memory" are slower than sequential reads - nearly everything in a computer is designed to exploit sequential locality "truly random" went the way of the dinosaur with the first CPU cache and memory bus wider than the size of read operations.
    • mangamadaiyan 3y ago
      The couple of filesystems whose guts I was familiar with used to perform sequential read detection to conditionally enable prefetching. Random reads did not suffer, and neither did sequential reads.
    • kevincox 3y ago
      In theory if everything was block sizes random could be the same as sequential. But as soon as you are pulling in extra data due to alignment you will have lost. Then you have to deal with prefetching being harder and other things that are arguably over-optimizations for sequential access.
    • numpad0 3y ago
      Truly random reads means constantly de-selecting and re-selecting which chips and banks and upper addresses to read, so it'll never be faster or easier than sequential reads, unless the underlying technology would have (hypothetical)limitations for consecutive read operations.
• dylan604 3y ago
  Even when interleaved, you still have things like Program Time Stamps that are used to sync A/V playback. With in the MP4, you can have A/V streams that are not aligned in the mux, and only playback in sync from PTS alignment. There are some apps that would not consider the PTS, so the audio and video would be out of sync. In the early days of DLSR mp4 recording, the audio could be +/- multiple frames out of sync with the video. I no longer deal with that kind of prosumer format (thank the gawds), so I don't know if it is still as much of an issue. Played back in a player that respects the PTS values, everything looks in sync as intended