3 ms·

I challenge you to record 30 seconds of a fixed test pattern from TV on two separate occasions and have it encode to identical bits. I don't think it can be don

I challenge you to record 30 seconds of a fixed test pattern from TV on two separate occasions and have it encode to identical bits. I don't think it can be done. Many media formats (including all digital TV formats) encode a rolling hardware timestamp of 33 bits (or more) which won't be the same for two separate segments by random chance. Audio, video, subtitles and other metadata will be ordered differently in the stream because they all comes from sources that have their own separate clocks. Synchronization between clocks at different stages in the media pipeline will cause different frames in the sequence to be dropped, padded, made into keyframes, etc, which then affects every bit in subsequent frames. TV stations use time-based watermarks. TV stations use digital compositing software that may retain bits from previous frames on an ongoing basis. Many studio media pipelines still involve analog steps. The list of complications goes on. Unless your pipeline is lossless, uses only digital sources and uses totally synchronized clocks for all stages, you're about as likely to get an MD5 hash collision by accident as you are to get any two non-trivial sequences of compressed video to be bit-identical.

a-dub 12y ago
Could make for an interesting codec idea though! A codebook based compressor that works on corpora of audio/video seems doable. More interesting would be a codec that aims not only to exploit models of human perception to throw away bits, but also exploits those models to produce "canonical" compressed forms that would be suitable for byte/block level deduplication. Of course, interesting things are hard, and errors would be hilarious. (See: Xerox and their copiers that change letters and numbers in documents...)
- Kayou 12y ago
  Also, I don't know if this is already implemented in some codec, but it would be interesting to re-use key frames from previous shots. For instance when you have a dialog and the camera switches from one person to another, a conventional codec probably creates a new i-frame for each shot whereas an intelligent codec would recognize that this is the continuation of the previous shot.
Kayou 12y ago
I agree for TV, but what about the case of the author, where he gets the episode from iTunes? Blu-Ray and iTunes episodes have a chance to have the intro timed perfectly the same as they are probably exported from a kind-of video editing workflow where they copy-paste the intro. At least, that's what I imagine. So the intro in their workflow is probably the same (I mean, the same video file), but somehow the export to h264 makes it different for each episode. Maybe the psy optimizations use dithering or another trick involving randomization.

a-dub 12y ago

Could make for an interesting codec idea though! A codebook based compressor that works on corpora of audio/video seems doable. More interesting would be a codec that aims not only to exploit models of human perception to throw away bits, but also exploits those models to produce "canonical" compressed forms that would be suitable for byte/block level deduplication. Of course, interesting things are hard, and errors would be hilarious. (See: Xerox and their copiers that change letters and numbers in documents...)

Kayou 12y ago
Also, I don't know if this is already implemented in some codec, but it would be interesting to re-use key frames from previous shots. For instance when you have a dialog and the camera switches from one person to another, a conventional codec probably creates a new i-frame for each shot whereas an intelligent codec would recognize that this is the continuation of the previous shot.

Kayou 12y ago

I agree for TV, but what about the case of the author, where he gets the episode from iTunes? Blu-Ray and iTunes episodes have a chance to have the intro timed perfectly the same as they are probably exported from a kind-of video editing workflow where they copy-paste the intro. At least, that's what I imagine. So the intro in their workflow is probably the same (I mean, the same video file), but somehow the export to h264 makes it different for each episode. Maybe the psy optimizations use dithering or another trick involving randomization.