3 ms·

I'm expecting next gen video codecs to all be neutral network based. Ie. every frame is generated from the previous internal state and a few more bytes of inpu

I'm expecting next gen video codecs to all be neutral network based. Ie. every frame is generated from the previous internal state and a few more bytes of input data which defines how the car moves or who the bad guy shoots. Current state of the art systems require huge GPU's and minutes per frame. But smaller models will do a decentish job faster, and better hardware and algorithms are coming out every few months.

ClumsyPilot 2y ago
Will ‘lossy’ compression aspect act with accuracy of human memory - the attacker becomes a man, because it’s usually a man, etc? Will the encoding of a movie give you different ethnicity for the bad guy?
- bawolff 2y ago
  We've basically already been there done that with jpeg2000 (edit: jbig2 not jpeg2000 as pointed out below)
  - ot 2y ago
    You might be thinking of JBIG2 https://en.wikipedia.org/wiki/JBIG2#Character_substitution_errors_in_scanned_documents https://en.wikipedia.org/wiki/JBIG2#Character_substitution_e...
- deleted 2y ago
  [deleted]
mihaic 2y ago
Sure, that's being predicted for about 10 years now. I think it's far more than one generation away simply because you'd need a decoder everywhere, and that takes a boatload of standardization. JpegXL failed to gain traction and got dumped for instance. Best we can get next gen would be an experimental fork of x265 or some av1 codec with neural net baseline to improve on the defaut, like mkv was at first.
deleted 2y ago
[deleted]
abound 2y ago
I worked at a startup (WaveOne) doing neural net-based compression. Your description of the state of the art seems a bit off, here's a 2021 paper [1] from WaveOne for comparison. Large GPUs, maybe, but only seconds (or less) for 720p+ frames. [1] https://openaccess.thecvf.com/content/ICCV2021/papers/Rippel_ELF-VC_Efficient_Learned_Flexible-Rate_Video_Coding_ICCV_2021_paper.pdf https://openaccess.thecvf.com/content/ICCV2021/papers/Rippel...

ClumsyPilot 2y ago

Will ‘lossy’ compression aspect act with accuracy of human memory - the attacker becomes a man, because it’s usually a man, etc? Will the encoding of a movie give you different ethnicity for the bad guy?

bawolff 2y ago
We've basically already been there done that with jpeg2000 (edit: jbig2 not jpeg2000 as pointed out below)
- ot 2y ago
  You might be thinking of JBIG2 https://en.wikipedia.org/wiki/JBIG2#Character_substitution_errors_in_scanned_documents https://en.wikipedia.org/wiki/JBIG2#Character_substitution_e...
deleted 2y ago
[deleted]

mihaic 2y ago

Sure, that's being predicted for about 10 years now. I think it's far more than one generation away simply because you'd need a decoder everywhere, and that takes a boatload of standardization. JpegXL failed to gain traction and got dumped for instance. Best we can get next gen would be an experimental fork of x265 or some av1 codec with neural net baseline to improve on the defaut, like mkv was at first.

deleted 2y ago

[deleted]

abound 2y ago

I worked at a startup (WaveOne) doing neural net-based compression. Your description of the state of the art seems a bit off, here's a 2021 paper [1] from WaveOne for comparison. Large GPUs, maybe, but only seconds (or less) for 720p+ frames. [1] https://openaccess.thecvf.com/content/ICCV2021/papers/Rippel_ELF-VC_Efficient_Learned_Flexible-Rate_Video_Coding_ICCV_2021_paper.pdf https://openaccess.thecvf.com/content/ICCV2021/papers/Rippel...