3 ms·

Seems like a sensible project for them since shaving 4% of YouTube's traffic translates to millions of dollars in savings. But I'm more excited about the possib

Seems like a sensible project for them since shaving 4% of YouTube's traffic translates to millions of dollars in savings. But I'm more excited about the possibility of using deep learning image models to get incredibly higher compression rates. Some of the work I've seen on de-noising and super-resolution suggests that we are barely scratching the surface on what might be possible in terms of high-def video compression. Of course there is something of a time vs. space tradeoff, since these techniques would require way more compute for both encoding and decoding. But compute is pretty cheap and underutilized on the client side now, and Google probably has a huge amount of excess compute power to handle usage spikes that could be used for background processing.

smaddox 5y ago
Deep-learning-based image compression is particularly interesting because it turns the problem into a massively parallel one. As GPUs and TPUs become more powerful and more common in commodity hardware, there will be a drive to move many existing algorithms/applications to that hardware.
zitterbewegung 5y ago
In procedurally generated Anime there is Waifu 2x https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x . After procedural generation it is recommended to do denoiseing on the resulting image to improve quality.
alar44 5y ago
It's really a new paradigm in compression, imo. For example with a video conference, rather than trying to compress the pixel data and put it back together accurately, you can just send pose and gaze data and recreate a "fake" video on the other side for a rediculous compression ratio.
AlexanderTheGr8 5y ago
I am also very interested in DL-based lossy image compression!! I am starting research in it. I believe that there's a lot of scope for DL to improve lossy image compression, since lossy relies on what humans see, and DL is really good at figuring out what humans see. Are you also interested in it? I have found some other people interested in it. Maybe we can start a Discord channel for this?
mchusma 5y ago
It's definitely cool, but the 4% win here is nothing compared to continuing AV1 rollout. I also agree that YouTube and Google in particular seem well suited to incorporate additional AI models to massively improve compression, particularly in their mobile app. A 300MB, finely tuned model (either as an AV1 upscaler or a pure deep learning model) is highly likely to be able to allow mobile devices to stream at 4k for the same bitrate as v9 480p.

smaddox 5y ago

Deep-learning-based image compression is particularly interesting because it turns the problem into a massively parallel one. As GPUs and TPUs become more powerful and more common in commodity hardware, there will be a drive to move many existing algorithms/applications to that hardware.

zitterbewegung 5y ago

In procedurally generated Anime there is Waifu 2x https://github.com/nagadomi/waifu2x https://github.com/nagadomi/waifu2x . After procedural generation it is recommended to do denoiseing on the resulting image to improve quality.

alar44 5y ago

It's really a new paradigm in compression, imo. For example with a video conference, rather than trying to compress the pixel data and put it back together accurately, you can just send pose and gaze data and recreate a "fake" video on the other side for a rediculous compression ratio.

AlexanderTheGr8 5y ago

I am also very interested in DL-based lossy image compression!! I am starting research in it. I believe that there's a lot of scope for DL to improve lossy image compression, since lossy relies on what humans see, and DL is really good at figuring out what humans see. Are you also interested in it? I have found some other people interested in it. Maybe we can start a Discord channel for this?

mchusma 5y ago

It's definitely cool, but the 4% win here is nothing compared to continuing AV1 rollout. I also agree that YouTube and Google in particular seem well suited to incorporate additional AI models to massively improve compression, particularly in their mobile app. A 300MB, finely tuned model (either as an AV1 upscaler or a pure deep learning model) is highly likely to be able to allow mobile devices to stream at 4k for the same bitrate as v9 480p.