3 ms·

I wonder if this is an area actively being researched, using models like these for video compression?

• hobofan 4y ago
  Yes, with NVIDIA Maxine probably one of the most prominent examples of it. I haven't dug into the SDK to see if they actually delivered it, but they announced that with NVIDIA Maxine they can do live videoconferencing with 1/10th the bandwidth.
• mcbuilder 4y ago
  I believe they are already state of the art for image compression. That being said these shitty video models I believe are just an arms race between Meta and Google after the release of stable diffusion. Microsoft has a video version of CLIP that I believe will really change the game, but unless you have trained a model with video embeddings it's all going to look devoid of any narrative. Right now the models just look like a sequence of images with the same promt and some sort of continuity to make it look more video like.
• kromem 4y ago
  There was a very cool project recently using StableDiffusion to compress images better than JPEG. Also, there's some interesting work with ML taking diffused light from around a corner and recovering the original pre-diffused silhouette. In many ways, this is how we've learned the visual cortex is working. The amount of actual neutral data you are seeing is way less than you'd think given your perceived visual fidelity. The only practical issue is that distribution of AI hardware in consumer devices is going to noticeably lag behind POC on compounding cutting edge hardware in research environments, and no one wants to invest into obsolescence. Maybe it will happen in the cellphone market though given the hardware refresh rates from carrier subsidies.