6 ms·

It’s pretty amazing people are still finding ways to make video smaller. Is this just people being clever or is it also more processing power being thrown at t

• dajonker 1y ago
  I believe patents play a big role here as well. Anything new must be careful to not (accidentally) violate any active patent, so there might be some tricks that can't currently be used for AV1/AV2
  • cogman10 1y ago
    I think patents are quickly becoming less of a problem. A lot of the foundational encoding techniques have exited patent protection. H.264 and everything before it is patent free now. It's true you could still accidentally violate a patent but that minefield is clearing out as those patents simply have to become more esoteric in nature.
    • PunchyHamster 1y ago
      ...till someone decides to patent one of the new techniques used
      • cogman10 1y ago
        You can't patent something that's in use. Prior art is a defense to a patent claim/lawsuit. But that's not my main point. My main point is that we are going down a fitting path with codecs which makes it hard to come up with general patents that someone might stumble over. That makes patents developed by the MPEG group far less likely to apply to AOM. A lot of those more generally applicable patents, like the DCT for example, have expired.
  • mikeyouse 1y ago
    There are numerous patent trolls in this space with active litigation against many of the participants in the consortium who brought AV1. The EU was also threatening to investigate (likely to protect the royalty revenues of European companies)
  • bee_rider 1y ago
    It has always seemed very weird to me that compression algorithms were patentable. 1) it harms interoperability 2) I thought math wasn’t patentable?
• olowe 1y ago
  It’s more money and more user’s compute being thrown at the problem to get the streaming service’s CDN bill down.
  • occz 1y ago
    While funny, that's not really what I would call accurate. Users get reduced data consumption, potentially higher quality selection if the bandwidth now allows for a higher resolution to be streamed, and possibly lower disk usage should they decide to offline the videos. Better codecs are an overall win for everyone involved.
    • calcifer 1y ago
      > Users get reduced data consumption, potentially higher quality selection if the bandwidth now allows for a higher resolution to be streamed They also get increased power usage, lesser battery life, higher energy bills, and potentially earlier device failures. > Better codecs are an overall win for everyone involved. Right.
      • whatevaa 1y ago
        If you experience early failure your device is badly engineered. Mobile/power constrained devices don't use software decoding, that just a path to miserable experience. Hardware decoding is basically required. Meanwhile my desktop can SW decode 4k youtube with 3% reported cpu usage.
      • badpun 1y ago
        Not to mention making your device obsolete. My 12 year old laptop already can't decode some of the videos on Pirate Bay in real time, because the codec is too demanding.
        • bee_rider 1y ago
          Of course, we’re living in the future where Moore’s law has seriously slowed down. But, as a product of the 90’s this is a kind wild thing to see. I can’t imagine in the year 2000 being disappointed content wouldn’t play on a 386 or something. But, I mean, your expectation is not that unreasonable, computers were quite good by 2013. It is just an eye-opening framing.
• zweifuss 1y ago
  A bit of both. Also, the modern Codecs have slightly different tradeoffs (image quality (PSNR, SSIM), computational complexity (CPU vs DSP vs Memory), storage requirements, bit rate) and therefore there isn't one that is best for every use case.
• amiga386 1y ago
  Yes, and it's allowing the format to change to allow more cleverness or apply more processing power. For example, changes from one frame to the next are encoded in rectangular areas called "superblocks" (similar to a https://en.wikipedia.org/wiki/Macroblock https://en.wikipedia.org/wiki/Macroblock). You can "move" the blocks (warp them), define their change in terms of other parts of the same frame (intra-frame prediction) or by referencing previous frames (inter-frame prediction), and so on... but you have to do it within a block, as that's the basic element of the encoding. The more tightly you can define blocks around the areas that are actually changing from frame to frame, the better. Also, it takes data to describe where these blocks are, so there are special limitations on how blocks are defined, to minimise how many bits are needed to describe them. AV2 now lets you define blocks differently, which makes it easier to fit them around the areas of the frame that are changing. It has also doubled the size of the largest block, so if you have some really big movement on screen, it takes fewer blocks to encode that. That's just one change, the headline improvement comes from all the different changes, but this is an important one. There is new cleverness in the encoders, but they need to be given the tools to express that cleverness -- new agreement about what types of transforms, predictions, etc. are allowed and can be encoded in the bitstream. https://youtu.be/Se8E_SUlU3w?t=242 https://youtu.be/Se8E_SUlU3w?t=242
  • DoctorOetker 1y ago
    In general with movement through scenes it would seem that rectangular update windows seem like a poor match. Is there a reason codec's don't use the previous frame(s) as stored textures, and remap them on the screen? I can move a camera through room and a lot of the texture is just reprojectivetransformed.
    • dddgghhbbfblk 1y ago
      >Is there a reason codec's don't use the previous frame(s) as stored textures, and remap them on the screen? I can move a camera through room and a lot of the texture is just reprojectivetransformed. I mean, that's more or less how it works already. But you still need a unit of granularity for the remapping. So the frame will store eg this block moves by this shift, this block by that shift etc.
• cornholio 1y ago
  I wonder when we will see generative AI codecs in production. The concept seems simple enough, the encoder knows the exact model the decoder will use to generate the final image starting from a handful of pixels, and optimizes towards lowest bitrate and minimum subjective quality loss, for example, by letting the decoder generate a random human face in the crowd, or give it more data in that area to steer it towards the face of the team maskot, as the case may be. At the absolute compression limit, it's no longer video, but a machine description of the scene conceptually equivalent to a textual script.
  • ozgrakkurt 1y ago
    There was nvidia videoo upsampling or w/e it is called. It was putting age spots on every face when it was blurry and it used too much resources as far as I can remember
  • justsid 1y ago
    And then that script gets processed on hundreds of GPUs in the cloud and the video gets streamed to the client. Wait.
• whycome 1y ago
  Soon we will just have local AI processors which will just make stuff up between scenes but adhere to a “close enough” guideline where all narratively critical elements are maintained but other things (eg landscapes or trees) will be generated locally. Movies will practically be long cutscenes with photorealistic graphics.
  • bee_rider 1y ago
    At that point we aren’t even all watching the same movies. Which could be interesting. But very different—I mean, even stuff like talking with your friends about a movie you saw will change drastically. Maybe a service could be centered around sharing your movie prompts so have a shared movie experience to talk to your friends about.
    • CooCooCaCha 1y ago
      Entertainment is becoming increasingly customizable and personalized. It’ll get to the point, like you said, that we’re not watching the same movie, playing the same game, etc. It feels like we’re losing something, a shared experience, in favor of an increasingly narcissistic attitude that everything needs to shapeable to individual preferences instead of accepting things as they are.
      • bee_rider 1y ago
        I dunno. Entertainment is sort of inherently selfish, right? It is an unproductive thing we engage in to make us happy. I’d be somewhat interested in something like a git that generates movies, that my friends can push to. Extremely widespread mass media fiction broadcast are sort of an aberration of the last 75 years or so. I mean, you’d have works in ancient times—the Odyssey—that are shared across a culture. But, that was still a story customize by each teller, and those sorts of stories were rare. Canon was mainly a concern of religions. It’s just for fun, we give it far too much weight nowadays.
        • CooCooCaCha 1y ago
• toast0 1y ago
  New video codecs typically offer more options for how to represent the current frame in terms of other frames. That typically means more processing for the encoder, because it can check for all the similarities to see what works best; there's also harder math for arithmetic coding of the picture data. It will be more work for the encoder if it needs to keep more reference images and especially if it needs to do harder transformations, or if arithemetic decoding gets harder. Clever matters a lot more for encoding. If you can determine good ways to figure out the motion information without trying them all, that gets you faster encoding speed. Decoding doesn't tend to have as much room for cleverness; the stream says to calculate the output from specific data, so you need to do that.
• tverbeure 1y ago
  I don’t know the details of AV2, but going from h.265 to h.266, the number of angles for angular prediction doubled, they added a tool to predict chroma from luma, added the ability to do pixel block copies and a bunch of other techniques… And that’s just for intra predictions. They also added tons of new inter prediction techniques. All of this requires a significant amount of extra logic gates/silicon area for hardware decoders, but the bit rate reduction is worth it. For CPU decoders, the additional computational load is not so bad. The real additional cost is for encoding because there’s more prediction tools to choose from for optimal compression. That’s why Google only does AV1 encoding for videos that are very popular: it doesn’t make sense to do it on videos that are seen by few.
  • Gigachad 1y ago
    Iirc Facebook did the selective encoding too. And it would predict which videos would be popular so even the first streams would get the AV1 version.