8 ms·
Film grain synthesis in AV1 (2019)
- rootw0rm 4y agoFilm grain synthesis? Oh man, I don't know how I feel about this one. Without viewing any comparisons, my first reaction is...why not just higher bitrate instead?
- Ameo 4y agoFilm grain is similar to noise which by definition compresses extremely poorly. By actually modeling the way the film grain is produced, they can avoid sending what is essentially random data over the wire while still preserving the visual effect of the film grain itself. That way, the bitrate can be spent to encode the actual video data at a higher quality.
- simbas 4y agoyou did a good job
- pornel 4y agoThis can be said about every lossy compression technique. Why do quantization that throws away details, and not send higher bitrate instead? Why do edge prediction that can smudge things, and not send higher bitrate instead? Why do inter-frame prediction which can cause wobbly motion and not send higher bitrate instead? The answer always is that the technique allows better use of bandwidth, so you can have a better image without increasing bandwidth. Or if you're able to increase the bandwidth, you can have even better picture with the technique than without it (until the bandwidth is so high that you can send the video uncompressed, but that's not happening anytime soon for video on the web).
- babypuncher 4y agoThink of how much money Netflix saves by streaming movies to your TV at 5mbps instead of 10mbps. Serving a single user, the cost difference is negligible, but across 120 million users it probably saves them millions in bandwidth costs. I still buy blu-rays though so I am a firm believer in the "just throw more bits at it" solution.
- ubercow13 4y agoThat's a false dichtomy. At any bitrate Netflix chooses to stream at, this technique could improve perceived quality. Meanwhile blurays also use codecs with similar advanced reconstructive techniques - even with the higher bitrate they are essential to maintain a high perceived quality.
- klodolph 4y agoI like this quote from the article, > The correct answer to this question is that often the choice is not between the original film grain and the synthesized one. When video is transmitted over a channel with limited bandwidth, the choice is often between not having the film grain at all (or having the grain significantly distorted by compression) and having synthesized grain that looks subjectively similar to the original one. Film grain doesn’t compress well. If you buy a 1080p Blu-ray disk, you can get a bit rate of something like 40 Mbit/s. Go watch 1080p video on Netflix, and you’re going to get something closer to 5 Mbit/s. Yes, the codecs are different—yes, you can talk all you want about how everything is going to be fast when we all get 5G—but this is still the ground reality that most people are working with. The bit rate is more of a constraint you have to work within than a variable you can just tweak to get the results you want. By comparison, ProRes 422 has a target bit rate of 147 Mbit/s for 1920x1080@29.97.
- babypuncher 4y agoBlu-Ray came out 16 years ago and streaming services still haven't really caught up in image quality. Physical media still has some life left in it.
- Wowfunhappy 4y agoWell, the 4K ones didn't come out 16 years ago. Would you take a 1080p BluRay over 4K Netflix stream? I'm legitimately curious. I feel like we sometimes prioritize higher resolutions over overall detail.
- klodolph 4y agoI generally take a Blu-Ray 1080p over any streaming option. Note that plenty of movies, in the theater, were never done in 4K to begin with. 4K is great for selling TVs, but cinematographers and directors seem to be more lukewarm on using 4K. I think most movies are still done at 2K. Your UHD Blu-ray Disc might just be an upscale.
- 4y ago
- NonNefarious 4y agoI had no idea AV1 or any other codec provided for this. Pretty interesting! What would undermine it is imperfect removal of real noise or grain from the source material: Then you'd have some remnants of the original grain, plus the synthesized grain... an incorrect result. It seems that this scheme requires perfectly clean input images. I hope this idea is taken further, and it becomes the norm among codecs or wrappers to provide room for an author-defined post-processing shader to be applied after decompression.
- adgjlsfhk1 4y agoIt really doesn't. The whole reason this technique works is that our brain generally processes effects like noise and grain in terms of macro properties, and even if none of the pixels are anywhere like the right value, it will still look "right" as long as the noise distribution is closish.
- NonNefarious 4y agoNo it really doesn't what? I don't know what that comment refers to.
- NonNefarious 4y ago
- avhon1 4y agoIt doesn't require perfect removal of the original film grain. A mishmash composed of remnants of the original grain combined with new synthetic grain can still look correct to our brains.
- NonNefarious 4y agoAccording to this paper, they're going a lot of effort to reproduce the original grain very accurately. If they combine that with some half-assed residual grain from the original, why go to all the effort to faithfully reproduce it? Based on your assertion, it wasn't really necessary to begin with.
- Retr0id 4y agoI find it slightly funny that we go to such great lengths to encode noise, only for smart TVs to filter it out again with their noise reduction algorithms (at least you can turn it off, on most models).
- deleted 4y ago[deleted]
- Daub 4y agoFilm grain is often added to computor generated output that is to be composited onto live action footage. If it were not, it would look fake. We also add motion blur and depth refocus. Agreed though, it does feel weird as a vfx artist to be bringing down the quality of cg to that of film stock.
- danbmil99 4y agoIronic. AV1 is the descendant of On2's TrueMotion codecs -- better known as vp3-vp9-av1. The very first codecs from On2 (then called DuckCorp) -- TrueMotion and TrueMotion2 (used in games such as FF7) -- used a compression technique that did a good job of filtering, then on playback simulating film grain. We used this effect to good advantage, demoing exclusively 24 fps film-sourced material and properly dealing with frame rate using 3/2 pulldown. Everything old is new again...
- p0nce 4y agoI wonder how you can develop features like that, without looking at PSNR. Is it instead all subjective testing?
- pornel 4y agoSubjective testing is required to verify that the "tools" the codec has make sense. For day-to-day development and tweaking of the implementation developers use SSIM, which works better than PSNR for noisy images.
- m000 4y agoThis seems fantastic news for home archivists! But on the practical side, what is the adoption rate of this technology? Are there any user-accessible tools that use it? (Handbrake and VLC come to mind for encoding/decoding.)
- avhon1 4y agoStaxRip has a selection of AV1 encoders available, and Handbrake Nightly recently added one (I think svt-av1?).