5 ms·

Implementing tile encoding in rav1e

60 points by pplonski86 8y ago

lousken 8y ago
I wonder when the encoder will be good and fast enough so that I can start reencoding my old h264 library, any estimate? I wanted to start next year but I am not sure considering the speeds, it still needs a lot of optimizations.
- firethief 8y ago
  Probably never, unless you're also downsampling because you only plan to watch videos on your phone / downgraded your TV forever / are losing your eyesight. Lossy transcoding barely makes sense when the source is a hilariously bad codec like MPEG2 (by modern standards), and even then you lose a lot of texture (the main benefit is inverse telecine and dealing with any other weirdness of the source disc ahead of time). You pay in quality for every time it's been lossily encoded, but only reap the space savings of the last (to a first approximation). You might save a little HDD space, but you'd have very poor quality for the bitrate.
  - lousken 8y ago
    I am aware that some detail will be lost. However with h265 encoding I've tested about a year ago I could shrink it by 40% without any noticible loss. If AV1 can get me above 50%, the >2TB saving is worth it I think. It'll be a lot easier to handle backups.
namibj 8y ago
Presumably, the mentioned lack of a huge speedup is due to both hyperthreading and CPU clocks depending on how many cores are active (so as to stay within a power and thermal envelope). Otherwise, profiling would obviously need to be done on the sequential parts to figure out if any of them are amenable to parallelisation, while also being worth the effort based on them taking up a significant amount of time.
- Thaxll 8y ago
  And also to be careful when doing benchmark on a laptop since the OS / governor is throttling CPU...
- rom1v 8y ago
  > the mentioned lack of a huge speedup is due to both hyperthreading Correct :) I just updated the article after similar comments on reddit: https://blog.rom1v.com/2019/04/implementing-tile-encoding-in-rav1e/#limits https://blog.rom1v.com/2019/04/implementing-tile-encoding-in...
drmpeg 8y ago
Multi processor encoding goes way back to the early days of video compression. The first real-time MPEG-2 encoder chips from C-Cube Microsystems were specifically designed to be used in parallel. Back then (1993), encoding an SD (720x480) image required twelve processors. Here's a picture of a development board. This is half of a two board sandwich that supported ten video processors. The CL4000 was the first member of the family and was used for the DirecTV rollout. It was not capable of MPEG-2, and for a few months, DirecTV was actually MPEG-1. http://www.w6rz.net/scorpio.jpg http://www.w6rz.net/scorpio.jpg
Thaxll 8y ago
Why do they need unsafe for a video encoder / decoder? There is no other way around in their example?
- gok 8y ago
  This is perhaps the main point of this article. They needed `unsafe` to implement a safe way of concurrently modifying a single contiguous buffer, which is not allowed in safe Rust.
  - mtgx 8y ago
    Could such a workaround come to the Rust language in the future. Maybe other programs will need a "safe" way to implement something similar that is currently impossible with the safe methods offered by Rust.
    - unrealhoang 8y ago
      But why? That’s is exactly the reason why unsafe exists: to open an escape hatch to do something that is impossible to do in safe Rust (and then wrap that under a safe interface). Your safe Rust, standard library, is constructed exactly the same way: based on unsafe. Unsafe doesn’t magically make your code unsafe, it’s just that you tell the compiler to trust you to handle the problem correctly (usually by meddling directly with pointers).
      - Thaxll 8y ago
        For me the question was more, isn't Rust supposed to have fearless concurrency meaning no "unsafe" part to modify data between threads.
        mmastrac 8y ago
        What would most likely here would be abstracting this problem into a library that could isolate "unsafe" in a place where it could be specified, tested and examined in isolation.
pornel 8y ago
This is a great example of use of `unsafe` in Rust. The memory access pattern they wanted is technically safe, but can't be expressed directly in the safe Rust. But that wasn't a blocker, and the Rust language didn't need to be made more complex to handle such case. Instead, the "missing feature" could be added to Rust with a bit of `unsafe`. The big thing here is that instead of implementing actual encoding using unsafe code, and spreading the risk of unsafety all over the complex parts of the codebase, all of the dangerous parts were contained in a minimal, easy-to-verify abstraction layer. And all other code on top of that remained being safe Rust with the same safety guarantees.
- nn3 8y ago
  That's the positive spin. The negative spin would be: how to spend a lot of effort to solve a problem you would never have if you didn't use rust. Yes it makes great blogs but it doesn't necessarily make his programs better. I'm two minds here. On the one hand the safety is nice and it might or might not find a few bugs in advance. But in a real project do you really have time to go through all this effort every time just to solve something do something legitimate that the rust designer's didn't quite consider? If he had written it in C he might have had enough time to work on the performance to actually get a performance gain. I still haven't made up my mind if the trade off is worth it.
  - GolDDranks 8y ago
    It does make his programs better, because what's the alternative? If he would have used C, it would be equal to "spreading the risk of unsafety all over the complex parts of the codebase". What the writer is doing here is strictly better than that.
  - unrealhoang 8y ago
    > But in a real project do you really have time to go through all this effort every time just to solve something do something legitimate that the rust designer's didn't quite consider? That should be rare because if that case of yours is common, someone has already "safetify" (put in a safe interface) in a crate that you can pull in and use. Secondly, your time spending on "safetify" your unsafe code is just a trade off for time debugging safety issues in the future when you use your unsafe code in the future. Heck, Rust was born also to let people declare the way their "unsafe" code can be used in other to parallelize, because keeping all the unsafe contracts in your head (or even document) is not scalable.
catpolice 8y ago
A few years ago I implemented a vaguely similar image compression algorithm just for kicks. I basically did a separate binary space partition for each color channel of the image, with the splitting points determined by minimizing an error function. It was extremely simple to implement and worked pretty well, except it represented big curves with little compression. The fun part was when I dropped the error minimization in the partition choice function and set it to choose them randomly: https://pbs.twimg.com/media/DMr5LaMUQAA2h2E?format=jpg&name=900x900 https://pbs.twimg.com/media/DMr5LaMUQAA2h2E?format=jpg&name=... https://pbs.twimg.com/media/DMr4Vc4U8AAOlyn?format=jpg&name=small https://pbs.twimg.com/media/DMr4Vc4U8AAOlyn?format=jpg&name=...

60 points by pplonski86 8y ago

I wonder when the encoder will be good and fast enough so that I can start reencoding my old h264 library, any estimate? I wanted to start next year but I am not sure considering the speeds, it still needs a lot of optimizations.

firethief 8y ago
Probably never, unless you're also downsampling because you only plan to watch videos on your phone / downgraded your TV forever / are losing your eyesight. Lossy transcoding barely makes sense when the source is a hilariously bad codec like MPEG2 (by modern standards), and even then you lose a lot of texture (the main benefit is inverse telecine and dealing with any other weirdness of the source disc ahead of time). You pay in quality for every time it's been lossily encoded, but only reap the space savings of the last (to a first approximation). You might save a little HDD space, but you'd have very poor quality for the bitrate.
- lousken 8y ago
  I am aware that some detail will be lost. However with h265 encoding I've tested about a year ago I could shrink it by 40% without any noticible loss. If AV1 can get me above 50%, the >2TB saving is worth it I think. It'll be a lot easier to handle backups.

namibj 8y ago

Presumably, the mentioned lack of a huge speedup is due to both hyperthreading and CPU clocks depending on how many cores are active (so as to stay within a power and thermal envelope). Otherwise, profiling would obviously need to be done on the sequential parts to figure out if any of them are amenable to parallelisation, while also being worth the effort based on them taking up a significant amount of time.

Thaxll 8y ago
And also to be careful when doing benchmark on a laptop since the OS / governor is throttling CPU...
rom1v 8y ago
> the mentioned lack of a huge speedup is due to both hyperthreading Correct :) I just updated the article after similar comments on reddit: https://blog.rom1v.com/2019/04/implementing-tile-encoding-in-rav1e/#limits https://blog.rom1v.com/2019/04/implementing-tile-encoding-in...

drmpeg 8y ago

Multi processor encoding goes way back to the early days of video compression. The first real-time MPEG-2 encoder chips from C-Cube Microsystems were specifically designed to be used in parallel. Back then (1993), encoding an SD (720x480) image required twelve processors. Here's a picture of a development board. This is half of a two board sandwich that supported ten video processors. The CL4000 was the first member of the family and was used for the DirecTV rollout. It was not capable of MPEG-2, and for a few months, DirecTV was actually MPEG-1. http://www.w6rz.net/scorpio.jpg http://www.w6rz.net/scorpio.jpg

Thaxll 8y ago

Why do they need unsafe for a video encoder / decoder? There is no other way around in their example?

gok 8y ago
This is perhaps the main point of this article. They needed `unsafe` to implement a safe way of concurrently modifying a single contiguous buffer, which is not allowed in safe Rust.
- mtgx 8y ago
  Could such a workaround come to the Rust language in the future. Maybe other programs will need a "safe" way to implement something similar that is currently impossible with the safe methods offered by Rust.
  - unrealhoang 8y ago
    But why? That’s is exactly the reason why unsafe exists: to open an escape hatch to do something that is impossible to do in safe Rust (and then wrap that under a safe interface). Your safe Rust, standard library, is constructed exactly the same way: based on unsafe. Unsafe doesn’t magically make your code unsafe, it’s just that you tell the compiler to trust you to handle the problem correctly (usually by meddling directly with pointers).
    - Thaxll 8y ago
      For me the question was more, isn't Rust supposed to have fearless concurrency meaning no "unsafe" part to modify data between threads.
      - mmastrac 8y ago
        What would most likely here would be abstracting this problem into a library that could isolate "unsafe" in a place where it could be specified, tested and examined in isolation.

pornel 8y ago

This is a great example of use of `unsafe` in Rust. The memory access pattern they wanted is technically safe, but can't be expressed directly in the safe Rust. But that wasn't a blocker, and the Rust language didn't need to be made more complex to handle such case. Instead, the "missing feature" could be added to Rust with a bit of `unsafe`. The big thing here is that instead of implementing actual encoding using unsafe code, and spreading the risk of unsafety all over the complex parts of the codebase, all of the dangerous parts were contained in a minimal, easy-to-verify abstraction layer. And all other code on top of that remained being safe Rust with the same safety guarantees.

nn3 8y ago
That's the positive spin. The negative spin would be: how to spend a lot of effort to solve a problem you would never have if you didn't use rust. Yes it makes great blogs but it doesn't necessarily make his programs better. I'm two minds here. On the one hand the safety is nice and it might or might not find a few bugs in advance. But in a real project do you really have time to go through all this effort every time just to solve something do something legitimate that the rust designer's didn't quite consider? If he had written it in C he might have had enough time to work on the performance to actually get a performance gain. I still haven't made up my mind if the trade off is worth it.
- GolDDranks 8y ago
  It does make his programs better, because what's the alternative? If he would have used C, it would be equal to "spreading the risk of unsafety all over the complex parts of the codebase". What the writer is doing here is strictly better than that.
- unrealhoang 8y ago
  > But in a real project do you really have time to go through all this effort every time just to solve something do something legitimate that the rust designer's didn't quite consider? That should be rare because if that case of yours is common, someone has already "safetify" (put in a safe interface) in a crate that you can pull in and use. Secondly, your time spending on "safetify" your unsafe code is just a trade off for time debugging safety issues in the future when you use your unsafe code in the future. Heck, Rust was born also to let people declare the way their "unsafe" code can be used in other to parallelize, because keeping all the unsafe contracts in your head (or even document) is not scalable.

catpolice 8y ago

A few years ago I implemented a vaguely similar image compression algorithm just for kicks. I basically did a separate binary space partition for each color channel of the image, with the splitting points determined by minimizing an error function. It was extremely simple to implement and worked pretty well, except it represented big curves with little compression. The fun part was when I dropped the error minimization in the partition choice function and set it to choose them randomly: https://pbs.twimg.com/media/DMr5LaMUQAA2h2E?format=jpg&name=900x900 https://pbs.twimg.com/media/DMr5LaMUQAA2h2E?format=jpg&name=... https://pbs.twimg.com/media/DMr4Vc4U8AAOlyn?format=jpg&name=small https://pbs.twimg.com/media/DMr4Vc4U8AAOlyn?format=jpg&name=...