7 ms·

OxiPNG: PNG optimizer written in Rust

130 points by KenanSulayman 9y ago

pcwalton 9y ago
Great example of the use of Rust to add parallelism to existing algorithms.
tyingq 9y ago
"complete rewrite of the OptiPNG project, which was assumed to be dead as no commit had been made to it since March 2014" Version 0.7.6 of OptiPNG was released 2016-apr-03. Not super recent, but it seems reasonable, given the limited scope, that it would stabilize and not change much after a certain point
- unkown-unknowns 9y ago
  First commit of OxiPNG was 2015-12-15. The way I read the part you quoted is that they originally assumed OptiPNG to be dead, not that they are saying now that it's dead. Absolutely agree with your point about reaching a stage where not so many big changes are necessary though.
  - tyingq 9y ago
    Ahh, I did miss that, but the OxiPng readme doesn't mention that OptiPNG has had a release since their initial observation either.
ricardobeat 9y ago
Did the increased parallelism lead to meaningful performance gains?
- cesarb 9y ago
  It appears so. I did a quick test with a huge PNG (80 megabytes), just to see what would happen. optipng took 5m9s, oxipng with "--threads 1" took 3m21s, and oxipng with the default number of threads took 2m00s. Not quite "half the time", but not bad.
  - Houshalter 9y ago
    I imagine most usecases where this performance would matter would be if you were compressing a ton of pngs at once. In which case wouldn't it be much faster just to run several different processes instead of one process using multiple cores?
    - solidsnack9000 9y ago
      But running several different processes can be a nuisance...better performance by default is still great.
      - Houshalter 9y ago
        Well it must have taken them a ton of work to parallelize their algorithm also. It would have been better to just have a batch mode that lets you process multiple images at once on different cores.
        tomjakubowski 9y ago
        Well, not really, because implementing a batch mode is more work than letting xargs or GNU Parallel or Make handle the multiprocessing for the "many files" case. Parallelizing the algorithm helps in the "single file" case, which is still important.
        Houshalter 9y ago
        Well parent comment was complaining that it would be more work for the end user. I was just noting it would be less work to include that convenience in the tool itself than trying to make the algorithm parallel for less benefit.
        solidsnack9000 9y ago
        Managing lots of processes is a ton of work...
0x4a42 9y ago
>Windows users will need to ensure they have the Visual C++ 2015 Runtime installed. Why?
- kbrosnan 9y ago
  > I'm not sure how easy this is to resolve in Appveyor, but I would also prefer to have Windows binaries compiled statically. I'm much less familiar with compiling Rust binaries on Windows and don't have a Windows machine readily available, so I'll have to fire up a virtual machine and experiment. > Worst case scenario, I'll need to add an instruction for Windows users to install the Visual C++ 2015 runtime. I'd prefer to avoid that though. This also likely won't be a problem with the GCC-compiled version, but this runtime is a dependency for all programs compiled by MSVC. https://github.com/shssoichiro/oxipng/issues/49 https://github.com/shssoichiro/oxipng/issues/49
  - userbinator 9y ago
    You should be able to link with MSVCRT.DLL, yielding a small binary that will "just work" on a variety of systems going all the way back to Win95(!) (provided you don't use newer API calls): https://stackoverflow.com/questions/10166412/how-to-link-against-msvcrt-dll-instead-of-msvcr100-dll-in-vc-10-0 https://stackoverflow.com/questions/10166412/how-to-link-aga... Microsoft doesn't seem to like this, and will say it's "officially not supported" (if I remember correctly, there's a very strongly worded post by Raymond Chen on the topic...), but this is how essentially all the apps that come with Windows are compiled, and I believe MingW does it too. http://planet.jboss.org/post/fighting_the_msvcrt_dll_hell http://planet.jboss.org/post/fighting_the_msvcrt_dll_hell
    - nightcracker 9y ago
      Microsoft's the one to blame here. If they made sure their systems automatically come with MSVCRTs pre-installed, and new ones downloaded through the update system, this problem wouldn't exist in the first place. Asking your users to install some sketchy 'runtime' won't work. And not every app should need or want a specific install script to verify for the umpteenth time that something as basic as the C/C++ runtime library is available.
ac29 9y ago
Does this include the work from zopflipng? That usually makes more of a difference in size than OptiPNG does.
- dikaiosune 9y ago
  There are CLI options for supporting zopfli, not sure whether it's from zopflipng.
  - hobofan 9y ago
    It is not. It uses the native Rust implementation of zopfli [0], which doesn't include anything from zopflipng. [0]: https://crates.io/crates/zopfli https://crates.io/crates/zopfli
dikaiosune 9y ago
For the curious, I replicated some benchmarks from ~1yr ago (https://www.reddit.com/r/rust/comments/48vkjy/oxipng_a_multithreaded_png_optimizer_written_in/d0n7h6b/ https://www.reddit.com/r/rust/comments/48vkjy/oxipng_a_multi...). Note in that thread that the author implies that the optimization levels between the two tools are not necessarily equivalent or stable. $ optipng --version OptiPNG version 0.7.6 $ oxipng --version oxipng 0.16.3 Ran on a 2016 MBP, i7. Time: opt-level oxipng optipng 2 148.63s 30.80s 3 9.56s 45.99s 4 23.82s 90.57s Size of resulting image: opt-level original 12M oxipng optipng 2 12M 12M 3 12M "already optimized" 4 12M "already optimized" TBH I don't really know how to interpret these results. But with the strange exception of `oxipng -o 2`, it does seem like the naive parallelism offers some real performance wins. Disclaimer: this is an incredibly amateur attempt to benchmark, I really don't have enough knowledge to produce something authoritative.
- userbinator 9y ago
  It would be more useful if you didn't round the sizes to "12M", because I have a hard time believing they were all exactly the same. Bytes are a discrete and exact quantity, and ideally would be displayed to full precision to allow useful comparison.
  - dikaiosune 9y ago
    It would probably be more useful, yes. But that would require remembering what my ls alias without `-h` is, and also would maybe give more confidence in the results of the quick 'n' dirty analysis than is justified.
    - 62747478182 9y ago
      You can type `/bin/ls -l` to skip your shell aliases.
- vladdanilov 9y ago
  Comparing supposedly the best results. oxipng optipng optimage size 12 231 612B 12 231 612B 11 027 901B time 23.82s 90.57s 50.8s I keep on improving my single-threaded image compressor (http://getoptimage.com http://getoptimage.com).

130 points by KenanSulayman 9y ago

Great example of the use of Rust to add parallelism to existing algorithms.

"complete rewrite of the OptiPNG project, which was assumed to be dead as no commit had been made to it since March 2014" Version 0.7.6 of OptiPNG was released 2016-apr-03. Not super recent, but it seems reasonable, given the limited scope, that it would stabilize and not change much after a certain point

unkown-unknowns 9y ago
First commit of OxiPNG was 2015-12-15. The way I read the part you quoted is that they originally assumed OptiPNG to be dead, not that they are saying now that it's dead. Absolutely agree with your point about reaching a stage where not so many big changes are necessary though.
- tyingq 9y ago
  Ahh, I did miss that, but the OxiPng readme doesn't mention that OptiPNG has had a release since their initial observation either.

ricardobeat 9y ago

Did the increased parallelism lead to meaningful performance gains?

cesarb 9y ago
It appears so. I did a quick test with a huge PNG (80 megabytes), just to see what would happen. optipng took 5m9s, oxipng with "--threads 1" took 3m21s, and oxipng with the default number of threads took 2m00s. Not quite "half the time", but not bad.
- Houshalter 9y ago
  I imagine most usecases where this performance would matter would be if you were compressing a ton of pngs at once. In which case wouldn't it be much faster just to run several different processes instead of one process using multiple cores?
  - solidsnack9000 9y ago
    But running several different processes can be a nuisance...better performance by default is still great.
    - Houshalter 9y ago
      Well it must have taken them a ton of work to parallelize their algorithm also. It would have been better to just have a batch mode that lets you process multiple images at once on different cores.
      - tomjakubowski 9y ago
        Well, not really, because implementing a batch mode is more work than letting xargs or GNU Parallel or Make handle the multiprocessing for the "many files" case. Parallelizing the algorithm helps in the "single file" case, which is still important.
        Houshalter 9y ago
        Well parent comment was complaining that it would be more work for the end user. I was just noting it would be less work to include that convenience in the tool itself than trying to make the algorithm parallel for less benefit.
      - solidsnack9000 9y ago
        Managing lots of processes is a ton of work...

0x4a42 9y ago

>Windows users will need to ensure they have the Visual C++ 2015 Runtime installed. Why?

kbrosnan 9y ago
> I'm not sure how easy this is to resolve in Appveyor, but I would also prefer to have Windows binaries compiled statically. I'm much less familiar with compiling Rust binaries on Windows and don't have a Windows machine readily available, so I'll have to fire up a virtual machine and experiment. > Worst case scenario, I'll need to add an instruction for Windows users to install the Visual C++ 2015 runtime. I'd prefer to avoid that though. This also likely won't be a problem with the GCC-compiled version, but this runtime is a dependency for all programs compiled by MSVC. https://github.com/shssoichiro/oxipng/issues/49 https://github.com/shssoichiro/oxipng/issues/49
- userbinator 9y ago
  You should be able to link with MSVCRT.DLL, yielding a small binary that will "just work" on a variety of systems going all the way back to Win95(!) (provided you don't use newer API calls): https://stackoverflow.com/questions/10166412/how-to-link-against-msvcrt-dll-instead-of-msvcr100-dll-in-vc-10-0 https://stackoverflow.com/questions/10166412/how-to-link-aga... Microsoft doesn't seem to like this, and will say it's "officially not supported" (if I remember correctly, there's a very strongly worded post by Raymond Chen on the topic...), but this is how essentially all the apps that come with Windows are compiled, and I believe MingW does it too. http://planet.jboss.org/post/fighting_the_msvcrt_dll_hell http://planet.jboss.org/post/fighting_the_msvcrt_dll_hell
  - nightcracker 9y ago
    Microsoft's the one to blame here. If they made sure their systems automatically come with MSVCRTs pre-installed, and new ones downloaded through the update system, this problem wouldn't exist in the first place. Asking your users to install some sketchy 'runtime' won't work. And not every app should need or want a specific install script to verify for the umpteenth time that something as basic as the C/C++ runtime library is available.

ac29 9y ago

Does this include the work from zopflipng? That usually makes more of a difference in size than OptiPNG does.

dikaiosune 9y ago
There are CLI options for supporting zopfli, not sure whether it's from zopflipng.
- hobofan 9y ago
  It is not. It uses the native Rust implementation of zopfli [0], which doesn't include anything from zopflipng. [0]: https://crates.io/crates/zopfli https://crates.io/crates/zopfli

dikaiosune 9y ago

For the curious, I replicated some benchmarks from ~1yr ago (https://www.reddit.com/r/rust/comments/48vkjy/oxipng_a_multithreaded_png_optimizer_written_in/d0n7h6b/ https://www.reddit.com/r/rust/comments/48vkjy/oxipng_a_multi...). Note in that thread that the author implies that the optimization levels between the two tools are not necessarily equivalent or stable. $ optipng --version OptiPNG version 0.7.6 $ oxipng --version oxipng 0.16.3 Ran on a 2016 MBP, i7. Time: opt-level oxipng optipng 2 148.63s 30.80s 3 9.56s 45.99s 4 23.82s 90.57s Size of resulting image: opt-level original 12M oxipng optipng 2 12M 12M 3 12M "already optimized" 4 12M "already optimized" TBH I don't really know how to interpret these results. But with the strange exception of `oxipng -o 2`, it does seem like the naive parallelism offers some real performance wins. Disclaimer: this is an incredibly amateur attempt to benchmark, I really don't have enough knowledge to produce something authoritative.

userbinator 9y ago
It would be more useful if you didn't round the sizes to "12M", because I have a hard time believing they were all exactly the same. Bytes are a discrete and exact quantity, and ideally would be displayed to full precision to allow useful comparison.
- dikaiosune 9y ago
  It would probably be more useful, yes. But that would require remembering what my ls alias without `-h` is, and also would maybe give more confidence in the results of the quick 'n' dirty analysis than is justified.
  - 62747478182 9y ago
    You can type `/bin/ls -l` to skip your shell aliases.
vladdanilov 9y ago
Comparing supposedly the best results. oxipng optipng optimage size 12 231 612B 12 231 612B 11 027 901B time 23.82s 90.57s 50.8s I keep on improving my single-threaded image compressor (http://getoptimage.com http://getoptimage.com).