4 ms·
Waifu2x is not actually the first or only image scaler to use neural networks - NNEDI3[1], an Avisynth[2] filter used for deinterlacing can also do really nice
by Daiz 11y ago
Waifu2x is not actually the first or only image scaler to use neural networks - NNEDI3[1], an Avisynth[2] filter used for deinterlacing can also do really nice image upscaling (and it's a lot faster than waifu2x). Here's an example of what it can do to the images in the blogpost:
Image 1: http://i.imgur.com/4cXr51v.png http://i.imgur.com/4cXr51v.png
Image 2: http://i.imgur.com/PZAXeM8.png http://i.imgur.com/PZAXeM8.png
It doesn't come with any noise reduction, but nothing stops you from doing that separately from the upscaling process itself, and that way you should be able to control it better anyway (I find the reduction options provided by waifu2x really aggressive even with the low setting, it just kills tons of detail).
As a sidenote, when talking about something like image scaling, it would be a good idea to avoid saying something like "image scaled 2x (normally)" as there are lots of ways to scale images and what's "normal" can vary a lot depending on what you're using.
[1] http://bengal.missouri.edu/~kes25c/ http://bengal.missouri.edu/~kes25c/
[2] http://avisynth.org http://avisynth.org
- jeresig 11y agoNNEDI3 is fantastic - thank you for providing a link and some samples! You're absolutely right that I shouldn't have said "normal". I update the post to clarify that this was using "OSX Preview". I did some hunting but didn't find any obvious pointers as to which algorithm they're using. If anyone knows offhand I'll be happy to include it!
- Daiz 11y agoWell, OSX Preview seems to be doing something interesting as I can't seem to find an exact match with some quick attempts, but whatever method they use it looks rather similar to Lanczos scaling.
- jonah 11y agoTalk with the imgix.com folks about the CoreImage stuff. They're using the built-in re-sampling in their product. Also chat with @deepbluecea who's done a lot of image processing stuff, including for Apple.
- TD-Linux 11y agoI looked at what imgix was using a few weeks ago on HN. The resampling they do is really poor. You can do much better with imagemagick. https://news.ycombinator.com/item?id=9501601 https://news.ycombinator.com/item?id=9501601
- jonah 11y agoYeah, why they're going through all that hardware effort, I dunno. Simpler developer workflow I guess. Would be interesting to do a cost/benefit vs. just using a Linux stack.
- nacs 11y agoThe 'hardware effort' is to get dramatically improved processing time by using the GPU since they're trying to do it on a much larger scale. I have/continue to use imagemagick and similar software-based solutions and they're pretty slow for multi-MB images (but most servers don't have good GPUs so it's the only solution unless you're building custom racks as imgix does).
- prewett 11y agoAt least use GraphicsMagick, it's a lot faster in my (limited) experience. Unfortunately, I couldn't turn up any decent benchmarks.
- nacs 11y agoThanks for the tip, I had assumed it would be similar in feature-set/speed to imagemagick so hadn't tried it yet.
- TD-Linux 11y agoYeah, I'm not super sure about the dramatically improved processing time. Especially compared to a SIMD-optimized scaler. You have to spend some time sending the image to the GPU and reading it back too. Especially if you set imagemagick to use the much worse scaler that imgix uses, I imagine it'd be pretty fast. On the other hand, if you replaced imgix's stack with the high quality scalers from mpv (written as OpenGL pixel shaders), and then compared to expensive CPU scalers, I would expect a GPU solution to be a win. Note that imgix also has to recompress the image as PNG or JPEG at the end. This has to be done on the CPU and is probably more resource intensive than any of the scaling.
- jeresig 11y agoAfter doing some more poking it appears as if Avisynth (and thus NNEDI3) is Windows-only. Do you happen to know if there are ways to run it in Linux or OSX? Or if there's a comparable set of software for those platforms?
- Daiz 11y agoAvisynth should run in Wine, but there is also Vapoursynth[1] (which works natively on OSX & Linux) and a NNEDI3 port[2] for it. After getting both of them up and running, a script like this[3] ran with the vspipe program that comes with Vapoursynth should do the trick. It's a bit cumbersome since Avisynth and Vapoursynth are primarily intended for processing video, not images, but it gets the job done in absence of a dedicated NNEDI3 resizing tool. I'm actually using this exact setup at work myself when I need to do any image upscaling. [1] http://www.vapoursynth.com/ http://www.vapoursynth.com/ [2] https://github.com/dubhater/vapoursynth-nnedi3 https://github.com/dubhater/vapoursynth-nnedi3 [3] http://pastebin.com/3k9TEL7Y http://pastebin.com/3k9TEL7Y