6 ms·

The magic (image resampling) kernel

bhouston 2y ago
Super cool. How did I not know about this before?
- BoingBoomTschak 2y ago
  I was also pretty surprised, as I consider myself decently knowledgeable in the field. Learned of it via https://github.com/libvips/libvips/issues/4089 https://github.com/libvips/libvips/issues/4089.
BoingBoomTschak 2y ago
These previous discussions (including the author in the second one) were pretty fruitful: https://news.ycombinator.com/item?id=10404517 https://news.ycombinator.com/item?id=10404517 (2015) https://news.ycombinator.com/item?id=26513518 https://news.ycombinator.com/item?id=26513518 (2021)
pseudosavant 2y ago
I was surprised I hadn't heard of this, or his related project JPEG-Clear. I have thought for years that the JPEG-Clear method is how responsive images should have been handled in a browser. A single-file format that can be progressively downloaded only up to the resolution it is being displayed at. If you zoom in, the rest of the data can be downloaded for more detail. Doesn't require complex multi-file image authoring steps, keeps simple <img src> grammar, and is more efficient than downloading multiple completely separate images.
- meindnoch 2y ago
  JPEG-Clear? The guy "reinvented" progressive JPEGs?
  - Dwedit 2y ago
    Loading a more detailed version of an image as you zoom in is different from what a progressive JPEG does. Loading a Progressive JPEG means you still unconditionally load the entire file, you just are able to show a low detail version before it is fully loaded. The last time I saw a progressive JPEG actually take time to load was when I had dialup.
    - meindnoch 2y ago
      1. You can terminate the loading process as soon as you're satisfied with the quality. It's just that browser don't do that. 2. The OPs JPEG-Clear proposal [1] also loads the entire file no matter what. It's literally just a reinvention of progressive JPEGs, presenting it as something novel. [1] https://johncostella.com/jpegclear/ https://johncostella.com/jpegclear/
svantana 2y ago
I feel like this article is really overselling this filter. A 4-point symmetric interpolation kernel can be parameterized as [k, 1-k, 1-k, k]/2, i.e. it has a single degree of freedom. k=-1/4 is bicubic, k=1/4 is this 'magic', and k=0 is bilinear. Bicubic is sharper, and 'magic' has better alias rejection. Which looks better depends on the image and the viewer's subjective preference. For insta photos, it's probably better to go for 'magic', while for text, one might prefer bicubic. Neither is "simpler" as this article keeps suggesting, they just have different filter coefficients, that's all. But any other value of k is an equally valid choice.
- BoingBoomTschak 2y ago
  It certainly is. Especially lacking in proper comparisons of the final filter with the competition. I myself default to RobidouxSharp for downscaling and something like https://www.imagemagick.org/discourse-server/viewtopic.php?t=21695 https://www.imagemagick.org/discourse-server/viewtopic.php?t... for upscaling.
  - BoingBoomTschak 2y ago
    Took the time to make such a comparison using the article's sample images (even if the filter isn't sharpened in those), same thrice doubling of the small picture: http://0x0.st/XEEZ.png http://0x0.st/XEEZ.png I find such a test strange and irrelevant, though.
    - dahart 2y ago
      Ooh the animated comparison is really helpful. I couldn’t see it in the article, but with your version the Magic version feels flatter, almost like it’s a bokeh blur and not just a low pass. The Sigmoid seems far better than either Magic or Bicubic.
    - svantana 2y ago
      Nice, I agree the lanczos is the best out of the pack.
CyberDildonics 2y ago
I guess anything is magic if you don't know how it works or if you need some clicks to promote your personal site. This is basically a slightly different gaussian kernel and the "incredible results" of a small image becoming a larger resolution blurry image is completely normal. Also you don't want negative lobes in image kernels no matter how theoretically ideal it is, because it will give you ringing artifacts. If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.
- pixelpoet 2y ago
  > If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel. Strongly disagree, and my commercial software is known for its high image quality and antialiasing. Gaussian is way too blurry unless you're rendering for film.
  - CyberDildonics 2y ago
    Then your filter width is too wide. Try 2.2 - 2.5
  - dahart 2y ago
    In my film experience, I think most film people don’t like Gaussian either; too blurry for them as well. At least, I’ve sat in on filter evaluations with a couple of directors & VFX sups many years ago, and they said Gaussian was too soft and preferred a sharper Mitchell. But I am curious, perhaps similar to the sibling comment - how do you determine the optimal Gaussian width? You can certainly go narrower/sharper and get less blur at the cost of more artifacts similar to a sharper filter, right? BTW have we discussed this recently? ;) I love Gaussian’s ability to hide any hint of the pixel grid, which I find very few filters can do. I also tend to believe that, perceptually speaking, over-blurring slightly doesn’t hurt while under-blurring does, especially for moving things, but that might be more personal bias than objective reality. I would be interested to look at any comparisons or results from your software or in general, if you have some.
DustinBrett 2y ago
Would be cooler if images on FB didn't suck.
rnhmjoj 2y ago
> Fourthly, and most importantly, as noted above: m(x) is a partition of unity: it “fits into itself”; [...] if we place a copy of m(x) at integral positions, and sum up the results, we get a constant (unity) across all x. [...] This remarkable property can help prevent “beat” artifacts across a resized image. So, basically the reason why this works better than other visually similar filters is that it happens to satify the Nyquist ISI criterion[1]. [1]: https://en.wikipedia.org/wiki/Nyquist_ISI_criterion https://en.wikipedia.org/wiki/Nyquist_ISI_criterion
layer8 2y ago
In the “Bicubic: note the artifacts” comparison images, the bicubic version, regardless of the aliasing, is less blurry and has more detail than the “magic kernel” version. I therefore don’t agree that the latter is “visually, far superior”. There is at least some trade-off.
herf 2y ago
This uniform b-spline is the same one used often as a "Gaussian" approximation (three box filters) - see Paul Heckbert's 1986 paper here (apparently done at NYIT in the early 1980s with help from Ken Perlin): https://dl.acm.org/doi/pdf/10.1145/15886.15921 https://dl.acm.org/doi/pdf/10.1145/15886.15921
raphlinus 2y ago
The page mostly talks about image resampling where the goal is more or less preserving all frequencies, but it's also extremely effective at implementing Gaussian blur. Basically, you do n iterations of sampling by 1/2 using this kernel, followed by a very small FIR filter, then n iterations of upsampling 2x using the same kernel. Here, n is essentially log2 of the blur radius, and the total amount of computation is essentially invariant to that radius. All these computations are efficient on GPU - in particular, the upsampling can be done using vanilla bilinear texture sampling (which is very cheap), just being slightly clever about the fractional coordinates. It works well because, as stated, the kernel does a good job rejecting frequencies prone to aliasing. So, in particular, you don't get any real quality loss from doing 2x scale changes as opposed to bigger steps (and thus considerably larger FIR support). I have some Python notebooks with some of these results, haven't gotten around to publishing them yet.
- kragen 2y ago
  I look forward to being able to read your notebooks!
- leguminous 2y ago
  I have done something like this with a Lanczos kernel (a=1) downsizing repeatedly by 2x, a small Gaussian kernel, and then repeatedly upsizing by 2x with simple hardware bilinear sampling. The (2D) Lanczos downsizing can be done with only four samples using the bilinear sampling tricks that you mention, and I avoided expensive trigonometric functions, divisions, and the singularity at 0 by using an even 8th order polynomial approximation. I would be curious to see the results using this kernel, but the Lanczos is so far the best that I've tried.

Super cool. How did I not know about this before?

BoingBoomTschak 2y ago
I was also pretty surprised, as I consider myself decently knowledgeable in the field. Learned of it via https://github.com/libvips/libvips/issues/4089 https://github.com/libvips/libvips/issues/4089.

These previous discussions (including the author in the second one) were pretty fruitful: https://news.ycombinator.com/item?id=10404517 https://news.ycombinator.com/item?id=10404517 (2015) https://news.ycombinator.com/item?id=26513518 https://news.ycombinator.com/item?id=26513518 (2021)

pseudosavant 2y ago

I was surprised I hadn't heard of this, or his related project JPEG-Clear. I have thought for years that the JPEG-Clear method is how responsive images should have been handled in a browser. A single-file format that can be progressively downloaded only up to the resolution it is being displayed at. If you zoom in, the rest of the data can be downloaded for more detail. Doesn't require complex multi-file image authoring steps, keeps simple <img src> grammar, and is more efficient than downloading multiple completely separate images.

meindnoch 2y ago
JPEG-Clear? The guy "reinvented" progressive JPEGs?
- Dwedit 2y ago
  Loading a more detailed version of an image as you zoom in is different from what a progressive JPEG does. Loading a Progressive JPEG means you still unconditionally load the entire file, you just are able to show a low detail version before it is fully loaded. The last time I saw a progressive JPEG actually take time to load was when I had dialup.
  - meindnoch 2y ago
    1. You can terminate the loading process as soon as you're satisfied with the quality. It's just that browser don't do that. 2. The OPs JPEG-Clear proposal [1] also loads the entire file no matter what. It's literally just a reinvention of progressive JPEGs, presenting it as something novel. [1] https://johncostella.com/jpegclear/ https://johncostella.com/jpegclear/

svantana 2y ago

I feel like this article is really overselling this filter. A 4-point symmetric interpolation kernel can be parameterized as [k, 1-k, 1-k, k]/2, i.e. it has a single degree of freedom. k=-1/4 is bicubic, k=1/4 is this 'magic', and k=0 is bilinear. Bicubic is sharper, and 'magic' has better alias rejection. Which looks better depends on the image and the viewer's subjective preference. For insta photos, it's probably better to go for 'magic', while for text, one might prefer bicubic. Neither is "simpler" as this article keeps suggesting, they just have different filter coefficients, that's all. But any other value of k is an equally valid choice.

BoingBoomTschak 2y ago
It certainly is. Especially lacking in proper comparisons of the final filter with the competition. I myself default to RobidouxSharp for downscaling and something like https://www.imagemagick.org/discourse-server/viewtopic.php?t=21695 https://www.imagemagick.org/discourse-server/viewtopic.php?t... for upscaling.
- BoingBoomTschak 2y ago
  Took the time to make such a comparison using the article's sample images (even if the filter isn't sharpened in those), same thrice doubling of the small picture: http://0x0.st/XEEZ.png http://0x0.st/XEEZ.png I find such a test strange and irrelevant, though.
  - dahart 2y ago
    Ooh the animated comparison is really helpful. I couldn’t see it in the article, but with your version the Magic version feels flatter, almost like it’s a bokeh blur and not just a low pass. The Sigmoid seems far better than either Magic or Bicubic.
  - svantana 2y ago
    Nice, I agree the lanczos is the best out of the pack.

CyberDildonics 2y ago

I guess anything is magic if you don't know how it works or if you need some clicks to promote your personal site. This is basically a slightly different gaussian kernel and the "incredible results" of a small image becoming a larger resolution blurry image is completely normal. Also you don't want negative lobes in image kernels no matter how theoretically ideal it is, because it will give you ringing artifacts. If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel.

pixelpoet 2y ago
> If you work with image kernels / reconstruction filters long enough you will eventually learn that 90% of the time you want a gauss kernel. Strongly disagree, and my commercial software is known for its high image quality and antialiasing. Gaussian is way too blurry unless you're rendering for film.
- CyberDildonics 2y ago
  Then your filter width is too wide. Try 2.2 - 2.5
- dahart 2y ago
  In my film experience, I think most film people don’t like Gaussian either; too blurry for them as well. At least, I’ve sat in on filter evaluations with a couple of directors & VFX sups many years ago, and they said Gaussian was too soft and preferred a sharper Mitchell. But I am curious, perhaps similar to the sibling comment - how do you determine the optimal Gaussian width? You can certainly go narrower/sharper and get less blur at the cost of more artifacts similar to a sharper filter, right? BTW have we discussed this recently? ;) I love Gaussian’s ability to hide any hint of the pixel grid, which I find very few filters can do. I also tend to believe that, perceptually speaking, over-blurring slightly doesn’t hurt while under-blurring does, especially for moving things, but that might be more personal bias than objective reality. I would be interested to look at any comparisons or results from your software or in general, if you have some.

DustinBrett 2y ago

Would be cooler if images on FB didn't suck.

rnhmjoj 2y ago

> Fourthly, and most importantly, as noted above: m(x) is a partition of unity: it “fits into itself”; [...] if we place a copy of m(x) at integral positions, and sum up the results, we get a constant (unity) across all x. [...] This remarkable property can help prevent “beat” artifacts across a resized image. So, basically the reason why this works better than other visually similar filters is that it happens to satify the Nyquist ISI criterion[1]. [1]: https://en.wikipedia.org/wiki/Nyquist_ISI_criterion https://en.wikipedia.org/wiki/Nyquist_ISI_criterion

layer8 2y ago

In the “Bicubic: note the artifacts” comparison images, the bicubic version, regardless of the aliasing, is less blurry and has more detail than the “magic kernel” version. I therefore don’t agree that the latter is “visually, far superior”. There is at least some trade-off.

herf 2y ago

This uniform b-spline is the same one used often as a "Gaussian" approximation (three box filters) - see Paul Heckbert's 1986 paper here (apparently done at NYIT in the early 1980s with help from Ken Perlin): https://dl.acm.org/doi/pdf/10.1145/15886.15921 https://dl.acm.org/doi/pdf/10.1145/15886.15921

raphlinus 2y ago

The page mostly talks about image resampling where the goal is more or less preserving all frequencies, but it's also extremely effective at implementing Gaussian blur. Basically, you do n iterations of sampling by 1/2 using this kernel, followed by a very small FIR filter, then n iterations of upsampling 2x using the same kernel. Here, n is essentially log2 of the blur radius, and the total amount of computation is essentially invariant to that radius. All these computations are efficient on GPU - in particular, the upsampling can be done using vanilla bilinear texture sampling (which is very cheap), just being slightly clever about the fractional coordinates. It works well because, as stated, the kernel does a good job rejecting frequencies prone to aliasing. So, in particular, you don't get any real quality loss from doing 2x scale changes as opposed to bigger steps (and thus considerably larger FIR support). I have some Python notebooks with some of these results, haven't gotten around to publishing them yet.

kragen 2y ago
I look forward to being able to read your notebooks!
leguminous 2y ago
I have done something like this with a Lanczos kernel (a=1) downsizing repeatedly by 2x, a small Gaussian kernel, and then repeatedly upsizing by 2x with simple hardware bilinear sampling. The (2D) Lanczos downsizing can be done with only four samples using the bilinear sampling tricks that you mention, and I avoided expensive trigonometric functions, divisions, and the singularity at 0 by using an even 8th order polynomial approximation. I would be curious to see the results using this kernel, but the Lanczos is so far the best that I've tried.