4 ms·

Let me insert my personal pet-peeve: have you thought of making it colour-space-aware? Most (all?) images you'll encounter are stored in sRGB colourspace, which

by Asooka 9y ago

Let me insert my personal pet-peeve: have you thought of making it colour-space-aware? Most (all?) images you'll encounter are stored in sRGB colourspace, which isn't linear, so you can't do the convolution by just multiplying and adding (the result will be slightly off). The easiest way would be to convert it to 16-bit-per-channel linear colour space using a lookup table, do the convolution in linear 16-bit space, then convert back to 8-bit sRGB.

pishpash 9y ago
For that matter, the conversion should be to CIELAB, which is perceptually uniform.
- pornel 9y ago
  No, it shouldn't be done in perceptually uniform color space. LAB, like sRGB, is non-linear. Resampling simulates physical properties of the light, not subjective perception of the eye.
  - jacobolus 9y ago
    Resampling in CIELAB space tends to work better than resampling in gamma-adjusted R′G′B′ space, because at least you never end up averaging two pixels and getting a lightness which is outside the range of input lightnesses, which is what causes the worst artifacts in R′G′B′. A linear space will give a better result, but CIELAB results are usually acceptable.
- lcrs 9y ago
  Perceptual uniformity is in some ways opposite to the linearization suggested above - the L* component of CIELAB is much more like the gamma-encoded values of sRGB than a linear light measure. It seems tough to come up with hard and fast rules for whether to mimic the linear physical processes, or work in a perceptual space more like the human visual system. I'd love to hear about more rigorous work in this area - most things I read have boiled down to "this way works better on these images". It's interesting for example that using Sinc-type filters to resize truly linear data, like that from HDR cameras, usually gives rise to horrible dark haloing artifacts around small specular highlights, despite that being the most "physically correct" way to do it. Doing the same operation in a more perceptual space immediately sorts out the problem.
homm 9y ago
Color space conversion is a hard topic in terms of performance. First of all, not all images are stored in sRGB. Most of them have another color profiles (such as P3 or ProPhoto). So, sRGB conversion is not enough, you need the full color management. Second, you'll see a real profit of color management only on a few images. Most time you'll see the difference only when you see both images at the same time on the same screen. For now, I came up to the resizing in original non-linear color space and saving the original color profile with the resulting image.
- mark-r 9y ago
  You don't really need the full color management for resizing, only fixing the gamma curve. I believe Apple's implementation of P3 uses the common 2.2 gamma, which is also approximately what sRGB uses. Unfortunately ProPhoto uses 1.8.
- jacobolus 9y ago
  By far most images in the wild are sRGB, and those that aren’t should typically be tagged with their color space. Resizing in gamma-adjusted space (sometimes) causes nasty artifacts when resizing. If you can afford the CPU use, always convert to an approximately linear space first, then downsize, then convert back. If you get the gamma curve slightly wrong (e.g. gamma = 2.0 vs. 2.2) it’s not too big a deal, the resulting artifacts won’t really be noticeable, so feel free to use a square root routine or something if it has better performance.
  - jacobolus 9y ago
    But note if you go to a linear space, you need to bump up the precision of your numbers. 8 bits per channel doesn’t cut it anymore, you’ll want 12 or 16 bits per channel (or even better, a floating point number). This might have a big effect on performance.
mark-r 9y ago
I particularly like this example image for testing: https://transloadit.com/blog/2010/10/is-your-image-scaling-software-up-to-scratch/ https://transloadit.com/blog/2010/10/is-your-image-scaling-s... You might be able to do this as a 3-part process instead of expecting the resizing to handle it natively. But that brings up a good question, does the new SIMD goodness work on anything other than 8-bit data? You couldn't do linear in anything less than 16 bits.
Matumio 9y ago
You can even see this problem in the article. The convolution-based sample image is clearly darker than the nearest-neighbour one. People go all crazy about interpolation and then get the brightness wrong. It's even more obvious for high resolution photos of a tree or grass in bright sunlight. Once you start looking you notice the change in brightness everywhere, when you click on a thumbnail or while a JPEG is loading.

by Asooka 9y ago

pishpash 9y ago

For that matter, the conversion should be to CIELAB, which is perceptually uniform.

pornel 9y ago
No, it shouldn't be done in perceptually uniform color space. LAB, like sRGB, is non-linear. Resampling simulates physical properties of the light, not subjective perception of the eye.
- jacobolus 9y ago
  Resampling in CIELAB space tends to work better than resampling in gamma-adjusted R′G′B′ space, because at least you never end up averaging two pixels and getting a lightness which is outside the range of input lightnesses, which is what causes the worst artifacts in R′G′B′. A linear space will give a better result, but CIELAB results are usually acceptable.
lcrs 9y ago
Perceptual uniformity is in some ways opposite to the linearization suggested above - the L* component of CIELAB is much more like the gamma-encoded values of sRGB than a linear light measure. It seems tough to come up with hard and fast rules for whether to mimic the linear physical processes, or work in a perceptual space more like the human visual system. I'd love to hear about more rigorous work in this area - most things I read have boiled down to "this way works better on these images". It's interesting for example that using Sinc-type filters to resize truly linear data, like that from HDR cameras, usually gives rise to horrible dark haloing artifacts around small specular highlights, despite that being the most "physically correct" way to do it. Doing the same operation in a more perceptual space immediately sorts out the problem.

homm 9y ago

Color space conversion is a hard topic in terms of performance. First of all, not all images are stored in sRGB. Most of them have another color profiles (such as P3 or ProPhoto). So, sRGB conversion is not enough, you need the full color management. Second, you'll see a real profit of color management only on a few images. Most time you'll see the difference only when you see both images at the same time on the same screen. For now, I came up to the resizing in original non-linear color space and saving the original color profile with the resulting image.

mark-r 9y ago
You don't really need the full color management for resizing, only fixing the gamma curve. I believe Apple's implementation of P3 uses the common 2.2 gamma, which is also approximately what sRGB uses. Unfortunately ProPhoto uses 1.8.
jacobolus 9y ago
By far most images in the wild are sRGB, and those that aren’t should typically be tagged with their color space. Resizing in gamma-adjusted space (sometimes) causes nasty artifacts when resizing. If you can afford the CPU use, always convert to an approximately linear space first, then downsize, then convert back. If you get the gamma curve slightly wrong (e.g. gamma = 2.0 vs. 2.2) it’s not too big a deal, the resulting artifacts won’t really be noticeable, so feel free to use a square root routine or something if it has better performance.
- jacobolus 9y ago
  But note if you go to a linear space, you need to bump up the precision of your numbers. 8 bits per channel doesn’t cut it anymore, you’ll want 12 or 16 bits per channel (or even better, a floating point number). This might have a big effect on performance.

mark-r 9y ago

I particularly like this example image for testing: https://transloadit.com/blog/2010/10/is-your-image-scaling-software-up-to-scratch/ https://transloadit.com/blog/2010/10/is-your-image-scaling-s... You might be able to do this as a 3-part process instead of expecting the resizing to handle it natively. But that brings up a good question, does the new SIMD goodness work on anything other than 8-bit data? You couldn't do linear in anything less than 16 bits.

Matumio 9y ago

You can even see this problem in the article. The convolution-based sample image is clearly darker than the nearest-neighbour one. People go all crazy about interpolation and then get the brightness wrong. It's even more obvious for high resolution photos of a tree or grass in bright sunlight. Once you start looking you notice the change in brightness everywhere, when you click on a thumbnail or while a JPEG is loading.