4 ms·

So, I'm assuming you know this, but the theoretically optimal pixel reconstruction filter is the sinc function. https://en.wikipedia.org/wiki/Sinc_filter https:

by theoh 8y ago

So, I'm assuming you know this, but the theoretically optimal pixel reconstruction filter is the sinc function. https://en.wikipedia.org/wiki/Sinc_filter https://en.wikipedia.org/wiki/Sinc_filter For very high-quality work, I guess we have enough processing power to approximate sinc quite well these days. Is that what you mean by trying to get close to the Nyquist limit? The sinc filter does tend to introduce ringing artifacts. Very high resolutions are now practical, too, which changes things a lot. Probably in favour of using filters which never introduce ringing artifacts, like the Gaussian. (And things like the Lanczos or Mitchell filters exist as compromises between sinc and Gaussian.)

dahart 8y ago
Yeah, I know about sinc and the ideal theory, but nobody uses sinc for production work. Your filter has to be too wide, and the ringing artifacts are too undesirable. The max sharpness in a digital image, the Nyquist limit, is 1 cycle across 2 pixels. It’s not possible to get any sharper than that. But actually rendering images with frequencies exactly at that limit generally cause aliasing artifacts of some kind. That’s what I mean about getting close to Nyquist. > Very high resolutions are now practical, too, which changes things a lot. Probably in favor of using filters which never introduce ringing artifacts, like the Gaussian. Yeah, this is mostly true, and it is one way I justify using a Gaussian. But it’s important to know and remember that without the right filter, you can get visible ringing artifacts no matter how high your resolution is, and no matter how much you super-sample your pixels. This doesn’t come up all that much in CG film or photography, but I have a digital art project where it matters a lot and comes up all the time. (And I order expensive prints too, where artifacts not very visible on my monitor can become plain & obvious on paper). > And things like the Lanzos or Mitchell filters exist as compromises between since and Gaussian. Yes, and I suspect those are the most common ones used in VFX and CG film & print production. But those are balancing sharpness and aliasing. For situations where I really need no aliasing, they aren’t good enough. I can’t usually see the difference between Gaussian, Lanczos and Mitchell without zooming and studying high frequency regions, but I’ve watched VFX supervisors comment on the softness they feel after watching like 2 seconds of film. Some people have an incredible sensitivity to sharpness.
- brudgers 8y ago
  + My understanding of the way the Nyquist limit plays out in digital photography is that it guarantees no moire. If moire isn't an issue in a scene then it is possible, but not guaranteed to get higher resolution (that of the individual pixel). Of course resolution and sharpness are not exactly the same thing...that's why a 16 mega pixels is a resolution not a sharpness and why a lot of studio photography uses cameras without anti-aliasing filters (e.g. medium format digital). In the studio, conditions can be set up to avoid moire in ways that aren't practical for sports or wedding photography. Similarly, moire isn't going to be a problem with astro-photography and some subjects in landscape photography. + For what it's worth, ringing artifacts are often caused by diffraction or lens aberrations rather than aliasing from sampling frequency (though they can be). + The shape of the modulation transfer function (MTF) is different between film and digital. The line pair comparison in the article illustrates it without being explicit. The Nyquist limit means that sharpness falls off a cliff in digital images and the example line pairs go from resolved to gray abruptly. The example line pairs for film fade to gray, what is and isn't sharp is determined by observation rather than an equation. Digital images often appear sharper because they are only ever so sharp and film appears softer because it is kinda sorta sharp beyond the point where it is clearly sharp. It's why film has a different look...or is a different artistic medium than digital when it comes to some forms of expression.
  - burfog 8y ago
    Getting close to the Nyquist limit is unsafe. The limit is theoretical; it applies only to an infinitely repeating signal. You'd have to create and then observe an infinite signal for the Nyquist limit to be correct, but of course you can't do this. That would be 2 samples for each feature detail you want. Things work OK in the real world at about 4 or 5 samples, but it takes 10 or 12 samples to work pretty well.
    - jimduk 8y ago
      I'm doing some amateur work on sub pixel matching at the block-level for stereo, currently using cheap APS-C prime lens commercial cameras ( Ricoh GR) and my background is maths not cameras, but what I've observed (all in RAW) is a) Real world scenes often go below the Nyquist limit ( e.g pattern from a brick wall at sufficient distance) b)In a camera without a low-pass filter (e.g. the Ricoh GR) in practice you don't see Moire but you can definitely generate it artificially off real world objects c) The Point Spread function seems quite important (e.g a 1-pixel scene feature for the GR will have half the effect on the sensor as a 2-pixel function because of the point spread) d) I'm a bit confused as why Fourier/ DCT all work as well as they do, as the Nyquist assumption for a modern digital camera seems incorrect, but on the whole sub-pixel matching in a Fourier space (while Nyquist assumption is not true) seems roughly on par with matching in the spatial domain - I would love to see something that explains why this is so Happy to discuss this more as I am still learning
burfog 8y ago
Isn't the sync more for 1-dimensional things, like audio? I would expect images to need the Airy disk. https://en.wikipedia.org/wiki/Airy_disk https://en.wikipedia.org/wiki/Airy_disk
- theoh 8y ago
  Sinc is closely related to Bessel functions (which is what the Airy disc is). In image processing afaik it is interpreted as a radially-symmetric filter, if it's not just used as if it was separable (i.e. on the x axis, then on the y axis). I don't think it is exactly the same as the Airy disc, but it's a good question. http://mathworld.wolfram.com/SincFunction.html http://mathworld.wolfram.com/SincFunction.html
  - jacobolus 8y ago
    There isn’t really any perfect choice for images with a square lattice of sample points. With a one-dimensional signal you don’t get the same annoying patterns from the interaction of the two grids along different dimensions. If you use a equilateral triangle grid (if you like, think of hexagonal pixels) you can do better.
madengr 8y ago
This is all very similar to Inter Symbol Interfence in digital comm systems. You don’t want your pixel bleeding into the next, so choosing the appropriate filter to reduce ISI is important. Typical root raised cosine filters are used, as it is a matched filter for itself, and minimizes ISI depending on the bandwidth chosen. I suppose Shannon’s capacity theorem can be applied to images, trading off bandwidth for SNR. If resolution is bandwidth, and brightness the SNR, what would capacity be in a photo? The sharpness? That would imply sharper photos would be better off with higher pixel sensors than with dynamic range. You can always decimate to increase SNR. In electronics, it’s easier to make faster converters than increase dynamic range, so oversample and decimate. Same probably with image sensors.
- theoh 8y ago
  Um, the first point, in digital imaging, is that a pixel is not a little square. It's a sample. So it doesn't have any extension to blur. Any filtering invariably happens either at sampling or at reconstruction. https://graphics.stanford.edu/courses/cs248-04/ho6/ https://graphics.stanford.edu/courses/cs248-04/ho6/ Of course sharpness is a function of resolution. I'm not sure it makes sense to talk about capacity, because typically the resolution of the image is non-negotiable. The number of bits needed to encode it is variable and depends on compression technique used. So in general, Shannon-style channel thinking operates on a different layer from the sampling and reconstruction processes we are discussing here. It's more relevant to how discrete pixel values are coded, or more elaborate compression mechanisms which exploit coherence in the image.
- dreamcompiler 8y ago
  > I suppose Shannon’s capacity theorem can be applied to images, trading off bandwidth for SNR. If resolution is bandwidth, and brightness the SNR, what would capacity be in a photo? The sharpness? That would imply sharper photos would be better off with higher pixel sensors than with dynamic range. To perform this tradeoff and capture e.g 1-bit images with super-high resolution you'd need the sensor to do error diffusion a la sigma-delta modulation, and AFAIK image sensors cannot currently do this.
  - 8y ago

by theoh 8y ago

dahart 8y ago

Yeah, I know about sinc and the ideal theory, but nobody uses sinc for production work. Your filter has to be too wide, and the ringing artifacts are too undesirable. The max sharpness in a digital image, the Nyquist limit, is 1 cycle across 2 pixels. It’s not possible to get any sharper than that. But actually rendering images with frequencies exactly at that limit generally cause aliasing artifacts of some kind. That’s what I mean about getting close to Nyquist. > Very high resolutions are now practical, too, which changes things a lot. Probably in favor of using filters which never introduce ringing artifacts, like the Gaussian. Yeah, this is mostly true, and it is one way I justify using a Gaussian. But it’s important to know and remember that without the right filter, you can get visible ringing artifacts no matter how high your resolution is, and no matter how much you super-sample your pixels. This doesn’t come up all that much in CG film or photography, but I have a digital art project where it matters a lot and comes up all the time. (And I order expensive prints too, where artifacts not very visible on my monitor can become plain & obvious on paper). > And things like the Lanzos or Mitchell filters exist as compromises between since and Gaussian. Yes, and I suspect those are the most common ones used in VFX and CG film & print production. But those are balancing sharpness and aliasing. For situations where I really need no aliasing, they aren’t good enough. I can’t usually see the difference between Gaussian, Lanczos and Mitchell without zooming and studying high frequency regions, but I’ve watched VFX supervisors comment on the softness they feel after watching like 2 seconds of film. Some people have an incredible sensitivity to sharpness.

brudgers 8y ago
+ My understanding of the way the Nyquist limit plays out in digital photography is that it guarantees no moire. If moire isn't an issue in a scene then it is possible, but not guaranteed to get higher resolution (that of the individual pixel). Of course resolution and sharpness are not exactly the same thing...that's why a 16 mega pixels is a resolution not a sharpness and why a lot of studio photography uses cameras without anti-aliasing filters (e.g. medium format digital). In the studio, conditions can be set up to avoid moire in ways that aren't practical for sports or wedding photography. Similarly, moire isn't going to be a problem with astro-photography and some subjects in landscape photography. + For what it's worth, ringing artifacts are often caused by diffraction or lens aberrations rather than aliasing from sampling frequency (though they can be). + The shape of the modulation transfer function (MTF) is different between film and digital. The line pair comparison in the article illustrates it without being explicit. The Nyquist limit means that sharpness falls off a cliff in digital images and the example line pairs go from resolved to gray abruptly. The example line pairs for film fade to gray, what is and isn't sharp is determined by observation rather than an equation. Digital images often appear sharper because they are only ever so sharp and film appears softer because it is kinda sorta sharp beyond the point where it is clearly sharp. It's why film has a different look...or is a different artistic medium than digital when it comes to some forms of expression.
- burfog 8y ago
  Getting close to the Nyquist limit is unsafe. The limit is theoretical; it applies only to an infinitely repeating signal. You'd have to create and then observe an infinite signal for the Nyquist limit to be correct, but of course you can't do this. That would be 2 samples for each feature detail you want. Things work OK in the real world at about 4 or 5 samples, but it takes 10 or 12 samples to work pretty well.
  - jimduk 8y ago
    I'm doing some amateur work on sub pixel matching at the block-level for stereo, currently using cheap APS-C prime lens commercial cameras ( Ricoh GR) and my background is maths not cameras, but what I've observed (all in RAW) is a) Real world scenes often go below the Nyquist limit ( e.g pattern from a brick wall at sufficient distance) b)In a camera without a low-pass filter (e.g. the Ricoh GR) in practice you don't see Moire but you can definitely generate it artificially off real world objects c) The Point Spread function seems quite important (e.g a 1-pixel scene feature for the GR will have half the effect on the sensor as a 2-pixel function because of the point spread) d) I'm a bit confused as why Fourier/ DCT all work as well as they do, as the Nyquist assumption for a modern digital camera seems incorrect, but on the whole sub-pixel matching in a Fourier space (while Nyquist assumption is not true) seems roughly on par with matching in the spatial domain - I would love to see something that explains why this is so Happy to discuss this more as I am still learning

burfog 8y ago

Isn't the sync more for 1-dimensional things, like audio? I would expect images to need the Airy disk. https://en.wikipedia.org/wiki/Airy_disk https://en.wikipedia.org/wiki/Airy_disk

theoh 8y ago
Sinc is closely related to Bessel functions (which is what the Airy disc is). In image processing afaik it is interpreted as a radially-symmetric filter, if it's not just used as if it was separable (i.e. on the x axis, then on the y axis). I don't think it is exactly the same as the Airy disc, but it's a good question. http://mathworld.wolfram.com/SincFunction.html http://mathworld.wolfram.com/SincFunction.html
- jacobolus 8y ago
  There isn’t really any perfect choice for images with a square lattice of sample points. With a one-dimensional signal you don’t get the same annoying patterns from the interaction of the two grids along different dimensions. If you use a equilateral triangle grid (if you like, think of hexagonal pixels) you can do better.

madengr 8y ago

This is all very similar to Inter Symbol Interfence in digital comm systems. You don’t want your pixel bleeding into the next, so choosing the appropriate filter to reduce ISI is important. Typical root raised cosine filters are used, as it is a matched filter for itself, and minimizes ISI depending on the bandwidth chosen. I suppose Shannon’s capacity theorem can be applied to images, trading off bandwidth for SNR. If resolution is bandwidth, and brightness the SNR, what would capacity be in a photo? The sharpness? That would imply sharper photos would be better off with higher pixel sensors than with dynamic range. You can always decimate to increase SNR. In electronics, it’s easier to make faster converters than increase dynamic range, so oversample and decimate. Same probably with image sensors.

theoh 8y ago
Um, the first point, in digital imaging, is that a pixel is not a little square. It's a sample. So it doesn't have any extension to blur. Any filtering invariably happens either at sampling or at reconstruction. https://graphics.stanford.edu/courses/cs248-04/ho6/ https://graphics.stanford.edu/courses/cs248-04/ho6/ Of course sharpness is a function of resolution. I'm not sure it makes sense to talk about capacity, because typically the resolution of the image is non-negotiable. The number of bits needed to encode it is variable and depends on compression technique used. So in general, Shannon-style channel thinking operates on a different layer from the sampling and reconstruction processes we are discussing here. It's more relevant to how discrete pixel values are coded, or more elaborate compression mechanisms which exploit coherence in the image.
dreamcompiler 8y ago
> I suppose Shannon’s capacity theorem can be applied to images, trading off bandwidth for SNR. If resolution is bandwidth, and brightness the SNR, what would capacity be in a photo? The sharpness? That would imply sharper photos would be better off with higher pixel sensors than with dynamic range. To perform this tradeoff and capture e.g 1-bit images with super-high resolution you'd need the sensor to do error diffusion a la sigma-delta modulation, and AFAIK image sensors cannot currently do this.
- 8y ago