7 ms·

Related: "A pixel is not a little square". http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf

centimeter 6y ago
Worth noting that this does not apply to physical cameras - a pixel is not, in fact, a point sample, but the integral over a sub-region of the sensor plane. It's also not a complete integral - the red pixels in an image are interpolated from squares that only cover a quarter of the image plane (on 95+% of sensors). Then you bring in low pass filters (or don't), and the signal theory starts to get a bit complicated.
- Kye 6y ago
  Bayer filter: https://en.wikipedia.org/wiki/Bayer_filter https://en.wikipedia.org/wiki/Bayer_filter There are other ways to do it, but they generally have a lot in common.
- dTal 6y ago
  It doesn't apply to screens either, as pixels are - manifestly - little squares. Your screen does not apply any sort of lovely reconstruction filter over this "array of point samples". In short, it's wrong. You can model an image as an array of point samples - however these are not "pixels".
  - JonathonW 6y ago
    Your screen is the "lovely reconstruction filter over this 'array of point samples'". This is, for LCDs, usually an array of little squares... sort of (probably more accurately described as an array of little rectangles of different color). Things get more complicated when you start talking about less traditional subpixel arrangements like PenTile, or the behavior of old CRTs (where you don't necessarily have fully discrete pixels at all).
    - zokier 6y ago
      Its a reconstruction of sorts, but arguably pretty much the worst possible reconstruction and pretty far from lovely, compared to e.g. the near perfect reconstruction filters in audio. I wonder if there have been any experiments constructing displays with optical filters to provide better reconstruction. I guess the visual analogue would be image upscaling, and in that sense the reconstruction what LCDs etc provide would be comparable to nearest neighbor scaling (which generally sucks)
      - raphlinus 6y ago
        It depends on the goals. If the goal is accurate frequency response and in particular absence of aliasing, then a little square is a bad reconstruction filter. But for graphics, why should this be the goal? Perhaps a better goal is high contrast of edges, and for this the box filter is one of the very best. An additional advantage of the box filter is that it has only positive support, so there's no clipping beyond white and black. This is especially helpful when rendering text. And honestly I believe that those huge sinc-approximating reconstruction filters are overly fetishized even in the audio space. The main reason they sound "nearly perfect" is that the cutoff is safely outside the audible range. Try filtering a perfect slow sawtooth through a brick wall with a cutoff in the audio range, say 8kHz. It sounds like a very audible "ping" at that frequency, with pre-echo at that because of the symmetry of sinc.
        zokier 6y ago
        One problem with pixelation is that if you take your beautifully sharp-edged pixel-aligned square and move it 1mm to the left, it will not be perfectly sharp edged anymore. So it lack certain uniformity. Comparing again to audio, afaik (please correct me if I'm wrong!) you can phase shift/delay signals however you wish and it should not add distortion. Afaik this would come up in text rendering where the glyphs and strokes inevitably will not align with the pixel grid, but you would know better about that.
        wtallis 6y ago
  - ckcheng 6y ago
    The memo interestingly talked about screens, and that it does not contribute to the pixels as squares model because there are "overlapping shapes that serve as natural reconstruction filters"... But it was in context of old CRT and Sony Trinitron monitors! I was wondering what it'd say about LCD screens but the memo is from 1995, and the first standalone LCDs only appeared in the mid-1990s and were expensive [1]. What it says about CRT electron beams no longer apply, but I'm guessing this still does: > The value of a pixel is converted, for each primary color, to a voltage level. This stepped voltage is passed through electronics which, by its very nature, rounds off the edges of the level steps > Your eye then integrates the light pattern from a group of triads into a color > There are, as usual in imaging, overlapping shapes that serve as natural reconstruction filters [1]: https://en.wikipedia.org/wiki/Computer_monitor#Liquid_crystal_display https://en.wikipedia.org/wiki/Computer_monitor#Liquid_crysta...
  - gorgoiler 6y ago
    Presented without comment: pixel art as rendered by a CRT: https://i.redd.it/gc0vsn8zqky11.jpg https://i.redd.it/gc0vsn8zqky11.jpg
- mark-r 6y ago
  The de-Bayering filter in a camera is a lot more sophisticated than a simple interpolation of pixels of the same color. Most real world images have a huge amount of correlation between the R,G,B channels because few colors are perfectly saturated. Filtering software takes advantage of this and gets much higher resolution than simple interpolation ever could.
  - centimeter 6y ago
    Do you have any examples of this? I’m a bit surprised to hear it, because spectral selection in modern bayers is pretty good. https://iopscience.iop.org/article/10.1088/1742-6596/536/1/012021/pdf https://iopscience.iop.org/article/10.1088/1742-6596/536/1/0...
    - mark-r 6y ago
      Many RAW developers give you a choice of which de-bayering algorithm to use. Some are optimized for maximum detail retention. You can easily see the difference on a black-and-white resolution test target, which has perfect correlation between the R,G,B values.
pornel 6y ago
However, image is not a wave either. I have mixed feelings about this memo. It's right about practical aspects of resampling filters, but tries too hard to justify that with sampling theory. For example, pixel-aligned sharp edges exist and are meaningful in images, unlike perfectly square waves in sampling theory.
- sudosysgen 6y ago
  Pixel aligned sharp edges do not actually exist and are not meaningful in images, because a perfectly sharp lens does not exist (and cannot exist) and as a result you can never form a sharp edge in an image. You also have de-focus that prevents you from doing so, and a lens that has a wider depth of field has an immediately noticeable limit in sharpness. Even if you were somehow able to create a perfect lens, you would not be able to create a perfectly sharp edge with real world objects.
  - wtallis 6y ago
    See, here you're committing the same error that the paper does: pretending pixels are all about photography and optics, and ignoring that some computer-generated graphics actually are supposed to represent perfect squares. Sometimes, pixels really are little squares. Not always, but not never, either.
    - chaorace 6y ago
      I disagree. Would you say a specific address in an array has edges? No, because it's an address, a fixed point. Even if that array accidentally described a square, the individual array addresses would have nothing to do with that. It's not a question of the representation, it's a question of quanta. Pixels are data, the little squares are the artifacts your LCD eventually produces with the help of that data.
      - wtallis 6y ago
        When your GPU is rasterizing the edges of polygons, it computes (sometimes just approximates) how much of a little square is covered by that polygon and uses that as the weight when averaging what color to assign to that pixel. The resulting rendered image is most correctly interpreted as an array of little squares, not point samples and definitely not truncated gaussians.
DougBTX 6y ago
The main argument from that article is "if you disagree with me I'll call you silly names", it isn't that convincing.
- jml7c5 6y ago
  What do you mean? The only objectionable content I see is a single mention of "little-squarists" [quotation marks included in original text], but that seems hardly enough of an insult to disregard the rest of the paper.
  - DougBTX 6y ago
    The first half of the paper is spent building up to the name-calling, it starts with "little squares", reaching a crescendo with the revelation that it is not the squares that are little, but the people! The closest it gets to a comparison ends abruptly with "we will not even further consider this possibility". The second half of the article continues as if the first half has convincing, just asserting that his solution is "so easy after all" that it must be right.
starmole 6y ago
This is pedantic BS that has nothing to do with the issue in the article. The article talks about a very specific issue of offsets in practice cases. This is equivalent to somebody talking about simple circuits and then complaining that "but what about relativity".
p1necone 6y ago
On every display I've ever used a pixel has corresponded to either a little square or a rectangle, that's good enough for me.
- astrange 6y ago
  This isn't true on modern Pentile OLEDs. It's also not true on camera sensors.
  - bzb3 6y ago
    And that's why pentile technologies are practically abandoned now.

centimeter 6y ago

Worth noting that this does not apply to physical cameras - a pixel is not, in fact, a point sample, but the integral over a sub-region of the sensor plane. It's also not a complete integral - the red pixels in an image are interpolated from squares that only cover a quarter of the image plane (on 95+% of sensors). Then you bring in low pass filters (or don't), and the signal theory starts to get a bit complicated.

Kye 6y ago
Bayer filter: https://en.wikipedia.org/wiki/Bayer_filter https://en.wikipedia.org/wiki/Bayer_filter There are other ways to do it, but they generally have a lot in common.
dTal 6y ago
It doesn't apply to screens either, as pixels are - manifestly - little squares. Your screen does not apply any sort of lovely reconstruction filter over this "array of point samples". In short, it's wrong. You can model an image as an array of point samples - however these are not "pixels".
- JonathonW 6y ago
  Your screen is the "lovely reconstruction filter over this 'array of point samples'". This is, for LCDs, usually an array of little squares... sort of (probably more accurately described as an array of little rectangles of different color). Things get more complicated when you start talking about less traditional subpixel arrangements like PenTile, or the behavior of old CRTs (where you don't necessarily have fully discrete pixels at all).
  - zokier 6y ago
    Its a reconstruction of sorts, but arguably pretty much the worst possible reconstruction and pretty far from lovely, compared to e.g. the near perfect reconstruction filters in audio. I wonder if there have been any experiments constructing displays with optical filters to provide better reconstruction. I guess the visual analogue would be image upscaling, and in that sense the reconstruction what LCDs etc provide would be comparable to nearest neighbor scaling (which generally sucks)
    - raphlinus 6y ago
      It depends on the goals. If the goal is accurate frequency response and in particular absence of aliasing, then a little square is a bad reconstruction filter. But for graphics, why should this be the goal? Perhaps a better goal is high contrast of edges, and for this the box filter is one of the very best. An additional advantage of the box filter is that it has only positive support, so there's no clipping beyond white and black. This is especially helpful when rendering text. And honestly I believe that those huge sinc-approximating reconstruction filters are overly fetishized even in the audio space. The main reason they sound "nearly perfect" is that the cutoff is safely outside the audible range. Try filtering a perfect slow sawtooth through a brick wall with a cutoff in the audio range, say 8kHz. It sounds like a very audible "ping" at that frequency, with pre-echo at that because of the symmetry of sinc.
      - zokier 6y ago
        One problem with pixelation is that if you take your beautifully sharp-edged pixel-aligned square and move it 1mm to the left, it will not be perfectly sharp edged anymore. So it lack certain uniformity. Comparing again to audio, afaik (please correct me if I'm wrong!) you can phase shift/delay signals however you wish and it should not add distortion. Afaik this would come up in text rendering where the glyphs and strokes inevitably will not align with the pixel grid, but you would know better about that.
        wtallis 6y ago
- ckcheng 6y ago
  The memo interestingly talked about screens, and that it does not contribute to the pixels as squares model because there are "overlapping shapes that serve as natural reconstruction filters"... But it was in context of old CRT and Sony Trinitron monitors! I was wondering what it'd say about LCD screens but the memo is from 1995, and the first standalone LCDs only appeared in the mid-1990s and were expensive [1]. What it says about CRT electron beams no longer apply, but I'm guessing this still does: > The value of a pixel is converted, for each primary color, to a voltage level. This stepped voltage is passed through electronics which, by its very nature, rounds off the edges of the level steps > Your eye then integrates the light pattern from a group of triads into a color > There are, as usual in imaging, overlapping shapes that serve as natural reconstruction filters [1]: https://en.wikipedia.org/wiki/Computer_monitor#Liquid_crystal_display https://en.wikipedia.org/wiki/Computer_monitor#Liquid_crysta...
- gorgoiler 6y ago
  Presented without comment: pixel art as rendered by a CRT: https://i.redd.it/gc0vsn8zqky11.jpg https://i.redd.it/gc0vsn8zqky11.jpg
mark-r 6y ago
The de-Bayering filter in a camera is a lot more sophisticated than a simple interpolation of pixels of the same color. Most real world images have a huge amount of correlation between the R,G,B channels because few colors are perfectly saturated. Filtering software takes advantage of this and gets much higher resolution than simple interpolation ever could.
- centimeter 6y ago
  Do you have any examples of this? I’m a bit surprised to hear it, because spectral selection in modern bayers is pretty good. https://iopscience.iop.org/article/10.1088/1742-6596/536/1/012021/pdf https://iopscience.iop.org/article/10.1088/1742-6596/536/1/0...
  - mark-r 6y ago
    Many RAW developers give you a choice of which de-bayering algorithm to use. Some are optimized for maximum detail retention. You can easily see the difference on a black-and-white resolution test target, which has perfect correlation between the R,G,B values.

pornel 6y ago

However, image is not a wave either. I have mixed feelings about this memo. It's right about practical aspects of resampling filters, but tries too hard to justify that with sampling theory. For example, pixel-aligned sharp edges exist and are meaningful in images, unlike perfectly square waves in sampling theory.

sudosysgen 6y ago
Pixel aligned sharp edges do not actually exist and are not meaningful in images, because a perfectly sharp lens does not exist (and cannot exist) and as a result you can never form a sharp edge in an image. You also have de-focus that prevents you from doing so, and a lens that has a wider depth of field has an immediately noticeable limit in sharpness. Even if you were somehow able to create a perfect lens, you would not be able to create a perfectly sharp edge with real world objects.
- wtallis 6y ago
  See, here you're committing the same error that the paper does: pretending pixels are all about photography and optics, and ignoring that some computer-generated graphics actually are supposed to represent perfect squares. Sometimes, pixels really are little squares. Not always, but not never, either.
  - chaorace 6y ago
    I disagree. Would you say a specific address in an array has edges? No, because it's an address, a fixed point. Even if that array accidentally described a square, the individual array addresses would have nothing to do with that. It's not a question of the representation, it's a question of quanta. Pixels are data, the little squares are the artifacts your LCD eventually produces with the help of that data.
    - wtallis 6y ago
      When your GPU is rasterizing the edges of polygons, it computes (sometimes just approximates) how much of a little square is covered by that polygon and uses that as the weight when averaging what color to assign to that pixel. The resulting rendered image is most correctly interpreted as an array of little squares, not point samples and definitely not truncated gaussians.

DougBTX 6y ago

The main argument from that article is "if you disagree with me I'll call you silly names", it isn't that convincing.

jml7c5 6y ago
What do you mean? The only objectionable content I see is a single mention of "little-squarists" [quotation marks included in original text], but that seems hardly enough of an insult to disregard the rest of the paper.
- DougBTX 6y ago
  The first half of the paper is spent building up to the name-calling, it starts with "little squares", reaching a crescendo with the revelation that it is not the squares that are little, but the people! The closest it gets to a comparison ends abruptly with "we will not even further consider this possibility". The second half of the article continues as if the first half has convincing, just asserting that his solution is "so easy after all" that it must be right.

starmole 6y ago

This is pedantic BS that has nothing to do with the issue in the article. The article talks about a very specific issue of offsets in practice cases. This is equivalent to somebody talking about simple circuits and then complaining that "but what about relativity".

p1necone 6y ago

On every display I've ever used a pixel has corresponded to either a little square or a rectangle, that's good enough for me.

astrange 6y ago
This isn't true on modern Pentile OLEDs. It's also not true on camera sensors.
- bzb3 6y ago
  And that's why pentile technologies are practically abandoned now.