5 ms·

A pixel is not a little square, a pixel is not a little square

46 points by ajg1977 16y ago

davi 16y ago
This came up recently in the Fiji project (a distribution of ImageJ and associated plugins): http://groups.google.com/group/fiji-devel/browse_thread/thread/01d7aa64a7df2145/6d3ccd8cb3894d15?#6d3ccd8cb3894d15 http://groups.google.com/group/fiji-devel/browse_thread/thre...
jacobolus 16y ago
For more on pixels, a talk and a paper by Dick Lyon: http://video.google.com/videoplay?docid=-5655850487750051532# http://video.google.com/videoplay?docid=-5655850487750051532... http://www.dicklyon.com/tech/Photography/Pixel-SPIE06-Lyon.pdf http://www.dicklyon.com/tech/Photography/Pixel-SPIE06-Lyon.p...
DLWormwood 16y ago
I've read this before, and I sympathize with his gripes, but it strikes me as a UX minefield to abandon the conceit now. If for no other reason, it's very useful to see the "box filter" effect when magnifying images when doing pixel art manipulation (still relevant for most cellphones and portable devices) as well as examining image compression quality. If most image viewers default to an interpolated view for magnification, I can see many graphics professionals' jobs get worse as it becomes even harder to convince to clients provide vector image files or raster images of sufficiently high resolution, since end users wouldn't see such an obvious distinction anymore.
- jacobolus 16y ago
  The “box filter effect” is only really helpful to someone who understands that the appearance of the magnified boxes is not remotely like the appearance of the final image. Its main utility is showing pixel artist that the samples are discrete, and very explicitly indicating their spacing. I don’t agree with you that the upsized box UI is necessarily the most intuitive or useful, but instead is just the only UI that has really been tested. I can imagine several other possibilities, for example showing a smooth interpolation (indicating something closer to real-world appearance) overlaid by a circle at each sample center (indicating color of the discrete sample). [That's an idea off the top of my head; I don't know if it would work in practice.]
sesqu 16y ago
Summary: Pixels are image samples, where the sampling methodology is undefined. Many people assume a square area per sample, probably because it's the easiest interpolation to make, but the author prefers point samples, which are easiest to do math on. The correctest interpretation depends on the sampling device, which has often been something like gaussian for optics, and representation, which has typically been ignored as complicated. Ironically, modern technology uses (to my understanding) rectangular spaced subpixels in both sampling and representation, which are still not square but much closer. Since technologies change and pixel density is high, the point-sample interpretation is both useful and reasonable, but I would caution against it until we all have retina displays and 4 megapixel digital cameras.
- jacobolus 16y ago
  > but the author prefers point samples, which are easiest to do math on. Easiest, and most correct, or at least a lot harder to screw up. > The correctest interpretation depends on the sampling device The “point-samples interpretation” is always perfectly correct, encompassing other definitions. The thing that differs from one device to another is the reconstruction filter used to interpret the meaning of those point samples. Given that a nearest-neighbor type reconstruction filter is pretty much never the best, using the “little square” interpretation that ties us down to that reconstruction is counterproductive. > Ironically, modern technology uses rectangular spaced subpixels in both sampling and representation. In sampling, I assume you mean the sensors in digital cameras? Treating them as squares/rectangles instead of point samples is not especially useful, because the optical system is pretty complicated, and so lens blur/chromatic aberration/etc. are at a larger scale than the pixels themselves. Better is to just wrap the knowledge of pixel shape and the knowledge of optical effects into the same part of the model, the reconstruction filter. The point-sample interpretation is perfectly relevant for current-resolution devices. In the case of LCDs for example, you just have to keep in mind that the sample points for R, G, and B subpixels are not the same.
tjmaxal 16y ago
This is from 1995, is this still relevant?
- mattmillr 16y ago
  The "until we all have retina displays and 4 megapixel digital cameras." from sesqu's summary is what makes it still relevant. Or newly relevant?
- corysama 16y ago
  If you are doing any kind of image processing, yes. It will be relevant forever.
CamperBob 16y ago
Unless I'm missing something (likely given the author's reputation), he goes off the rails on the very first page when he defines a pixel's coverage as [0,1] and not [0,1). Fill-convention discipline is at least as important as sampling considerations.
- jacobolus 16y ago
  We’re talking about samples of a (might-as-well-be) continuous space. Whether the interval is open or closed is irrelevant. If you want to think of that as an open paren, go ahead; the rest doesn’t depend on that at all.
  - CamperBob 16y ago
    It's irrelevant until you start applying rasterops to those pixels and areas.
    - deleted 16y ago
      [deleted]
- astrange 16y ago
  > he goes off the rails on the very first page when he defines a pixel's coverage as [0,1] and not [0,1) You mean, when he's defining the model he says isn't true? A pixel has no coverage; the reconstruction filter is what has the coverage, and it might overlap more than one pixel (e.g. when doing texture mapping).
Robin_Message 16y ago
Also, isn't he missing LCDs, where you really do have little squares of colour (albeit with borders)? Or is that the point? In fact, what is the point? All I got was rant and sampling theorems.
- astrange 16y ago
  LCD pixels are a set of three monochrome rectangles placed next to each other horizontally. Except for the ones that are vertical. And PenTile, which is missing some subpixels. And TVs, which display anamorphic video (or might be 768p) and whose pixels are not mapped to the monitor pixels. None of them are little squares of one color. What's wrong with sampling theorems anyway?
silly2 16y ago
this one dies at the mention that stoch screening for print is new. yeah, in the eighties boys. ooh, he didn't call it that. or mention any relevant color model all the rest is similar shick horror of anyone who never paid any atenntion to anything.
- silly2 16y ago
  sorry, credit must be given to an actual pdf of >4k words trying to look serious with awful, really unreadable. typography. yup, that's how i learned to con, mod me down etc, but believe it.
silly2 16y ago
god of mercy, can someone link to the old ttf hinting pages on msft opentype stuff? got author nailed there, promise you
bbulkow 16y ago
This is a 15 year old paper. Alvy Ray is a great guy, a luminary, love the paper, but at this point it's a historical document. The basic discussion was covered in my college graphics class in 1988. Anyone working in graphics, fonts, rendering or gaming who thinks this is news should have their literature examined.
rryan 16y ago
Am I the only one who was horrified by the typesetting of that document? It made it impossible to read.

46 points by ajg1977 16y ago

davi 16y ago

This came up recently in the Fiji project (a distribution of ImageJ and associated plugins): http://groups.google.com/group/fiji-devel/browse_thread/thread/01d7aa64a7df2145/6d3ccd8cb3894d15?#6d3ccd8cb3894d15 http://groups.google.com/group/fiji-devel/browse_thread/thre...

jacobolus 16y ago

For more on pixels, a talk and a paper by Dick Lyon: http://video.google.com/videoplay?docid=-5655850487750051532# http://video.google.com/videoplay?docid=-5655850487750051532... http://www.dicklyon.com/tech/Photography/Pixel-SPIE06-Lyon.pdf http://www.dicklyon.com/tech/Photography/Pixel-SPIE06-Lyon.p...

DLWormwood 16y ago

I've read this before, and I sympathize with his gripes, but it strikes me as a UX minefield to abandon the conceit now. If for no other reason, it's very useful to see the "box filter" effect when magnifying images when doing pixel art manipulation (still relevant for most cellphones and portable devices) as well as examining image compression quality. If most image viewers default to an interpolated view for magnification, I can see many graphics professionals' jobs get worse as it becomes even harder to convince to clients provide vector image files or raster images of sufficiently high resolution, since end users wouldn't see such an obvious distinction anymore.

jacobolus 16y ago
The “box filter effect” is only really helpful to someone who understands that the appearance of the magnified boxes is not remotely like the appearance of the final image. Its main utility is showing pixel artist that the samples are discrete, and very explicitly indicating their spacing. I don’t agree with you that the upsized box UI is necessarily the most intuitive or useful, but instead is just the only UI that has really been tested. I can imagine several other possibilities, for example showing a smooth interpolation (indicating something closer to real-world appearance) overlaid by a circle at each sample center (indicating color of the discrete sample). [That's an idea off the top of my head; I don't know if it would work in practice.]

sesqu 16y ago

Summary: Pixels are image samples, where the sampling methodology is undefined. Many people assume a square area per sample, probably because it's the easiest interpolation to make, but the author prefers point samples, which are easiest to do math on. The correctest interpretation depends on the sampling device, which has often been something like gaussian for optics, and representation, which has typically been ignored as complicated. Ironically, modern technology uses (to my understanding) rectangular spaced subpixels in both sampling and representation, which are still not square but much closer. Since technologies change and pixel density is high, the point-sample interpretation is both useful and reasonable, but I would caution against it until we all have retina displays and 4 megapixel digital cameras.

jacobolus 16y ago
> but the author prefers point samples, which are easiest to do math on. Easiest, and most correct, or at least a lot harder to screw up. > The correctest interpretation depends on the sampling device The “point-samples interpretation” is always perfectly correct, encompassing other definitions. The thing that differs from one device to another is the reconstruction filter used to interpret the meaning of those point samples. Given that a nearest-neighbor type reconstruction filter is pretty much never the best, using the “little square” interpretation that ties us down to that reconstruction is counterproductive. > Ironically, modern technology uses rectangular spaced subpixels in both sampling and representation. In sampling, I assume you mean the sensors in digital cameras? Treating them as squares/rectangles instead of point samples is not especially useful, because the optical system is pretty complicated, and so lens blur/chromatic aberration/etc. are at a larger scale than the pixels themselves. Better is to just wrap the knowledge of pixel shape and the knowledge of optical effects into the same part of the model, the reconstruction filter. The point-sample interpretation is perfectly relevant for current-resolution devices. In the case of LCDs for example, you just have to keep in mind that the sample points for R, G, and B subpixels are not the same.

tjmaxal 16y ago

This is from 1995, is this still relevant?

mattmillr 16y ago
The "until we all have retina displays and 4 megapixel digital cameras." from sesqu's summary is what makes it still relevant. Or newly relevant?
corysama 16y ago
If you are doing any kind of image processing, yes. It will be relevant forever.

CamperBob 16y ago

Unless I'm missing something (likely given the author's reputation), he goes off the rails on the very first page when he defines a pixel's coverage as [0,1] and not [0,1). Fill-convention discipline is at least as important as sampling considerations.

jacobolus 16y ago
We’re talking about samples of a (might-as-well-be) continuous space. Whether the interval is open or closed is irrelevant. If you want to think of that as an open paren, go ahead; the rest doesn’t depend on that at all.
- CamperBob 16y ago
  It's irrelevant until you start applying rasterops to those pixels and areas.
  - deleted 16y ago
    [deleted]
astrange 16y ago
> he goes off the rails on the very first page when he defines a pixel's coverage as [0,1] and not [0,1) You mean, when he's defining the model he says isn't true? A pixel has no coverage; the reconstruction filter is what has the coverage, and it might overlap more than one pixel (e.g. when doing texture mapping).

Robin_Message 16y ago

Also, isn't he missing LCDs, where you really do have little squares of colour (albeit with borders)? Or is that the point? In fact, what is the point? All I got was rant and sampling theorems.

astrange 16y ago
LCD pixels are a set of three monochrome rectangles placed next to each other horizontally. Except for the ones that are vertical. And PenTile, which is missing some subpixels. And TVs, which display anamorphic video (or might be 768p) and whose pixels are not mapped to the monitor pixels. None of them are little squares of one color. What's wrong with sampling theorems anyway?

silly2 16y ago

this one dies at the mention that stoch screening for print is new. yeah, in the eighties boys. ooh, he didn't call it that. or mention any relevant color model all the rest is similar shick horror of anyone who never paid any atenntion to anything.

silly2 16y ago
sorry, credit must be given to an actual pdf of >4k words trying to look serious with awful, really unreadable. typography. yup, that's how i learned to con, mod me down etc, but believe it.

silly2 16y ago

god of mercy, can someone link to the old ttf hinting pages on msft opentype stuff? got author nailed there, promise you

bbulkow 16y ago

This is a 15 year old paper. Alvy Ray is a great guy, a luminary, love the paper, but at this point it's a historical document. The basic discussion was covered in my college graphics class in 1988. Anyone working in graphics, fonts, rendering or gaming who thinks this is news should have their literature examined.

rryan 16y ago

Am I the only one who was horrified by the typesetting of that document? It made it impossible to read.