6 ms·
Is this not outdated? It uses examples of how pixels on scanners, printers, and CRT monitors are not squares. Fair enough. Back when this was written, one coul
by crazygringo 6y ago
Is this not outdated?
It uses examples of how pixels on scanners, printers, and CRT monitors are not squares. Fair enough. Back when this was written, one could very reasonably argue that pixels were intended to be point samples with a convolution.
But modern pixels on most LCD screens certainly are exceedingly crisp squares (with subpixels), and processing RAW camera sensor data, antialiasing techniques, and image resampling seems to "mostly" assume that pixels do represent the average color over a square rather than a point sample. Not perfectly, but as a reasonable first-order approximation.
I understand that 25 years ago it might have been reasonable to argue that pixels aren't squares.
But today... aren't they, at least as a first-order approximation? (With some further adjustments to avoid antialiasing artifacts, desired sharpness, and so forth.)
The idea that pixels are primarily point samples seems far more misleading -- as if they were analagous to audio samples which are point samples as a first-order approximation, but pixels are nothing like audio samples.
- 3dee 6y agoThey are exactly like audio point samples. Pixels are always interpolated, just like audio. The fact that 1080 pixels exactly fit 1080 screen pixels don't give pixels a shape. And by the way, most modern display pixels are not squares. They come in all kinds of shapes. Triangles for example.
- skavi 6y agoThe AMOLED displays in most higher end phones definitely do not have square pixels.
- kragen 6y agoNo, there are two fundamental misunderstandings in your comment. You posit an opposition between "[arrays of] point samples with a convolution" and "[arrays of] exceedingly crisp squares". This is incorrect. If the kernel you convolve your array of point samples with is an exceedingly crisp square, the result is an array of exceedingly crisp squares. Convolution with a square is just a special case. Smith explains this, perhaps not very clearly, in Fig. 3, p. 5. Conceptualizing this as a convolution allows you not only to handle the special case you're saying is "a reasonable first-order approximation," but also to analyze what effects that display or sensor convolution is having on your image, and possibly to correct them. Moreover, it can handle some other kinds of image degradation as well that go beyond that first-order approximation, such as bokeh, diffraction, defocus, and nonuniformity within pixels, as long as it's the same for every pixel. That's how you find out what those "further adjustments to avoid antialiasing artifacts, desired sharpness, and so forth" are. And it's how you can go about analyzing what happens to the image when you, for example, resample to a different sampling grid, as Smith explains on p. 4. If you analyze different linear resampling algorithms using the theory of convolutions, you can predict precisely how good the results will be; if you do not use this theory, you will be puzzled and will not understand your results at all. Believe me, I know; I wandered in that desert for years. But you don't have to. Learn from my mistakes. Second, audio samples are frequently handled exactly the same way as pixels in this sense, using a "zero-order hold". If you oversample by taking 64 consecutive samples from your ADC and summing them, or set the PWM duty cycle on your Arduino to a new value every 125 μs and leave it there until the next sample, or shift a new sample into the shift register connected to your R-2R DAC and leave it there until the next sample, you're doing exactly the same thing as trying to draw a little square for each pixel—just in one temporal dimension instead of two spatial dimensions. A zero-order hold is a perfectly reasonable thing to do with audio samples for both input and output, although it's not the only option, and it can produce audible artifacts. The math for analyzing this—and correcting it, if that's what you want—is exactly the same as the analogous math for displays or camera sensors made out of little squares. Far from being "far more misleading", the analogy between pixels and audio samples is deeply insightful and unlocks a wealth of mathematical tools that can be applied in both domains, as well as, for example, to linear control theory and GPS position estimation. The results this analogy leads you to are profound truths, not falsehoods as you seem to think. And some of them are very beautiful. None of this has changed in 25 years (except the detail that now we have LCDs with little rectangles on them to complement the little squares of dye-sublimation printers that Smith mentions) and little of it has changed in 50 years.
- Zarel 6y agoI think your phrase "with subpixels" directly contradicts "exceedingly crisp squares". I'm actually somewhat curious what you think a subpixel is, if you think it can combine into an exceedingly crisp square. For instance, this is the Nexus One screen under a microscope: https://commons.wikimedia.org/wiki/File:Nexus_one_screen_microscope.jpg https://commons.wikimedia.org/wiki/File:Nexus_one_screen_mic... Each of those lights is a subpixel. The bottom half of this image shows some other common LCD pixel layouts: https://commons.wikimedia.org/wiki/File:Pixel_geometry_01_Pengo.jpg https://commons.wikimedia.org/wiki/File:Pixel_geometry_01_Pe... As you can see, subpixels can be square but usually aren't, and pixels can't be square because they're made of subpixels. Out of curiosity, I took a picture of my monitor, and this is what it looks like: https://i.imgur.com/Ce7186v.jpg https://i.imgur.com/Ce7186v.jpg These look very different from the microscope pictures because the light sources are "glowing", which I think is probably more accurate to how we actually perceive pixels. But as you can see, "glowing" makes them not square, but round!
- crazygringo 6y agoThe most common modern geometry is the fourth one ("LCD") in your second link, where each pixel is indeed perfectly square. That's what I was referring to. The fact it is made of three rectangular subpixels in no way means "pixels can't be square". And the "glowing" effect in your picture is an artifact of light bleed from your camera. The only time we perceive pixels as round rather than square is if we have eyesight problems that blur the image projected onto our retina.
- Zarel 6y agoI mean, with the fourth one, each pixel is only approximately square when white. A red, blue, or green pixel would be very rectangular. That's what I mean. Because of the way subpixels combine into pixels, only some colors can be square.
- sunshineforever 6y agoThe RGB subpixels of your monitor would be more apparent if you had a higher magnification. Compared to what I see under my jewelers loupes, the first image you posted looks close to my screen under 90x magnification. They display different levels of RGB to show different colors.
- hazeii 6y agoStraight-forward example of non-square pixels is the Raspberry Pi 7" LCD display. Viewable area is 155x86 and resolution is 800x480; simple plotting of a circle on it is visibly non-circular because the horizontal and vertical pitch are different.
- pornel 6y agoIMHO it was never fully accurate, because the term "pixel" refers to many things, from abstract concepts to physical devices. In image formats and image processing you can define pixel to be whatever you like. In many cases it's useful to define/interpret it as an infinitesimal point. In other algorithms, it's not useful. For example, pixels of textures are filtered (mipmaps, AF, MSAA), because an actual ideal point sample of a texel being read as an ideal point sample of a pixel on the screen has awful aliasing artifacts. Pixel samples are not the same as audio samples. Screen isn't a 2D wave. There's no gibbs phenomenon for pixels on screen. Treating samples as a wave works for resampling to a large degree, but breaks down in a few places like crisp edges (see font hinting, pixel art) and makes no sense for the alpha channel (wave-based sharpening filters create opaque halos). And when a pixel is connected to a physical device, it's clearly not a point. Camera sensors' pixels aren't points. LCD pixels aren't points. Printed pixels aren't points. Our eyes actually see the difference. Pixel-grid-aligned lines are clearly sharper than the same lines shifted by half of a pixel. If sampling theorem applied to pixels, it would make no difference.
- CodesInChaos 6y ago> and makes no sense for the alpha channel (wave-based sharpening filters create opaque halos). Alpha channels are generally problematic, since if alpha is small, big changes in the rgb channels result in small changes in the actual color. You can't linearly interpolate in this color space, just like you can't in a polor coordinate space like HSL. Pre-multiplied alpha should fix most of these problems (though you still need to be careful with fancy filters, no not end up with values outside the valid range)
- wtallis 6y agoThe little squares/rectangles mindset is more or less appropriate depending on the context. This paper is unfairly dismissive of the contexts where the little squares mindset is clearly better than the point samples or gaussians.
- deleted 6y ago[deleted]
- chadaustin 6y agoI dunno, I just looked closely at my OLED TV and it’s point lights (at least to the resolution of my own eyes), not tiny squares butting up to each other, so I think the lesson of treating your images as samples of a continuous function still applies.
- jayd16 6y agoNo, not at all. They're still point samples. You still deal with aliasing and all the learnings there even talking about rendered color buffers. No, LCDs are not crisp squares. Pretty much every screen type is still RGB in some kind of array. RGB stripe or Pentile are most common. You can also think about things like VR or a projector where your screen in transformed through a lens. Everyone has to constantly compensate for this fundamental thing in computer graphics to this day. Edit: As other things come to mind with how relevant this paper still is... >antialiasing techniques, and image resampling seems to "mostly" assume that pixels do represent the average color over a square rather than a point sample. Not perfectly, but as a reasonable first-order approximation. Not in the least. For example in texturing (wrapping images onto meshes), mip maps and bi/triliner filtering are very common. Assuming average color leads to a jagged (if you mean avg over one pixel) or blobby (if you mean average over several) mess. Mip maps are essentially point samples of a texture taken at different sample resolutions that are then resampled at runtime to form the best approximation. Essentially no one uses "average color." https://en.wikipedia.org/wiki/Trilinear_filtering https://en.wikipedia.org/wiki/Trilinear_filtering
- ksec 6y agoWould it still matter when PPI are 400+ if not 800 ppi? I was always under the impression as we increase PPI, antialiasing and other texturing technique would become irrelevant.
- jayd16 6y agoI'm not sure but we're not there yet if that is the case. I really doubt it though. I can't imagine a world where we chose to waste massive amounts of resolution in the normal case just so we don't need to sample as intelligently as we do today. Even in upsampling things we do better than simple square pixel average so if we're always upsampling we'll probably always need to be smart.
- citrin_ru 6y ago> I can't imagine a world where we chose to waste massive amounts of resolution in the normal case just so we don't need to sample as intelligently as we do today. It is already happening, see e. g. [1]-[3]. And the more common will be HiDPI displays - the worse would be support for low-res (say 96 DPI) displays. It is not uncommon to encounter an opinion that tricks used for high quality rendering on no-HiDPI displays are legacy and cruft, but more common is just to tread rendering issues which not visible on HiDPI as low priority. It is the same old mantra - programmer's time is expensive, hardware is cheap, let's not spend time on creating efficient software and focus on making development fast and easy. [1] https://github.com/harfbuzz/harfbuzz/issues/1892 https://github.com/harfbuzz/harfbuzz/issues/1892 [2] https://github.com/harfbuzz/harfbuzz/issues/2394 https://github.com/harfbuzz/harfbuzz/issues/2394 [3] https://gitlab.gnome.org/GNOME/pango/-/issues/463 https://gitlab.gnome.org/GNOME/pango/-/issues/463
- munchbunny 6y ago> But today... aren't they, at least as a first-order approximation? (With some further adjustments to avoid antialiasing artifacts, desired sharpness, and so forth.) It still depends on what you’re using them for. Image that will be rendered straight to a webpage? Yeah, it’s probably a fine first order approximation. But that’s not the only context that gets rendered on a modern screen. Say you’re using it for 3-d scenes. Yeah you absolutely should pay attention to what the paper says about footprint and sampling and filtering because, when you render it onto the screen, all of the processing steps between pixels on a texture to pixels on a screen require awareness of pixel footprint.
- jayd16 6y ago>Image that will be rendered straight to a webpage? Yeah, it’s probably a fine I don't know, I still think the concept would appear in the context of image scaling and aliasing.
- munchbunny 6y agoI agree in the sense that image scaling definitely requires you to think of pixels as more than just squares, but IMO browsers and OSes have gotten good enough at it that as a web developer you mostly don’t have to think too hard about it other than to remember to use high enough resolutions.
- jayd16 6y agoOh sure. At that point though, are you even thinking as far as "little squares"?
- munchbunny 6y agoFair point. I'm not really thinking of it as little squares, I'm more just thinking in terms of integer coordinates in a 2-d Cartesian space. But I do think I'm inherently abstracting the "screen" as a grid of square pixels because it's a serviceable abstraction that is substantially simpler than "here is the whole signal processing pipeline that sits between my <image> tag and some phone user's eyes." I suppose you're right that what I'm describing is just the lack of conscious mental model for what the pixel represents, versus when I'm writing a shader, where I'm actively thinking through the mental model of a pixel.