10 ms·
The Center of the Pixel is (0.5, 0.5)
- qubex 6y agoAgree. Hard to disagree, really.
- TwoBit 6y agoI wonder what drove Microsoft to fo it differently (and wrongly), despite that GL had already solved this.
- LeoPanthera 6y agoRelated: "A pixel is not a little square". http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf
- centimeter 6y agoWorth noting that this does not apply to physical cameras - a pixel is not, in fact, a point sample, but the integral over a sub-region of the sensor plane. It's also not a complete integral - the red pixels in an image are interpolated from squares that only cover a quarter of the image plane (on 95+% of sensors). Then you bring in low pass filters (or don't), and the signal theory starts to get a bit complicated.
- Kye 6y agoBayer filter: https://en.wikipedia.org/wiki/Bayer_filter https://en.wikipedia.org/wiki/Bayer_filter There are other ways to do it, but they generally have a lot in common.
- dTal 6y agoIt doesn't apply to screens either, as pixels are - manifestly - little squares. Your screen does not apply any sort of lovely reconstruction filter over this "array of point samples". In short, it's wrong. You can model an image as an array of point samples - however these are not "pixels".
- JonathonW 6y agoYour screen is the "lovely reconstruction filter over this 'array of point samples'". This is, for LCDs, usually an array of little squares... sort of (probably more accurately described as an array of little rectangles of different color). Things get more complicated when you start talking about less traditional subpixel arrangements like PenTile, or the behavior of old CRTs (where you don't necessarily have fully discrete pixels at all).
- zokier 6y agoIts a reconstruction of sorts, but arguably pretty much the worst possible reconstruction and pretty far from lovely, compared to e.g. the near perfect reconstruction filters in audio. I wonder if there have been any experiments constructing displays with optical filters to provide better reconstruction. I guess the visual analogue would be image upscaling, and in that sense the reconstruction what LCDs etc provide would be comparable to nearest neighbor scaling (which generally sucks)
- raphlinus 6y agoIt depends on the goals. If the goal is accurate frequency response and in particular absence of aliasing, then a little square is a bad reconstruction filter. But for graphics, why should this be the goal? Perhaps a better goal is high contrast of edges, and for this the box filter is one of the very best. An additional advantage of the box filter is that it has only positive support, so there's no clipping beyond white and black. This is especially helpful when rendering text. And honestly I believe that those huge sinc-approximating reconstruction filters are overly fetishized even in the audio space. The main reason they sound "nearly perfect" is that the cutoff is safely outside the audible range. Try filtering a perfect slow sawtooth through a brick wall with a cutoff in the audio range, say 8kHz. It sounds like a very audible "ping" at that frequency, with pre-echo at that because of the symmetry of sinc.
- zokier 6y agoOne problem with pixelation is that if you take your beautifully sharp-edged pixel-aligned square and move it 1mm to the left, it will not be perfectly sharp edged anymore. So it lack certain uniformity. Comparing again to audio, afaik (please correct me if I'm wrong!) you can phase shift/delay signals however you wish and it should not add distortion. Afaik this would come up in text rendering where the glyphs and strokes inevitably will not align with the pixel grid, but you would know better about that.
- mark-r 6y agoThe de-Bayering filter in a camera is a lot more sophisticated than a simple interpolation of pixels of the same color. Most real world images have a huge amount of correlation between the R,G,B channels because few colors are perfectly saturated. Filtering software takes advantage of this and gets much higher resolution than simple interpolation ever could.
- centimeter 6y agoDo you have any examples of this? I’m a bit surprised to hear it, because spectral selection in modern bayers is pretty good. https://iopscience.iop.org/article/10.1088/1742-6596/536/1/012021/pdf https://iopscience.iop.org/article/10.1088/1742-6596/536/1/0...
- mark-r 6y agoMany RAW developers give you a choice of which de-bayering algorithm to use. Some are optimized for maximum detail retention. You can easily see the difference on a black-and-white resolution test target, which has perfect correlation between the R,G,B values.
- pornel 6y agoHowever, image is not a wave either. I have mixed feelings about this memo. It's right about practical aspects of resampling filters, but tries too hard to justify that with sampling theory. For example, pixel-aligned sharp edges exist and are meaningful in images, unlike perfectly square waves in sampling theory.
- sudosysgen 6y agoPixel aligned sharp edges do not actually exist and are not meaningful in images, because a perfectly sharp lens does not exist (and cannot exist) and as a result you can never form a sharp edge in an image. You also have de-focus that prevents you from doing so, and a lens that has a wider depth of field has an immediately noticeable limit in sharpness. Even if you were somehow able to create a perfect lens, you would not be able to create a perfectly sharp edge with real world objects.
- wtallis 6y agoSee, here you're committing the same error that the paper does: pretending pixels are all about photography and optics, and ignoring that some computer-generated graphics actually are supposed to represent perfect squares. Sometimes, pixels really are little squares. Not always, but not never, either.
- chaorace 6y agoI disagree. Would you say a specific address in an array has edges? No, because it's an address, a fixed point. Even if that array accidentally described a square, the individual array addresses would have nothing to do with that. It's not a question of the representation, it's a question of quanta. Pixels are data, the little squares are the artifacts your LCD eventually produces with the help of that data.
- wtallis 6y agoWhen your GPU is rasterizing the edges of polygons, it computes (sometimes just approximates) how much of a little square is covered by that polygon and uses that as the weight when averaging what color to assign to that pixel. The resulting rendered image is most correctly interpreted as an array of little squares, not point samples and definitely not truncated gaussians.
- DougBTX 6y agoThe main argument from that article is "if you disagree with me I'll call you silly names", it isn't that convincing.
- jml7c5 6y agoWhat do you mean? The only objectionable content I see is a single mention of "little-squarists" [quotation marks included in original text], but that seems hardly enough of an insult to disregard the rest of the paper.
- DougBTX 6y agoThe first half of the paper is spent building up to the name-calling, it starts with "little squares", reaching a crescendo with the revelation that it is not the squares that are little, but the people! The closest it gets to a comparison ends abruptly with "we will not even further consider this possibility". The second half of the article continues as if the first half has convincing, just asserting that his solution is "so easy after all" that it must be right.
- starmole 6y agoThis is pedantic BS that has nothing to do with the issue in the article. The article talks about a very specific issue of offsets in practice cases. This is equivalent to somebody talking about simple circuits and then complaining that "but what about relativity".
- p1necone 6y agoOn every display I've ever used a pixel has corresponded to either a little square or a rectangle, that's good enough for me.
- Kednicma 6y agoI know that many languages have some sort of support for units. It would be nice to have libraries which explicitly say that (0,1) and (-1,1) are different, and support transforming between them. I think that this transformation comes up all the time when working with pixels that are properly aligned and centered.
- aspaceman 6y agoI know of yt-project, which has a lot of cool support for units in the context of sci-vis. Support for transforms between coordinate systems is nice though. Would love to have that. The only hard part is that systems which try and do this sort of thing lose some of the elegance of saying "V = (0, 1)" when you also have to specify the coordinate system you're working under for every vector. There have been some papers that do this though. I can't find the reference but I know it exists.
- draw_down 6y agoSure, and the center of a foursquare of pixels is 1,1.
- alleycat5000 6y agoThis comes up all the time in dealing with geospatial rasters. For instance, in GDAL there's a whole RFC for dealing with issues related to pixel corners versus pixel centers! https://gdal.org/development/rfc/rfc33_gtiff_pixelispoint.html https://gdal.org/development/rfc/rfc33_gtiff_pixelispoint.ht... "Traditionally GDAL has treated this flag as having no relevance to the georeferencing of the image despite disputes from a variety of other software developers and data producers. This was based on the authors interpretation of something said once by the GeoTIFF author. However, a recent review of section [section 2.5.2.2] of the GeoTIFF specificaiton has made it clear that GDAL behavior is incorrect and that PixelIsPoint georeferencing needs to be offset by a half a pixel when transformed to the GDAL georeferencing model."
- __abadams__ 6y agoPlacing pixel centers at 0.5, 0.5 is only the obvious choice if you think pixels are little squares rather than point samples. Pixels-as-squares makes intuitive sense to people, especially those raised on pixel art, but it's just one possible choice you can make for your reconstruction function. It's not even a particularly good choice. It doesn't model real sensors or real displays, and it doesn't have particularly nice theoretical properties. The only thing going for it is that it's cheaper to compute some things.
- anderskaseorg 6y agoIt’s true that pixels aren’t most accurately modeled as squares, but they should still be centered at (0.5, 0.5), because you want the center of mass of a W×H pixel image to be at exactly (W/2, H/2) no matter what shape the pixels are. Otherwise it shifts around when you resize the image—perhaps even by much more than 1 pixel if you resize it by a large factor.
- 0xfaded 6y agoUnfortunately doesn't make sense when you need to look up pixel 0.5,0.5 in the framebuffer. When dealing with cameras, the central point us rarely h/2,w/2. So you're really dealing with two sets of coordinates, camera coordinates and sensor coordinates, that need to be converted between. Integer coordinates are convenient for accessing the sensor pixels, and the camera-to-sensor space transform should theoretically include for the 0.5,0.5 offset. However, getting a calibration within 0.5 pixels accuracy is going to be hard to begin with.
- wnoise 6y agoSub-pixel accuracy is needed for good results when using stereo camera pairs, and can routinely be achieved.
- anderskaseorg 6y agoNobody’s suggesting that pixels are stored at half-integer memory addresses. After all, only a small subset of the continuous image space will lie exactly on the grid of pixel centers—and this is true no matter how the grid is offset. The point is that the grid should be considered as being lined up with (0.5, 0.5) rather than with (0, 0). So, for example, if you’re scaling an image up by 10× with bilinear interpolation, and you need to figure out what to store at address (7, 23) in the output framebuffer, you should convert that to continuous coordinates (7.5, 23.5), scale these continuous coordinates down to (0.75, 2.35), and use that to take the appropriate weighted average of the surrounding input pixels centered at (0.5, 1.5), (1.5, 1.5), (0.5, 2.5), and (1.5, 2.5), which are located at address (0, 1), (1, 1), (0, 2), and (1, 2) in the input framebuffer. The result will be different and visually more correct than if you had done the computation without taking the (0.5, 0.5) offset into account. In this case the naive computation would instead give you a combination of the pixels at (0, 2), (1, 2), (0, 3), and (1, 3) in the input framebuffer, and the result would appear to be shifted by a subpixel offset. This was essentially the cause of a GIMP bug that I reported in 2009: https://bugzilla.gnome.org/show_bug.cgi?id=592628 https://bugzilla.gnome.org/show_bug.cgi?id=592628.
- fallingfrog 6y agoI once made a half-pixel error trying to write a fast Fourier transform as a series of fragment shaders. That was nasty to track down! The error was smeared across all the pixels in the frequency domain causing weird subtle frequency biases in certain directions.
- VikingCoder 6y agoYes, and enjoy working with MIPMaps.
- mark-r 6y agoI once made a persuasive argument that this was the proper interpretation of a pixel, and got a major app to adopt the convention. It wasn't until much later that I discovered the error. The problem comes when you try to align raster images with vector ones. Instead of starting a line at 0,0 you need to start it at 0.5,0.5. And heaven help you if you're combining raster images at different resolutions, they'll never line up. The proper way to work is to put the pixel center at 0,0 and let it extend from -0.5 to 0.5. This works out well with reconstruction or resizing filters, because they're symmetric around 0,0 too.
- codeflo 6y agoPixel centers at (0.5, 0.5) is the convention adopted by all major graphics APIs (OpenGL, DirectX, Vulkan, Metal). That’s because it makes the math, especially around image scaling, becomes a lot simpler. What exactly the alignment problem you say isn’t solvable? One common mistake is to not think in floating point coordinates all the way through. E.g. a rectangle that covers a single pixel should have coordinates like (100.0,100.0)-(101.0,101.0), NOT involving a 0.5 offset. You almost never offset anything by 0.5 in this convention. 1 pixel wide lines are an exception, but only because then the edges of the line are exactly at pixel boundaries.
- redis_mlc 6y agoMy understanding is that graphics are smeared on Unix systems, but sharp on Windows systems, because Unix libraries use 0.5, 0.5 as the center and Windows uses 0,0. Can anybody confirm that they investigated that deeply and found that to be true, or is there another explanation?
- codeflo 6y agoI have never heard such a thing. Are you talking about font rendering differences? There are different anti-aliasing algorithms, but they don’t have anything to do with this convention.
- ur-whale 6y agoNot exactly news