8 ms·

A Pixel Is Not a Little Square (1995) [pdf]

118 points by rinesh 12y ago

davrosthedalek 12y ago
Stopped reading at "A pixel is a point sample." No, it's not. We wouldn't have sub-pixel font rendering and hundreds of anti-aliasing filter modes for the graphics cards. The pixel is an /area/ element of your image. And yes, many times, it's a square.
- RubyPinch 12y ago
  It mentioned colors right after that statement. edit: I do feel we are all smart enough to realize that the circle-vs-square, when considering a single channel, isn't affected much when colors are then considered offsets for each color are not unimaginable
- angersock 12y ago
  So, you made it to the middle of the second page? Anti-aliasing is a direct result of the fact that pixels are point-samples--the idea is to combine multiple samples and weight them so that the collection of point samples (in the form of a frame buffer) still represents a perceptually-reasonable form of the image.
  - davrosthedalek 12y ago
    Sorry, but you got it backwards. Exactly because pixels are /not/ point samples, but have an area, point sampling does not work, and you need anti-aliasing. In the limit of infinite resolution/area of the pixels gets infinitely small, point-sampling would be enough. You say it yourself: Since solving the integration over the pixel area is generally hard, we approximate the integration with multiple point samples.
    - angersock 12y ago
      Hm, I think I see the point you're getting at. It's still a good paper. :)
- gumby 12y ago
  > Stopped reading at "A pixel is a point sample." > ...yes, many times, it's a square. While "appeal to authority" is generally a bad way to end a discussion, I will start by pointing out that this is Alvy Ray Smith writing, and he was writing in 1995. So while you could end up disagreeing with him (though I hope you wouldn't), you might consider why he was writing. Your comment shows that this paper is just as applicable today. (since you stopped on page two you never saw the several examples of why this matters in the article, including one on scanning and one on pre-rendering geometric computation). Those graphics cards you refer to reflect Smith's influence on their design.You have sub pixel rendering today because pixels aren't squares. This was computationally prohibitive in 1995. The TL;DR of this brief paper is "IO is really really hard, and though accepting shortcuts and defaults only kinda works and here's why." You benefit from a lot of work done in the last 20 years to improve the quality of the libraries and hardware available to you, but still, "photo-realistic" graphics aren't. By the way, rendered pixels weren't squares in those days (they were not-always-circular dots on a screen which itself wasn't flat) and they aren't on today's flat LCDs (you can think of them as rectangular, though the color elements do not present uniformly though the area) and they sure as hell aren't on any device with lenses or mirrors (including projectors and headsets)
  - davrosthedalek 12y ago
    I realize that sub-pixel rendering is a misleading term, there are two concepts: a) Making use of fact that a physical pixel is a composite of multiple, disjunct areas of single-colored elements. b) Rendering objects with higher-than-pixel-density resolution, for example lines which appear thinner than a pixel. /Both/ relies on the fact that a pixel has an area, i.e. it is not a point, which is the point I was trying to make. A pixel on LCDs is indeed square most of the time, the subpixel elements are not. CRTs, of course, different story. All the "problems" he describes actually arise from the misconception that a pixel is a /point/ sample. It is an area sample, and if you treat it as such, the rest follows.
  - gaze 12y ago
    I think everyone in signal processing writes the same rant against zero-order-hold at some point. The Xiph people did (http://xiph.org/video/ http://xiph.org/video/) and so has this guy.
- corysama 12y ago
  What is behind the pixel is an area. But, the pixel itself is a pinhole viewport into that area. This article is about resampling and reconstructing images that are already encoded as pixels (texels would be more accurate, but I don't know if that term was around back in 93). So, when zooming in on a pixelated image, how do we fill in the gaps between the pinholes? Declaring pixels to be square areas means that larger views of a given image really should be composed of larger, hard-edged squares for each source pixel. I.e point sampling is truth. The reality is that what's really between the pixel centerpoints is unknown --lost forever or never calculated at all. To fill in the gaps we can only make intelligent guesses. Therefore: sampling theory.
- akira2501 12y ago
  > And yes, many times, it's a square. Are you sure? In my experience, it depends on the context, but the pixels themselves are _not_ always of equal height:width ratio. http://en.wikipedia.org/wiki/Pixel_aspect_ratio http://en.wikipedia.org/wiki/Pixel_aspect_ratio
  - davrosthedalek 12y ago
    not always = many times?
    - akira2501 12y ago
      "And yes, many times, ... " I took it to mean, "yes, many times over." As in, without a doubt, all pixels are square. My error.
dsjoerg 12y ago
I skimmed it and failed to find an explanation of what problems are caused by thinking of a pixel as a little square. Anyone?
- Roboprog 12y ago
  Indeed. As far as I could tell, the article was an exercise in "TMTOWTDI" - sometimes squares, sometimes subdivided squares (trinitron), sometimes little circles with gaps or overlaps, sometimes linear interpolation of a sample. OK, it's an abstraction, but the article didn't seem to cover the strength of weakness of various implementations.
  - Roboprog 12y ago
    And sometimes, it was an asterisk in a 60 x 80 grid (with a messed up aspect ratio) on a piece of printer paper, back in the stone age :-)
  - fsloth 12y ago
    I think there is a slight overload of terminology here. The samples in the source image, and the tiny light sources that vary in intensity and are driven by a voltage after the information in the bits in the source image has traveled through the display pipeline are two different things. The author discusses the best way to think about the source image. It will get really awkward if we want to discuss imaging algorithms and have to define "my pixel is a square... if it's a bunch of weird shapes the transform is this... etc." It's easiest from this point of view to think of the source image as a lattice of point samples and figure out what the sampling function (our physical pixels) do to the image after we have figured out the math on the point sample source.
- fl0wenol 12y ago
  Programmers would implicitly use a box filter when sampling from an underlying process (like 3d rendering, image scaling, scanning) into screen coordinates. It's one thing when writing a compositing window manager or blitting sprites to the screen; it's another thing entirely when doing any kind of image processing to make that assumption. It's similar to assuming that RGB (0-255) is perceptually linear and consistent among monitors (not understanding monitor profiles and color spaces)
sehugg 12y ago
Since this is 1995, pixels didn't map 1-to-1 to phosphors on the CRT. Now with LED and LCD displays they do. Not that this invalidates the paper, but it certainly adds some context.
- fit2rule 12y ago
  Um, sorry but .. no. They do not just magically 'map 1-to-1', and that is anyway not the point. The point is: a pixel is not a point. It is a window.
  - gaze 12y ago
    Other way around. A pixel IS a point. It's a sample.
    - robert_tweed 12y ago
      That's the definition used in the paper. The other definition is that a pixel is in fact, a square window containing many samples that fall within that region and rendered as their average. This is known as antialiasing.
      - rspeer 12y ago
        The article illustrates antialiasing, but chooses to use sinc-based circular windows, not square windows. Am I right to think that you'd only use square windows in the 3D-rendered graphics case, where you've already assumed the sub-pixels are square for efficiency?
        robert_tweed 12y ago
        The main point is that there are two conflicting views of what a "pixel" is. One is the representation of the samples and the other is what is actually displayed on-screen. Take for instance a ray tracer. Each ray is a sample and these are cast from the focal point, through the view frustum, and the colour is calculated where it intersects an object in the scene. In the naive case, this is done through the precise centre of each pixel, so there is a one-to-one mapping between display pixels and rendered pixels. However, that leads to very bad sampling error (aliasing), so typically you instead take lots of samples (which throws up more problems of sampling bias which I'll ignore for now). Now you have to decide how to map those samples to screen pixels. This is where the phrase "a pixel is a window" comes from. If the point sample falls within the current pixel boundaries, it's a contributor to its value. Pixels are generally considered to be rectangular windows, because that maps nicely to the way they are laid out in a grid. In particular, when displayed on-screen, each pixel represents a rectangular area of the screen. However, that's independent of how the image data was quantised originally. The point the paper seems to be making is that this rectangular mapping isn't very good, and overlapping circular windows are better. That's debatable and it depends what you're doing. With this model some point samples are contributors to more than one screen pixel, but on the upside, contributors to each pixel are all within an equal distance from the midpoint. A square allows samples that are further away on the diagonals, which is not right. It's correct for display, but not for manipulation. Either way, a pixel is still a window. You can choose different shapes for that window, with different trade-offs, but a pixel still describes the average of samples within the region it covers, rather than any single sample. That unfortunately is then quantised into a single sample for the pixel, ostensibly at its midpoint. Of course, the bigger problem is that quantisation step. If each point sample were stored as a vector, you could have lots of randomly positioned samples and re-quantize them into any shape pixels you like, which might be useful for something like the barrel-distorted projection used in the Oculus Rift, where it would be advantageous for every pixel to have a unique size and shape depending on how close it is to the centre (or think of a spherical projection where it also makes sense for pixels to be non-square). Unfortunately, that's not the way pixels are represented in any image formats we have, which is the particular view of pixels the paper is concerned with.
fsloth 12y ago
A little bit of background for the paper and a brief summary might be beneficial: The author is Alvy Ray Smith. He's one of the pioneers of computer graphics (http://en.wikipedia.org1/wiki/Alvy_Ray_Smith http://en.wikipedia.org1/wiki/Alvy_Ray_Smith). As the author states in the introduction: "If you find yourself thinking that a pixel is a little square, please read this paper." To get from the point to the squares -interpretation you need to read a little bit forwards: "Sampling Theorem tells us that we can reconstruct a continuous entity from such a discrete entity using an appropriate reconstruction filter" This might be proper background reading before or after reading the paper: http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_samplin...
- k__ 12y ago
  The problem most people have here is probably, they don't sample anything.
  - tacotime 12y ago
    Yeah at least here in America, bet you can't develop just one metabolic condition nudge nudge am I right?
- swah 12y ago
  "He is cofounder, with Edwin Catmull, of the animation studio Pixar, financed by Steve Jobs." Nice thing to have on your resume...
j2kun 12y ago
Very interesting article. He does say some strange things, though, like "the device integrates over [blah]." Is there some natural phenomenon allowing it to compute the integral without resorting to approximating it by essentially using little squares? (perhaps the little squares are not directly related to squares on the image surface, but it's not entirely clear to me that they aren't just a few steps removed from that)
- fsloth 12y ago
  We are discussing the mathematical model of the system that has been found to be the most proper to deal with the practical problems that arise about image processing. When he says "the device integrates over..." what he means is that we can model the display device as a reconstruction filter integrating over the point sample lattice. Remember, we are discussing the mathematical model of image processing. In the purely physical model the pixels in the memory are probably a linear array of n-bit numbers, that drive voltages over the illuminants of the display. However, the "linear array of n-bit numbers in a framebuffer" is a really awkward model to start discussing e.g. image processing algorithms. The point sample model, as the author tries to persuade, provides a simple formalism for a lot of practical problems and is also, from the point of view of sampling theory, the correct model.
  - wtallis 12y ago
    This paper only mentions integration in the context of sensors, not displays. So it's pretty straightforward: the scanner/camera/eyeball is integrating photons over time and over the surface of the sensor. None of those are inherently discretized by little squares, because even the photoreceptors on a chip aren't perfect rectangles and are subject to optical effects like things being imperfectly focused.
- bcoates 12y ago
  In a camera, light bounces from a light source, off an object, into receptor, with at least one aperture in the path to ensure that light arriving from different directions reaches different sensors. These sensors collect light from some area over time which can be recovered as information (chemically or electrically). Light passing through an aperture is subject to diffraction (which rules out a perfect square beam of light) and possibly other distortions, and the sensor may or may not be a perfect, uniform square, but in any case it's all equivalent to a perfect point in space that receives undistorted, refractionless light then applies a filter and integrates it over time and area. The underlying physical phenomenon that allows integration to happen is the ability for a film grain or ccd cell to count up incoming photons. These ideal sensors may be arranged on a grid, but the filter that maps source light to sensor value won't look anything like non-overlapping square boxes.
bajsejohannes 12y ago
I can see why you should look at pixels as point samples when resampling a picture. However, if I'm sampling, say, a fractal to make a picture, I can't see a better way than to sample in the square (assuming it's grayscale) that each pixel provides. Should I do a gaussian sample instead?
- fsloth 12y ago
  "Should I do a gaussian sample instead?" There are a ton of sampling filters. The main metric of quality for them is how they look (or rather, how the image looks after being processed by them). So if the square looks nice, there's absolutely nothing wrong with it. However, if you feel like geeking out, as resampling filters go, the last time I looked the "best" by some empirical estimate was the Mitchell-Netravali filter (they did a "scientific" sampling of asking a bunch of computer graphics people what they thought looked best). There's a paper http://www.cs.utexas.edu/~fussell/courses/cs384g-fall2013/lectures/mitchell/Mitchell.pdf http://www.cs.utexas.edu/~fussell/courses/cs384g-fall2013/le... and an entry in the german (no english page!) wikipedia http://de.wikipedia.org/wiki/Mitchell-Netravali-Filter http://de.wikipedia.org/wiki/Mitchell-Netravali-Filter Gives the brief definition
  - jacobolus 12y ago
    The best resource about this I know of on the web is: http://www.imagemagick.org/Usage/filter/ http://www.imagemagick.org/Usage/filter/ and also see http://www.imagemagick.org/Usage/filter/nicolas/ http://www.imagemagick.org/Usage/filter/nicolas/ The Mitchell–Netravali paper is definitely not the last word on the subject.
    - bajsejohannes 12y ago
      Thanks! (And thanks for the other replies as well) This resource is the most interesting to me. As I got a lot of answers about zooming and resampling, I'll try to restate my original question: What I have is some mechanism to sample something as much and as detailed as I want. This is why I mentioned a fractal. I don't want to zoom or resample, just find the optimal way to sample the original data to construct, say, a 500x500 pixel image. Up until I read this article, I would just sample randomly in the [0.0, 0.0]-[1.0, 1.0] space and avergage out the value. If there's a better way, I'd love to learn it. The best answer might just be "main metric of quality for them is how they look" from fsloth.
arielby 12y ago
LCD and LED displays are made of little coloured rectangles, and many programs are writing for them.
- fsloth 12y ago
  We get to the display device after the display pipeline has transformed the source image. The author is discussing the best way to think about and model the source image. The best model for the source image is a lattice of point samples if you want to manipulate it in any way.
agumonkey 12y ago
I knew the name was familiar http://alvyray.com/Pixar/PixarHistoryRevisited.htm http://alvyray.com/Pixar/PixarHistoryRevisited.htm
enthdegree 12y ago
Great read! Very similar ideas apply for image capture: http://enthdegree.tumblr.com/post/102480374444/a-model-for-image-capture http://enthdegree.tumblr.com/post/102480374444/a-model-for-i... Under this model, a pixel can be interpreted as both a point-sample and a square.
pixel 12y ago
I can verify that I am not a little square.
Camillo 12y ago
Nowadays, a digitized image comes with a color profile that describes the source device's color model. This is used to maintain accurate color reproduction as the image is processed and output again. Shouldn't there also be a "sampling profile" that describes what sort of filter was used when the image was originally sampled? That would help programs choose the best resampling method when processing the image.
marze 12y ago
Yawn. Steve Jobs was right to fire this guy, you don't need to anything more than this paper to see that.
kejaed 12y ago
A link to the last time this was discussed on HN: https://news.ycombinator.com/item?id=1472175 https://news.ycombinator.com/item?id=1472175

118 points by rinesh 12y ago

Stopped reading at "A pixel is a point sample." No, it's not. We wouldn't have sub-pixel font rendering and hundreds of anti-aliasing filter modes for the graphics cards. The pixel is an /area/ element of your image. And yes, many times, it's a square.

RubyPinch 12y ago
It mentioned colors right after that statement. edit: I do feel we are all smart enough to realize that the circle-vs-square, when considering a single channel, isn't affected much when colors are then considered offsets for each color are not unimaginable
angersock 12y ago
So, you made it to the middle of the second page? Anti-aliasing is a direct result of the fact that pixels are point-samples--the idea is to combine multiple samples and weight them so that the collection of point samples (in the form of a frame buffer) still represents a perceptually-reasonable form of the image.
- davrosthedalek 12y ago
  Sorry, but you got it backwards. Exactly because pixels are /not/ point samples, but have an area, point sampling does not work, and you need anti-aliasing. In the limit of infinite resolution/area of the pixels gets infinitely small, point-sampling would be enough. You say it yourself: Since solving the integration over the pixel area is generally hard, we approximate the integration with multiple point samples.
  - angersock 12y ago
    Hm, I think I see the point you're getting at. It's still a good paper. :)
gumby 12y ago
> Stopped reading at "A pixel is a point sample." > ...yes, many times, it's a square. While "appeal to authority" is generally a bad way to end a discussion, I will start by pointing out that this is Alvy Ray Smith writing, and he was writing in 1995. So while you could end up disagreeing with him (though I hope you wouldn't), you might consider why he was writing. Your comment shows that this paper is just as applicable today. (since you stopped on page two you never saw the several examples of why this matters in the article, including one on scanning and one on pre-rendering geometric computation). Those graphics cards you refer to reflect Smith's influence on their design.You have sub pixel rendering today because pixels aren't squares. This was computationally prohibitive in 1995. The TL;DR of this brief paper is "IO is really really hard, and though accepting shortcuts and defaults only kinda works and here's why." You benefit from a lot of work done in the last 20 years to improve the quality of the libraries and hardware available to you, but still, "photo-realistic" graphics aren't. By the way, rendered pixels weren't squares in those days (they were not-always-circular dots on a screen which itself wasn't flat) and they aren't on today's flat LCDs (you can think of them as rectangular, though the color elements do not present uniformly though the area) and they sure as hell aren't on any device with lenses or mirrors (including projectors and headsets)
- davrosthedalek 12y ago
  I realize that sub-pixel rendering is a misleading term, there are two concepts: a) Making use of fact that a physical pixel is a composite of multiple, disjunct areas of single-colored elements. b) Rendering objects with higher-than-pixel-density resolution, for example lines which appear thinner than a pixel. /Both/ relies on the fact that a pixel has an area, i.e. it is not a point, which is the point I was trying to make. A pixel on LCDs is indeed square most of the time, the subpixel elements are not. CRTs, of course, different story. All the "problems" he describes actually arise from the misconception that a pixel is a /point/ sample. It is an area sample, and if you treat it as such, the rest follows.
- gaze 12y ago
  I think everyone in signal processing writes the same rant against zero-order-hold at some point. The Xiph people did (http://xiph.org/video/ http://xiph.org/video/) and so has this guy.
corysama 12y ago
What is behind the pixel is an area. But, the pixel itself is a pinhole viewport into that area. This article is about resampling and reconstructing images that are already encoded as pixels (texels would be more accurate, but I don't know if that term was around back in 93). So, when zooming in on a pixelated image, how do we fill in the gaps between the pinholes? Declaring pixels to be square areas means that larger views of a given image really should be composed of larger, hard-edged squares for each source pixel. I.e point sampling is truth. The reality is that what's really between the pixel centerpoints is unknown --lost forever or never calculated at all. To fill in the gaps we can only make intelligent guesses. Therefore: sampling theory.
akira2501 12y ago
> And yes, many times, it's a square. Are you sure? In my experience, it depends on the context, but the pixels themselves are _not_ always of equal height:width ratio. http://en.wikipedia.org/wiki/Pixel_aspect_ratio http://en.wikipedia.org/wiki/Pixel_aspect_ratio
- davrosthedalek 12y ago
  not always = many times?
  - akira2501 12y ago
    "And yes, many times, ... " I took it to mean, "yes, many times over." As in, without a doubt, all pixels are square. My error.

dsjoerg 12y ago

I skimmed it and failed to find an explanation of what problems are caused by thinking of a pixel as a little square. Anyone?

Roboprog 12y ago
Indeed. As far as I could tell, the article was an exercise in "TMTOWTDI" - sometimes squares, sometimes subdivided squares (trinitron), sometimes little circles with gaps or overlaps, sometimes linear interpolation of a sample. OK, it's an abstraction, but the article didn't seem to cover the strength of weakness of various implementations.
- Roboprog 12y ago
  And sometimes, it was an asterisk in a 60 x 80 grid (with a messed up aspect ratio) on a piece of printer paper, back in the stone age :-)
- fsloth 12y ago
  I think there is a slight overload of terminology here. The samples in the source image, and the tiny light sources that vary in intensity and are driven by a voltage after the information in the bits in the source image has traveled through the display pipeline are two different things. The author discusses the best way to think about the source image. It will get really awkward if we want to discuss imaging algorithms and have to define "my pixel is a square... if it's a bunch of weird shapes the transform is this... etc." It's easiest from this point of view to think of the source image as a lattice of point samples and figure out what the sampling function (our physical pixels) do to the image after we have figured out the math on the point sample source.
fl0wenol 12y ago
Programmers would implicitly use a box filter when sampling from an underlying process (like 3d rendering, image scaling, scanning) into screen coordinates. It's one thing when writing a compositing window manager or blitting sprites to the screen; it's another thing entirely when doing any kind of image processing to make that assumption. It's similar to assuming that RGB (0-255) is perceptually linear and consistent among monitors (not understanding monitor profiles and color spaces)

sehugg 12y ago

Since this is 1995, pixels didn't map 1-to-1 to phosphors on the CRT. Now with LED and LCD displays they do. Not that this invalidates the paper, but it certainly adds some context.

fit2rule 12y ago
Um, sorry but .. no. They do not just magically 'map 1-to-1', and that is anyway not the point. The point is: a pixel is not a point. It is a window.
- gaze 12y ago
  Other way around. A pixel IS a point. It's a sample.
  - robert_tweed 12y ago
    That's the definition used in the paper. The other definition is that a pixel is in fact, a square window containing many samples that fall within that region and rendered as their average. This is known as antialiasing.
    - rspeer 12y ago
      The article illustrates antialiasing, but chooses to use sinc-based circular windows, not square windows. Am I right to think that you'd only use square windows in the 3D-rendered graphics case, where you've already assumed the sub-pixels are square for efficiency?
      - robert_tweed 12y ago
        The main point is that there are two conflicting views of what a "pixel" is. One is the representation of the samples and the other is what is actually displayed on-screen. Take for instance a ray tracer. Each ray is a sample and these are cast from the focal point, through the view frustum, and the colour is calculated where it intersects an object in the scene. In the naive case, this is done through the precise centre of each pixel, so there is a one-to-one mapping between display pixels and rendered pixels. However, that leads to very bad sampling error (aliasing), so typically you instead take lots of samples (which throws up more problems of sampling bias which I'll ignore for now). Now you have to decide how to map those samples to screen pixels. This is where the phrase "a pixel is a window" comes from. If the point sample falls within the current pixel boundaries, it's a contributor to its value. Pixels are generally considered to be rectangular windows, because that maps nicely to the way they are laid out in a grid. In particular, when displayed on-screen, each pixel represents a rectangular area of the screen. However, that's independent of how the image data was quantised originally. The point the paper seems to be making is that this rectangular mapping isn't very good, and overlapping circular windows are better. That's debatable and it depends what you're doing. With this model some point samples are contributors to more than one screen pixel, but on the upside, contributors to each pixel are all within an equal distance from the midpoint. A square allows samples that are further away on the diagonals, which is not right. It's correct for display, but not for manipulation. Either way, a pixel is still a window. You can choose different shapes for that window, with different trade-offs, but a pixel still describes the average of samples within the region it covers, rather than any single sample. That unfortunately is then quantised into a single sample for the pixel, ostensibly at its midpoint. Of course, the bigger problem is that quantisation step. If each point sample were stored as a vector, you could have lots of randomly positioned samples and re-quantize them into any shape pixels you like, which might be useful for something like the barrel-distorted projection used in the Oculus Rift, where it would be advantageous for every pixel to have a unique size and shape depending on how close it is to the centre (or think of a spherical projection where it also makes sense for pixels to be non-square). Unfortunately, that's not the way pixels are represented in any image formats we have, which is the particular view of pixels the paper is concerned with.

fsloth 12y ago

A little bit of background for the paper and a brief summary might be beneficial: The author is Alvy Ray Smith. He's one of the pioneers of computer graphics (http://en.wikipedia.org1/wiki/Alvy_Ray_Smith http://en.wikipedia.org1/wiki/Alvy_Ray_Smith). As the author states in the introduction: "If you find yourself thinking that a pixel is a little square, please read this paper." To get from the point to the squares -interpretation you need to read a little bit forwards: "Sampling Theorem tells us that we can reconstruct a continuous entity from such a discrete entity using an appropriate reconstruction filter" This might be proper background reading before or after reading the paper: http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem http://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_samplin...

k__ 12y ago
The problem most people have here is probably, they don't sample anything.
- tacotime 12y ago
  Yeah at least here in America, bet you can't develop just one metabolic condition nudge nudge am I right?
swah 12y ago
"He is cofounder, with Edwin Catmull, of the animation studio Pixar, financed by Steve Jobs." Nice thing to have on your resume...

j2kun 12y ago

Very interesting article. He does say some strange things, though, like "the device integrates over [blah]." Is there some natural phenomenon allowing it to compute the integral without resorting to approximating it by essentially using little squares? (perhaps the little squares are not directly related to squares on the image surface, but it's not entirely clear to me that they aren't just a few steps removed from that)

fsloth 12y ago
We are discussing the mathematical model of the system that has been found to be the most proper to deal with the practical problems that arise about image processing. When he says "the device integrates over..." what he means is that we can model the display device as a reconstruction filter integrating over the point sample lattice. Remember, we are discussing the mathematical model of image processing. In the purely physical model the pixels in the memory are probably a linear array of n-bit numbers, that drive voltages over the illuminants of the display. However, the "linear array of n-bit numbers in a framebuffer" is a really awkward model to start discussing e.g. image processing algorithms. The point sample model, as the author tries to persuade, provides a simple formalism for a lot of practical problems and is also, from the point of view of sampling theory, the correct model.
- wtallis 12y ago
  This paper only mentions integration in the context of sensors, not displays. So it's pretty straightforward: the scanner/camera/eyeball is integrating photons over time and over the surface of the sensor. None of those are inherently discretized by little squares, because even the photoreceptors on a chip aren't perfect rectangles and are subject to optical effects like things being imperfectly focused.
bcoates 12y ago
In a camera, light bounces from a light source, off an object, into receptor, with at least one aperture in the path to ensure that light arriving from different directions reaches different sensors. These sensors collect light from some area over time which can be recovered as information (chemically or electrically). Light passing through an aperture is subject to diffraction (which rules out a perfect square beam of light) and possibly other distortions, and the sensor may or may not be a perfect, uniform square, but in any case it's all equivalent to a perfect point in space that receives undistorted, refractionless light then applies a filter and integrates it over time and area. The underlying physical phenomenon that allows integration to happen is the ability for a film grain or ccd cell to count up incoming photons. These ideal sensors may be arranged on a grid, but the filter that maps source light to sensor value won't look anything like non-overlapping square boxes.

bajsejohannes 12y ago

I can see why you should look at pixels as point samples when resampling a picture. However, if I'm sampling, say, a fractal to make a picture, I can't see a better way than to sample in the square (assuming it's grayscale) that each pixel provides. Should I do a gaussian sample instead?

fsloth 12y ago
"Should I do a gaussian sample instead?" There are a ton of sampling filters. The main metric of quality for them is how they look (or rather, how the image looks after being processed by them). So if the square looks nice, there's absolutely nothing wrong with it. However, if you feel like geeking out, as resampling filters go, the last time I looked the "best" by some empirical estimate was the Mitchell-Netravali filter (they did a "scientific" sampling of asking a bunch of computer graphics people what they thought looked best). There's a paper http://www.cs.utexas.edu/~fussell/courses/cs384g-fall2013/lectures/mitchell/Mitchell.pdf http://www.cs.utexas.edu/~fussell/courses/cs384g-fall2013/le... and an entry in the german (no english page!) wikipedia http://de.wikipedia.org/wiki/Mitchell-Netravali-Filter http://de.wikipedia.org/wiki/Mitchell-Netravali-Filter Gives the brief definition
- jacobolus 12y ago
  The best resource about this I know of on the web is: http://www.imagemagick.org/Usage/filter/ http://www.imagemagick.org/Usage/filter/ and also see http://www.imagemagick.org/Usage/filter/nicolas/ http://www.imagemagick.org/Usage/filter/nicolas/ The Mitchell–Netravali paper is definitely not the last word on the subject.
  - bajsejohannes 12y ago
    Thanks! (And thanks for the other replies as well) This resource is the most interesting to me. As I got a lot of answers about zooming and resampling, I'll try to restate my original question: What I have is some mechanism to sample something as much and as detailed as I want. This is why I mentioned a fractal. I don't want to zoom or resample, just find the optimal way to sample the original data to construct, say, a 500x500 pixel image. Up until I read this article, I would just sample randomly in the [0.0, 0.0]-[1.0, 1.0] space and avergage out the value. If there's a better way, I'd love to learn it. The best answer might just be "main metric of quality for them is how they look" from fsloth.

arielby 12y ago

LCD and LED displays are made of little coloured rectangles, and many programs are writing for them.

fsloth 12y ago
We get to the display device after the display pipeline has transformed the source image. The author is discussing the best way to think about and model the source image. The best model for the source image is a lattice of point samples if you want to manipulate it in any way.

agumonkey 12y ago

I knew the name was familiar http://alvyray.com/Pixar/PixarHistoryRevisited.htm http://alvyray.com/Pixar/PixarHistoryRevisited.htm

enthdegree 12y ago

Great read! Very similar ideas apply for image capture: http://enthdegree.tumblr.com/post/102480374444/a-model-for-image-capture http://enthdegree.tumblr.com/post/102480374444/a-model-for-i... Under this model, a pixel can be interpreted as both a point-sample and a square.

pixel 12y ago

I can verify that I am not a little square.

Camillo 12y ago

Nowadays, a digitized image comes with a color profile that describes the source device's color model. This is used to maintain accurate color reproduction as the image is processed and output again. Shouldn't there also be a "sampling profile" that describes what sort of filter was used when the image was originally sampled? That would help programs choose the best resampling method when processing the image.

marze 12y ago

Yawn. Steve Jobs was right to fire this guy, you don't need to anything more than this paper to see that.

kejaed 12y ago

A link to the last time this was discussed on HN: https://news.ycombinator.com/item?id=1472175 https://news.ycombinator.com/item?id=1472175