6 ms·

A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube) [1] [1] http://alvyray.com/M

by danbruc 10y ago

A Pixel Is Not A Little Square, A Pixel Is Not A Little Square, A Pixel Is Not A Little Square! (And a Voxel is Not a Little Cube) [1] [1] http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf http://alvyray.com/Memos/CG/Microsoft/6_pixel.pdf

Udo 10y ago
This mantra was also the reason why MagicaVoxel, the "voxel" editor mentioned in the article was received relatively poorly on HN about a year ago: https://news.ycombinator.com/item?id=10953918 https://news.ycombinator.com/item?id=10953918 Yes, so technically we should probably call the modern rendering technique of pixels and voxels block-based rendering. I would say though it's fair game to call the underlying data "pixels" and "voxels" - in most retro-style games these assets are indeed stored internally at that level of abstraction. The issue now becomes how to render that data on the screen, and blocks are still a way to make this data look good. While the block is not a pixel or voxel in the strictest sense, I do think it's a reasonable linguistic shortcut to use that word in this context, and it's not unprecedented to use the description of an abstract thing interchangeably for its more concrete representation. In my opinion, there is nothing to freak out about, and I don't see why this article specifically prompts such a needlessly absolutist response.
- colanderman 10y ago
  I personally find "block-based" rendering of 8-bit era pixel art quite ugly. There's way too much high-frequency content – too many edges – making it very visually busy. As others have said, 8-bit art was made in a context where the "pixels" were viewed through an analogue filter, that being an NTSC display. My understanding is that to a first approximation, such a display (i.e. the electron gun + shadow/aperture mask) acts as a spatial lowpass filter with a resolution of 640x480 pixels, with maybe a 1-pixel-wide horizontal "smearing" effect due to the horizontal movement of the gun. Of course various console + display combinations deviate wildly from this ideal, but the lowpass filtering is the basis for understanding how 8-bit art was expected to be viewed.
  - Udo 10y ago
    There is no question about the heritage of 8bit-style art, and whether you find modern renditions ugly or not is not really a concern here. But I would like to point out as someone who grew up in the eighties, people referred to those little atomic picture cells rendered on the display as "pixels", not as "my monitor's representation of a pixel". It's also worth mentioning that for most of us today, graphics tastes have changed quite a lot, so emulating the original blur and smear of an NTSC display is perceived as far less of a generally acceptable aesthetic solution than rendering rectangular blocks is. Of course, opinions differ, but you can totally vote with your wallet on that. You're talking about how art is designed with an expectation about the manner in which it is later consumed. To the degree this is even a realistic expectation, you would have to make the same concession to today's pixel artists who are creating works loosely based on an 8bit heritage, but which are often clearly and intentionally distanced from that heritage. You might personally object to viewing original games from the 8bit era on modern systems at all on the grounds of authenticity and distortion of artistic intent, but that's not an objection you can raise against modern pixel artists and their working premises. Overall, I object to this notion that, say, the average 80s CRT pixel representation is somehow the canonical rendering of the ur-pixel and that we should strive to somehow emulate that. I think we should instead do whatever works for the modern context. You might counter that blocks don't meet this goal for your tastes, or that we should stop using recognizably blocky pixels altogether, but again that's a personal preference. The actual argument above was not about personal preference at all, it was about what can rightfully be referred to as a pixel and what can't (and I do maintain that in this context linguistic pedantry does not yield any meaningful benefits).
    - smallnamespace 10y ago
      > I object to this notion that, say, the average 80s CRT pixel representation is somehow the _canonical_ rendering of the ur-pixel You'd have to imagine it's closer to the original artist's end goals though. The guy who drew Mario probably didn't intend for us to see a blocky representation of an Italian plumber, they just wanted us to see a plumber. OTOH, Seurat was consciously making a point that you can represent a person with nothing but colored dots: https://en.wikipedia.org/wiki/Pointillism#/media/File:Seurat-La_Parade_detail.jpg https://en.wikipedia.org/wiki/Pointillism#/media/File:Seurat... > people referred to those little atomic picture cells rendered on the display as "_pixels_" 'These aren't the pixels you're looking for'. As the article points out, the pixels on a CRT do not have a 1-to-1 correspondence with the pixels in RAM. If you understand a pixel as a sample, then the mental picture becomes clearer: RAM pixels -> video card DAC converts to continuous signal -> CRT resamples again into monitor pixels -> your eye/brain reconverts into a continuous image
- zamalek 10y ago
  > block-based rendering. I just refer to them as "boxels."
barrkel 10y ago
I know where you're coming from, but really this perspective mostly applies when you're rasterising some underlying ideal. When art is composed out of the rendered elements directly, we still need a word for the square thing we see on screen. Pixel is the chosen word. This means it has two meanings, despite your reference.
- smallnamespace 10y ago
  > When art is composed out of the rendered elements directly I wonder though, would a lot of 'pixel art' actually look better with a different filter instead?
  - derefr 10y ago
    One increasingly-popular point of view is that console-game pixel art was intended to look like it does when sent through an analogue (e.g. composite) video cable to the average CRT television screen; and that artists relied on various "blending"-like effects that were inherent in this process, making the art assets look "off" when displayed crisply on an LCD. See http://www.tested.com/tech/gaming/2982-a-link-to-the-past-how-to-add-crt-filters-to-16-bit-games-on-pc/ http://www.tested.com/tech/gaming/2982-a-link-to-the-past-ho... for examples.
    - smallnamespace 10y ago
      In the context of GP's article, it's fun reading through the comments on the page: I actually prefer the crisp pixels of emulated 8 and 16 bit games, I have no desire to emulate the crappy monitors we used to use to play said games. - lordofultima I actually enjoy gigantic, crystal clear pixels. - rateoforange You'd have to imagine that actual graphics artists during the 8-bit era viewed low resolution as a limitation to overcome, not a purposeful feature of their artwork.
      - Ericson2314 10y ago
        I kinda enjoy the anachronism too, but http://d2rormqr1qwzpz.cloudfront.net/uploads/0/5408/30085-ffvi_ntsc.png http://d2rormqr1qwzpz.cloudfront.net/uploads/0/5408/30085-ff... from the article really does look great. All this reminds me of tge performance practice of old music.
smallnamespace 10y ago
This brings up a good reason why voxels often look so blocky and unappealing - 'square' pixels are fairly reasonable approximations to the underlying image, plus your pixel resolution can match exactly to the display, so this makes rasterization a no-op (and you don't even see the squares). But once you're in 3 dimensions, triangles do a much better job of approximating arbitrary shapes (also, hardware deals with triangles), plus the voxel dimensions have no relationship to your display. You're essentially picking a very clunky filter that obscures what you're trying to represent, out of some misguided analogy to 2D pixels. Just to be clear, cubic voxels can be great as a stylistic choice, but are inferior to triangles as a general way to describe 3D scenes.
- colanderman 10y ago
  To be pedantic, a 1px square does NOT properly rasterize as a single pixel. If you apply a low-pass filter (as you should, to avoid aliasing) – even an ideal one – those little squares blur and "ring" across several neighboring pixels. Rather, it is only an infinitesimally small point (an "impulse" or Dirac delta) which, when grid-aligned and ideally filtered, can rasterize as a single pixel.
  - deleted 10y ago
    [deleted]
  - Dylan16807 10y ago
    That just shows why you shouldn't be applying filters after rasterizing. If you want to avoid aliasing, you should be supersampling or calculating exact coverage before you reduce everything to the final resolution. In which case the tiny square survives perfectly.
    - colanderman 10y ago
      I never said filter after rasterizing. Pixel-sized squares have the same issue if you filter before rasterizing. The only reason pixel-sized square survive supersampling intact is because supersampling uses an imperfect low-pass filter (block-based averaging). Here is a good example showing the shortcoming of straight supersampling: https://people.cs.clemson.edu/~tadavis/cs809/aa/aliasing5.png https://people.cs.clemson.edu/~tadavis/cs809/aa/aliasing5.pn... You NEED to apply a true low-pass filter before rasterizing to completely eliminate Moiré patterns. Whether a round of supersampling occurs before this filtering is irrelevant. And pixel-sized squares don't survive low-pass filters intact, see https://en.wikipedia.org/wiki/Gibbs_phenomenon#Signal_processing_explanation https://en.wikipedia.org/wiki/Gibbs_phenomenon#Signal_proces...
cornstalks 10y ago
The fact that pixels are point samples and have no area is something I feel an inexplicable psychological aversion for, despite it making sense. However, Monty's "Digital Show and Tell" video[1] helps me feel more comfortable about it. His explanation of discrete audio samples being point samples[2] (and thus should be drawn as such, rather than as a continuous stair step which I find akin to pixels) was invaluable. I generally knew everything he was talking about, but he was able to explain and present it such that I felt like I understood it more deeply than I ever had before. It's definitely worth a watch. [1]: https://xiph.org/video/vid2.shtml https://xiph.org/video/vid2.shtml [2]: https://youtu.be/cIQ9IXSUzuM?t=5m59s https://youtu.be/cIQ9IXSUzuM?t=5m59s
paulddraper 10y ago
Well, sort of. There is a fair amount of graphics whose good looks are based around the fact that pixels are little squares. Like orthogonal lines.
Dylan16807 10y ago
Odd that it calls a reconstruction that doesn't blur "worst case" and "abominable". The smaller the details in an image, the less I want any filtering applied when it's sized up. (Or, you could say, the smaller I want filter regions to be.)
- astrange 10y ago
  In this theory the reconstruction filters need to work as both downscaling and upscaling filters. A box filter looks good for integer-ratio upscaling, looks bad for non-integer upscaling, and looks terrible for downscaling. What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale. > Or, you could say, the smaller I want filter regions to be. And that's the opposite of you get from linear filters. There's a resampling filter called sinc that's "perfect" - it reproduces every frequency in the input signal - but it looks terrible for images because it tries to reproduce pixels based on every other pixel in the image. So you get ringing all over the picture instead of sharpness.
  - Dylan16807 10y ago
    I am largely focusing on upscaling, because downscaling is easy to get close just by averaging pixels. >What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale. When I'm looking at really tiny details even those can cause problems, and I just want pixel blobs. >And that's the opposite of you get from linear filters. Huh? I'm talking about the footprint of the filter, like the diagrams in the pdf. The box filter and Figure 1 filter sample one pixel, so they have the smallest region, and sinc is the largest possible. What's opposite?
    - astrange 10y ago
      Oh, I read that as "filtered regions". Like if you blow up an image with some noise in it, the noise is "infinitely small", so it should remain the same size and not get bigger with the image content.
erikb 10y ago
Well, a model is not the reality, but a useful abstraction. Considering that does it really hurt to wrongfully think of pixels as squares? It hasn't made any troubles to me yet, at least.
lloeki 10y ago
Indeed, Marching Cubes and similar algorithms are voxels and drive home the "not a little cube" point.
dahart 10y ago
This is a classic, and a good point, and comes up on HN every time we talk about voxels. But I think it's worth considering that: - "Pixel" is an overloaded term. "Pixel" is a little square, sometimes, because common law - it's being used that way. - This is an article about art, aimed at artists, and if an artist intends to create a blocky image in two or three dimensions, then what they're doing is "correct" regardless of the terminology they use. - The pictures are really pretty, IMO. I realize some people don't like the block look, but I'm not alone when I say I love it and it's inspiring. The explanations in the article were nicely done. - The article does cover sampling. Just not filtering. - When Alvy Ray wrote his paper, the world was made of blurry CRTs. Now we have really crisp digital LCDs everywhere, and more now than then, pixels on the screen are little squares. - "pixels" even in the days of CRTs have always meant little squares to a lot of people. Video game art benefitted from a little blur, but tons of artists and programmers have always thought of pixels as squares, and for the most part it's a working analogy. This is why it's okay to call a pixel a little square, and overly rigid to claim it's not one. (And I think Alvy Ray was being funny and making an important point about math and graphics, for a siggraph audience, not pedantic about what words artists are using.)
- fenomas 10y ago
  I would extend this, and say that all terms are overloaded - so before taking issue with a definition it's always worth considering whether you're arguing that one definition is Wrong and the other is Right (which is reductionist and silly), or arguing that one definition is more useful for the context under discussion (which is often useful).
infogulch 10y ago
Honest question: If a pixel is a point sample, why do all the down-sampling algorithms that look good involve some form of averaging the values of neighboring pixels? Wouldn't it be more accurate to pick one sample and throw away the rest?
- jstelly 10y ago
  This is a fairly deep topic so your question has a really long & detailed answer. The simple answer is "aliasing." The concept is that picking one sample for each pixel makes an aggregate image that could be confused with other images, whereas a good downsampling algorithm only represents a (lower frequency, bandlimited) version of the original image.
- dahart 10y ago
  It's a good question. The name for that - pick one sample, toss the rest - is "nearest neighbor sampling". Whether it's accurate depends on what you are trying to achieve, but the short answer is that in general it is not the most accurate, and it can cause a lot of problems. But your intuition isn't wrong. Nearest neighbor can be a bit more "crisp" feeling on certain images, especially natural photos. A lot of photos have enough blurriness in the 1-3 pixels range that nearest neighbor sampling is a good choice if you're just resizing an image by a factor of 2. But to see why nearest neighbor isn't the best in general, imagine down-sampling a checkboard and what happens as individual checkers become smaller than a pixel. You'll get moire patterns if you don't filter. You'd also expect pixels to turn grey as multiple checkers land there, and maybe not stick to only black or white. Interestingly, some "sharpening" and up-sampling algorithms do the opposite - they subtract the value of surrounding pixels, rather than add. Those are assuming that some averaging already took place, and trying to un-average, if you will, to restore the original point sample.
- munchbunny 10y ago
  The short answer is that our eyes and brains don't work that way. The long answer is... really long.
pierrebai 10y ago
While true, this is a category of stances that I quite enjoy poking holes into. I usually bring this up when people deride folk wisdom or trying to show how common sense is incorrect. The basic problem with being petty is that, while you're right on one level, you're wrong on the global scale. In this case, that pixel are square or not is irrelevant. Do you look at your monitor straight on? Do you sit side-by-side with a friend playing a game? In that case, squares become trapezoid from your POV anyway. I'm going to bet you don't mind it, because our brain is very good are pretending perspective distortions don't exist. The same apply in many domains: color correction, for example. I use to write in printing and designers would be hung up on proof to verify colors. No matters that reader would read teh final product under varying lights anyway, making precisely 'correct' colors moot. In both cases, what is important is consistency. For prints, that all runs be the same. For monitors, well, as long as you look at a single monitor at a time, you won't notice distortion. (In a somewhat related anecdote, the piano my parents own is and always has been out of tune. I learned on it and for me, all other pianos tones were unsufferable. What is the right pitch is mostly learned, even though musicians would like us to believe there is but a single harmonious scale.)
tripzilch 10y ago
This is true for computational science applications. A voxel in an MRI-scan is indeed not a little cube, but a data-point. Same for sampling 2D images (e.g. photography). In modern pixel/voxel art, however, the whole point is that they are in fact little squares and cubes. Otherwise you don't get the aesthetic this style is aiming for. It should be noted for historical accuracy, that the origins of pixel art were displayed on relatively blurry CRT screens, and your pixels didn't actually look like little squares either, more like glowy blobs. That's a different point than the cited paper is making, however, which is about the sampling theorem where pixels and voxels are basically data points (that can be interpolated in various ways). On the other hand, when I played 16 colour 320x200 games like Commander Keen and Duke Nukem (2D) in the 90s on a VGA CRT monitor, the pixels definitely looked like little squares, not blobs. Edit: maybe I should mention the oldschool ANSI art as well (BBS/demoscene), which were made with various coloured ASCII-blocks, and there the "pixels" were in fact quite obviously little blocks (and sometimes other characters but that's more ASCII-art than ANSI).

by danbruc 10y ago

Udo 10y ago

This mantra was also the reason why MagicaVoxel, the "voxel" editor mentioned in the article was received relatively poorly on HN about a year ago: https://news.ycombinator.com/item?id=10953918 https://news.ycombinator.com/item?id=10953918 Yes, so technically we should probably call the modern rendering technique of pixels and voxels block-based rendering. I would say though it's fair game to call the underlying data "pixels" and "voxels" - in most retro-style games these assets are indeed stored internally at that level of abstraction. The issue now becomes how to render that data on the screen, and blocks are still a way to make this data look good. While the block is not a pixel or voxel in the strictest sense, I do think it's a reasonable linguistic shortcut to use that word in this context, and it's not unprecedented to use the description of an abstract thing interchangeably for its more concrete representation. In my opinion, there is nothing to freak out about, and I don't see why this article specifically prompts such a needlessly absolutist response.

colanderman 10y ago
I personally find "block-based" rendering of 8-bit era pixel art quite ugly. There's way too much high-frequency content – too many edges – making it very visually busy. As others have said, 8-bit art was made in a context where the "pixels" were viewed through an analogue filter, that being an NTSC display. My understanding is that to a first approximation, such a display (i.e. the electron gun + shadow/aperture mask) acts as a spatial lowpass filter with a resolution of 640x480 pixels, with maybe a 1-pixel-wide horizontal "smearing" effect due to the horizontal movement of the gun. Of course various console + display combinations deviate wildly from this ideal, but the lowpass filtering is the basis for understanding how 8-bit art was expected to be viewed.
- Udo 10y ago
  There is no question about the heritage of 8bit-style art, and whether you find modern renditions ugly or not is not really a concern here. But I would like to point out as someone who grew up in the eighties, people referred to those little atomic picture cells rendered on the display as "pixels", not as "my monitor's representation of a pixel". It's also worth mentioning that for most of us today, graphics tastes have changed quite a lot, so emulating the original blur and smear of an NTSC display is perceived as far less of a generally acceptable aesthetic solution than rendering rectangular blocks is. Of course, opinions differ, but you can totally vote with your wallet on that. You're talking about how art is designed with an expectation about the manner in which it is later consumed. To the degree this is even a realistic expectation, you would have to make the same concession to today's pixel artists who are creating works loosely based on an 8bit heritage, but which are often clearly and intentionally distanced from that heritage. You might personally object to viewing original games from the 8bit era on modern systems at all on the grounds of authenticity and distortion of artistic intent, but that's not an objection you can raise against modern pixel artists and their working premises. Overall, I object to this notion that, say, the average 80s CRT pixel representation is somehow the canonical rendering of the ur-pixel and that we should strive to somehow emulate that. I think we should instead do whatever works for the modern context. You might counter that blocks don't meet this goal for your tastes, or that we should stop using recognizably blocky pixels altogether, but again that's a personal preference. The actual argument above was not about personal preference at all, it was about what can rightfully be referred to as a pixel and what can't (and I do maintain that in this context linguistic pedantry does not yield any meaningful benefits).
  - smallnamespace 10y ago
    > I object to this notion that, say, the average 80s CRT pixel representation is somehow the _canonical_ rendering of the ur-pixel You'd have to imagine it's closer to the original artist's end goals though. The guy who drew Mario probably didn't intend for us to see a blocky representation of an Italian plumber, they just wanted us to see a plumber. OTOH, Seurat was consciously making a point that you can represent a person with nothing but colored dots: https://en.wikipedia.org/wiki/Pointillism#/media/File:Seurat-La_Parade_detail.jpg https://en.wikipedia.org/wiki/Pointillism#/media/File:Seurat... > people referred to those little atomic picture cells rendered on the display as "_pixels_" 'These aren't the pixels you're looking for'. As the article points out, the pixels on a CRT do not have a 1-to-1 correspondence with the pixels in RAM. If you understand a pixel as a sample, then the mental picture becomes clearer: RAM pixels -> video card DAC converts to continuous signal -> CRT resamples again into monitor pixels -> your eye/brain reconverts into a continuous image
zamalek 10y ago
> block-based rendering. I just refer to them as "boxels."

barrkel 10y ago

I know where you're coming from, but really this perspective mostly applies when you're rasterising some underlying ideal. When art is composed out of the rendered elements directly, we still need a word for the square thing we see on screen. Pixel is the chosen word. This means it has two meanings, despite your reference.

smallnamespace 10y ago
> When art is composed out of the rendered elements directly I wonder though, would a lot of 'pixel art' actually look better with a different filter instead?
- derefr 10y ago
  One increasingly-popular point of view is that console-game pixel art was intended to look like it does when sent through an analogue (e.g. composite) video cable to the average CRT television screen; and that artists relied on various "blending"-like effects that were inherent in this process, making the art assets look "off" when displayed crisply on an LCD. See http://www.tested.com/tech/gaming/2982-a-link-to-the-past-how-to-add-crt-filters-to-16-bit-games-on-pc/ http://www.tested.com/tech/gaming/2982-a-link-to-the-past-ho... for examples.
  - smallnamespace 10y ago
    In the context of GP's article, it's fun reading through the comments on the page: I actually prefer the crisp pixels of emulated 8 and 16 bit games, I have no desire to emulate the crappy monitors we used to use to play said games. - lordofultima I actually enjoy gigantic, crystal clear pixels. - rateoforange You'd have to imagine that actual graphics artists during the 8-bit era viewed low resolution as a limitation to overcome, not a purposeful feature of their artwork.
    - Ericson2314 10y ago
      I kinda enjoy the anachronism too, but http://d2rormqr1qwzpz.cloudfront.net/uploads/0/5408/30085-ffvi_ntsc.png http://d2rormqr1qwzpz.cloudfront.net/uploads/0/5408/30085-ff... from the article really does look great. All this reminds me of tge performance practice of old music.

smallnamespace 10y ago

This brings up a good reason why voxels often look so blocky and unappealing - 'square' pixels are fairly reasonable approximations to the underlying image, plus your pixel resolution can match exactly to the display, so this makes rasterization a no-op (and you don't even see the squares). But once you're in 3 dimensions, triangles do a much better job of approximating arbitrary shapes (also, hardware deals with triangles), plus the voxel dimensions have no relationship to your display. You're essentially picking a very clunky filter that obscures what you're trying to represent, out of some misguided analogy to 2D pixels. Just to be clear, cubic voxels can be great as a stylistic choice, but are inferior to triangles as a general way to describe 3D scenes.

colanderman 10y ago
To be pedantic, a 1px square does NOT properly rasterize as a single pixel. If you apply a low-pass filter (as you should, to avoid aliasing) – even an ideal one – those little squares blur and "ring" across several neighboring pixels. Rather, it is only an infinitesimally small point (an "impulse" or Dirac delta) which, when grid-aligned and ideally filtered, can rasterize as a single pixel.
- deleted 10y ago
  [deleted]
- Dylan16807 10y ago
  That just shows why you shouldn't be applying filters after rasterizing. If you want to avoid aliasing, you should be supersampling or calculating exact coverage before you reduce everything to the final resolution. In which case the tiny square survives perfectly.
  - colanderman 10y ago
    I never said filter after rasterizing. Pixel-sized squares have the same issue if you filter before rasterizing. The only reason pixel-sized square survive supersampling intact is because supersampling uses an imperfect low-pass filter (block-based averaging). Here is a good example showing the shortcoming of straight supersampling: https://people.cs.clemson.edu/~tadavis/cs809/aa/aliasing5.png https://people.cs.clemson.edu/~tadavis/cs809/aa/aliasing5.pn... You NEED to apply a true low-pass filter before rasterizing to completely eliminate Moiré patterns. Whether a round of supersampling occurs before this filtering is irrelevant. And pixel-sized squares don't survive low-pass filters intact, see https://en.wikipedia.org/wiki/Gibbs_phenomenon#Signal_processing_explanation https://en.wikipedia.org/wiki/Gibbs_phenomenon#Signal_proces...

cornstalks 10y ago

The fact that pixels are point samples and have no area is something I feel an inexplicable psychological aversion for, despite it making sense. However, Monty's "Digital Show and Tell" video[1] helps me feel more comfortable about it. His explanation of discrete audio samples being point samples[2] (and thus should be drawn as such, rather than as a continuous stair step which I find akin to pixels) was invaluable. I generally knew everything he was talking about, but he was able to explain and present it such that I felt like I understood it more deeply than I ever had before. It's definitely worth a watch. [1]: https://xiph.org/video/vid2.shtml https://xiph.org/video/vid2.shtml [2]: https://youtu.be/cIQ9IXSUzuM?t=5m59s https://youtu.be/cIQ9IXSUzuM?t=5m59s

paulddraper 10y ago

Well, sort of. There is a fair amount of graphics whose good looks are based around the fact that pixels are little squares. Like orthogonal lines.

Dylan16807 10y ago

Odd that it calls a reconstruction that doesn't blur "worst case" and "abominable". The smaller the details in an image, the less I want any filtering applied when it's sized up. (Or, you could say, the smaller I want filter regions to be.)

astrange 10y ago
In this theory the reconstruction filters need to work as both downscaling and upscaling filters. A box filter looks good for integer-ratio upscaling, looks bad for non-integer upscaling, and looks terrible for downscaling. What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale. > Or, you could say, the smaller I want filter regions to be. And that's the opposite of you get from linear filters. There's a resampling filter called sinc that's "perfect" - it reproduces every frequency in the input signal - but it looks terrible for images because it tries to reproduce pixels based on every other pixel in the image. So you get ringing all over the picture instead of sharpness.
- Dylan16807 10y ago
  I am largely focusing on upscaling, because downscaling is easy to get close just by averaging pixels. >What you want is a non-linear filter - it's pretty obvious that you can invent good upscaling filters (like Waifu2x / NNEDI3) that aren't based on FIR theory and can't be used to downscale. When I'm looking at really tiny details even those can cause problems, and I just want pixel blobs. >And that's the opposite of you get from linear filters. Huh? I'm talking about the footprint of the filter, like the diagrams in the pdf. The box filter and Figure 1 filter sample one pixel, so they have the smallest region, and sinc is the largest possible. What's opposite?
  - astrange 10y ago
    Oh, I read that as "filtered regions". Like if you blow up an image with some noise in it, the noise is "infinitely small", so it should remain the same size and not get bigger with the image content.

erikb 10y ago

Well, a model is not the reality, but a useful abstraction. Considering that does it really hurt to wrongfully think of pixels as squares? It hasn't made any troubles to me yet, at least.

lloeki 10y ago

Indeed, Marching Cubes and similar algorithms are voxels and drive home the "not a little cube" point.

dahart 10y ago

This is a classic, and a good point, and comes up on HN every time we talk about voxels. But I think it's worth considering that: - "Pixel" is an overloaded term. "Pixel" is a little square, sometimes, because common law - it's being used that way. - This is an article about art, aimed at artists, and if an artist intends to create a blocky image in two or three dimensions, then what they're doing is "correct" regardless of the terminology they use. - The pictures are really pretty, IMO. I realize some people don't like the block look, but I'm not alone when I say I love it and it's inspiring. The explanations in the article were nicely done. - The article does cover sampling. Just not filtering. - When Alvy Ray wrote his paper, the world was made of blurry CRTs. Now we have really crisp digital LCDs everywhere, and more now than then, pixels on the screen are little squares. - "pixels" even in the days of CRTs have always meant little squares to a lot of people. Video game art benefitted from a little blur, but tons of artists and programmers have always thought of pixels as squares, and for the most part it's a working analogy. This is why it's okay to call a pixel a little square, and overly rigid to claim it's not one. (And I think Alvy Ray was being funny and making an important point about math and graphics, for a siggraph audience, not pedantic about what words artists are using.)

fenomas 10y ago
I would extend this, and say that all terms are overloaded - so before taking issue with a definition it's always worth considering whether you're arguing that one definition is Wrong and the other is Right (which is reductionist and silly), or arguing that one definition is more useful for the context under discussion (which is often useful).

infogulch 10y ago

Honest question: If a pixel is a point sample, why do all the down-sampling algorithms that look good involve some form of averaging the values of neighboring pixels? Wouldn't it be more accurate to pick one sample and throw away the rest?

jstelly 10y ago
This is a fairly deep topic so your question has a really long & detailed answer. The simple answer is "aliasing." The concept is that picking one sample for each pixel makes an aggregate image that could be confused with other images, whereas a good downsampling algorithm only represents a (lower frequency, bandlimited) version of the original image.
dahart 10y ago
It's a good question. The name for that - pick one sample, toss the rest - is "nearest neighbor sampling". Whether it's accurate depends on what you are trying to achieve, but the short answer is that in general it is not the most accurate, and it can cause a lot of problems. But your intuition isn't wrong. Nearest neighbor can be a bit more "crisp" feeling on certain images, especially natural photos. A lot of photos have enough blurriness in the 1-3 pixels range that nearest neighbor sampling is a good choice if you're just resizing an image by a factor of 2. But to see why nearest neighbor isn't the best in general, imagine down-sampling a checkboard and what happens as individual checkers become smaller than a pixel. You'll get moire patterns if you don't filter. You'd also expect pixels to turn grey as multiple checkers land there, and maybe not stick to only black or white. Interestingly, some "sharpening" and up-sampling algorithms do the opposite - they subtract the value of surrounding pixels, rather than add. Those are assuming that some averaging already took place, and trying to un-average, if you will, to restore the original point sample.
munchbunny 10y ago
The short answer is that our eyes and brains don't work that way. The long answer is... really long.

pierrebai 10y ago

While true, this is a category of stances that I quite enjoy poking holes into. I usually bring this up when people deride folk wisdom or trying to show how common sense is incorrect. The basic problem with being petty is that, while you're right on one level, you're wrong on the global scale. In this case, that pixel are square or not is irrelevant. Do you look at your monitor straight on? Do you sit side-by-side with a friend playing a game? In that case, squares become trapezoid from your POV anyway. I'm going to bet you don't mind it, because our brain is very good are pretending perspective distortions don't exist. The same apply in many domains: color correction, for example. I use to write in printing and designers would be hung up on proof to verify colors. No matters that reader would read teh final product under varying lights anyway, making precisely 'correct' colors moot. In both cases, what is important is consistency. For prints, that all runs be the same. For monitors, well, as long as you look at a single monitor at a time, you won't notice distortion. (In a somewhat related anecdote, the piano my parents own is and always has been out of tune. I learned on it and for me, all other pianos tones were unsufferable. What is the right pitch is mostly learned, even though musicians would like us to believe there is but a single harmonious scale.)

tripzilch 10y ago

This is true for computational science applications. A voxel in an MRI-scan is indeed not a little cube, but a data-point. Same for sampling 2D images (e.g. photography). In modern pixel/voxel art, however, the whole point is that they are in fact little squares and cubes. Otherwise you don't get the aesthetic this style is aiming for. It should be noted for historical accuracy, that the origins of pixel art were displayed on relatively blurry CRT screens, and your pixels didn't actually look like little squares either, more like glowy blobs. That's a different point than the cited paper is making, however, which is about the sampling theorem where pixels and voxels are basically data points (that can be interpolated in various ways). On the other hand, when I played 16 colour 320x200 games like Commander Keen and Duke Nukem (2D) in the 90s on a VGA CRT monitor, the pixels definitely looked like little squares, not blobs. Edit: maybe I should mention the oldschool ANSI art as well (BBS/demoscene), which were made with various coloured ASCII-blocks, and there the "pixels" were in fact quite obviously little blocks (and sometimes other characters but that's more ASCII-art than ANSI).