4 ms·

Super Pyxelate converts images to 8-bit pixel art

115 points by minimaxir 5y ago

rcarmo 5y ago
This is very neat, but if you want to see a true (human) master at work, check out Mark Ferrari's GDC talk here: https://youtu.be/aMcJ1Jvtef0 https://youtu.be/aMcJ1Jvtef0 I'd love to see an algorithm that could approximate this degree of mastery :)
- grenoire 5y ago
  The problem is that none of these posterisation algorithms (reducing bit depth of channels and optionally dithering) match the quality of somebody placing the pixels by hand. It always looks more like 'deep fried' images as opposed to pixel art. None of the samples are appealing (except for perhaps that one with palm trees).
  - PostOnce 5y ago
    The Blade Runner one is missing the buildings, robocop has no mouth, BJ is missing part of the right side of his head.
  - jerojero 5y ago
    I would look at them more as a tool that you could use as part of your toolset rather than a complete replacement. You can put an image through it and then do manual touch ups, copy and paste other images and so on and start building a composition.
- anfractuosity 5y ago
  Just started watching that talk, lots of extremely impressive artworks, I'd never heard of the colour cycling techniques he talks about before which are very neat.
cupcake-unicorn 5y ago
Thanks so much for this! I'm not an artist at all so this will be great for my indie game dev needs
Animats 5y ago
Now you can generate cheezy NFT art from public domain images.
sekao 5y ago
See also... https://github.com/hpjansson/chafa https://github.com/hpjansson/chafa
elpocko 5y ago
"I made a program that makes images look worse" seems like a realistic, but weird claim.
- daneel_w 5y ago
  And accurate. It's just re-scaling and palette indexing, no different than what people toyed around with using Photoshop in 1998. It looks nothing at all like hand-pixeled art.
  - Groxx 5y ago
    I'm kinda curious how this compares to imagemagick -> limit the colorspace. Though getting imagemagick to produce some of those styles of images may be difficult or impossible (like the purple/pink pattern on the bottom left corgi).
  - andai 5y ago
    Actually, > Pyxelate downsamples images by (iteratively) dividing it to 3x3 tiles and calculating the orientation of edges inside them. Each tile is downsampled to a single pixel value based on the angle the magnitude of these gradients, resulting in the approximation of a pixel art. This method was inspired by the Histogram of Oriented Gradients computer vision technique. > Then an unsupervised machine learning method, a Bayesian Gaussian Mixture model is fitted (instead of conventional K-means) to find a reduced palette. The tied gaussians give a better estimate (than Euclidean distance) and allow smaller centroids to appear and then lose importance to larger ones further away. The probability mass function returned by the uncalibrated model is then used as a basis for different dithering techniques. > Preprocessing and color space conversion tricks are also applied for better results. But yeah for all the effort, it doesn't look that much better than just exporting with a small palette and positional dithering.
minimaxir 5y ago
Interactive demo here (which is how I became aware of the submitted repo, although neither are created by me): https://huggingface.co/spaces/akhaliq/Pyxelate https://huggingface.co/spaces/akhaliq/Pyxelate
mdp2021 5y ago
One case in which ANN intervention could possibly discard unwanted results.
- minimaxir 5y ago
  tbh I was looking into experimenting into this. Filtering bad results is how DALL-E works so it's not a crazy idea.
nope96 5y ago
This reminds me of a tool in the BBS days that would convert a GIF into ANSI Art. (I think it was called gif2ansi)
6581 5y ago
This may be 8 bit, but it's not pixel art.
gwbas1c 5y ago
The images on that page are much lower color than 8 bit. 8-bit color is 256 total colors on the image. A combination of a good pallet and dithering can be very photorealistic.
- placatedmayhem 5y ago
  I believe the "8-bit" descriptor is intended to reference 8-bit video game systems like the NES, but that seems poorly communicated in the project.
carrolldunham 5y ago
Wow, sorry to be blunt but 1) it looks AWFUL. I didn't expect it to robustly-successfully redraw sources but this is just a dismal implementation of depth-reduction. 2) Its offensive, insulting, to say that this converts into 'pixel art'. 3) Not only that but it is trivial. I'm sure almost anyone reading will have done something at this level for interest in two or three lines and then abandoned it seeing how bad it looks. As a pallet cleanser (ha) here's an article about actually good low-color automated processing https://bisqwit.iki.fi/story/howto/dither/jy/ https://bisqwit.iki.fi/story/howto/dither/jy/
- kingcharles 5y ago
  That's an amazing article. I had to dither all the 24-bit color sprite assets from the artists, when I was a video game developer in the 90s, into a single 256-color palette that was hand-picked. I just used a simple euclidian-distance algorithm and thought I was clever. As always with HN I come out looking like an imposter!
newobj 5y ago
Subsampling and dithering the shit out of something does not pixel art make. It's called "art" for a reason
drdeca 5y ago
I wonder if one alternate approach could perhaps be to do image segmentation on the original image, decide what segments are important enough to be preserved, and then, using other techniques to do the downscaling and color choosing, but while enforcing an additional constraint of sufficient contrast between the downscaled versions of the different segments? Like, if some detail would, if scaled down proportionally with everything else would become too small to make out, but where by making it a bit bigger than it should be would remain both recognizable, while still fitting ok within whatever larger thing? E.g. converting lips into a row of pixels when necessary. Similarly with eyes.
aaronbrethorst 5y ago
Why does its logo evoke the SNES if it’s converting images to an 8-bit style?
b20000 5y ago
for all the cool machine learning stuff and libs involved the results look like what i could do in the 90s in photoshop. it doesn’t really look like hand drawn pixel art IMHO.
butz 5y ago
This effect happens naturally, after re-saving print screen of an image in JPG format a few times. More interesting might be pixray project (https://github.com/dribnet/pixray https://github.com/dribnet/pixray) where pixel art is generated using machine learning from keyword prompts.

115 points by minimaxir 5y ago

This is very neat, but if you want to see a true (human) master at work, check out Mark Ferrari's GDC talk here: https://youtu.be/aMcJ1Jvtef0 https://youtu.be/aMcJ1Jvtef0 I'd love to see an algorithm that could approximate this degree of mastery :)

grenoire 5y ago
The problem is that none of these posterisation algorithms (reducing bit depth of channels and optionally dithering) match the quality of somebody placing the pixels by hand. It always looks more like 'deep fried' images as opposed to pixel art. None of the samples are appealing (except for perhaps that one with palm trees).
- PostOnce 5y ago
  The Blade Runner one is missing the buildings, robocop has no mouth, BJ is missing part of the right side of his head.
- jerojero 5y ago
  I would look at them more as a tool that you could use as part of your toolset rather than a complete replacement. You can put an image through it and then do manual touch ups, copy and paste other images and so on and start building a composition.
anfractuosity 5y ago
Just started watching that talk, lots of extremely impressive artworks, I'd never heard of the colour cycling techniques he talks about before which are very neat.

cupcake-unicorn 5y ago

Thanks so much for this! I'm not an artist at all so this will be great for my indie game dev needs

Animats 5y ago

Now you can generate cheezy NFT art from public domain images.

sekao 5y ago

See also... https://github.com/hpjansson/chafa https://github.com/hpjansson/chafa

elpocko 5y ago

"I made a program that makes images look worse" seems like a realistic, but weird claim.

daneel_w 5y ago
And accurate. It's just re-scaling and palette indexing, no different than what people toyed around with using Photoshop in 1998. It looks nothing at all like hand-pixeled art.
- Groxx 5y ago
  I'm kinda curious how this compares to imagemagick -> limit the colorspace. Though getting imagemagick to produce some of those styles of images may be difficult or impossible (like the purple/pink pattern on the bottom left corgi).
- andai 5y ago
  Actually, > Pyxelate downsamples images by (iteratively) dividing it to 3x3 tiles and calculating the orientation of edges inside them. Each tile is downsampled to a single pixel value based on the angle the magnitude of these gradients, resulting in the approximation of a pixel art. This method was inspired by the Histogram of Oriented Gradients computer vision technique. > Then an unsupervised machine learning method, a Bayesian Gaussian Mixture model is fitted (instead of conventional K-means) to find a reduced palette. The tied gaussians give a better estimate (than Euclidean distance) and allow smaller centroids to appear and then lose importance to larger ones further away. The probability mass function returned by the uncalibrated model is then used as a basis for different dithering techniques. > Preprocessing and color space conversion tricks are also applied for better results. But yeah for all the effort, it doesn't look that much better than just exporting with a small palette and positional dithering.

minimaxir 5y ago

Interactive demo here (which is how I became aware of the submitted repo, although neither are created by me): https://huggingface.co/spaces/akhaliq/Pyxelate https://huggingface.co/spaces/akhaliq/Pyxelate

mdp2021 5y ago

One case in which ANN intervention could possibly discard unwanted results.

minimaxir 5y ago
tbh I was looking into experimenting into this. Filtering bad results is how DALL-E works so it's not a crazy idea.

nope96 5y ago

This reminds me of a tool in the BBS days that would convert a GIF into ANSI Art. (I think it was called gif2ansi)

6581 5y ago

This may be 8 bit, but it's not pixel art.

gwbas1c 5y ago

The images on that page are much lower color than 8 bit. 8-bit color is 256 total colors on the image. A combination of a good pallet and dithering can be very photorealistic.

placatedmayhem 5y ago
I believe the "8-bit" descriptor is intended to reference 8-bit video game systems like the NES, but that seems poorly communicated in the project.

carrolldunham 5y ago

Wow, sorry to be blunt but 1) it looks AWFUL. I didn't expect it to robustly-successfully redraw sources but this is just a dismal implementation of depth-reduction. 2) Its offensive, insulting, to say that this converts into 'pixel art'. 3) Not only that but it is trivial. I'm sure almost anyone reading will have done something at this level for interest in two or three lines and then abandoned it seeing how bad it looks. As a pallet cleanser (ha) here's an article about actually good low-color automated processing https://bisqwit.iki.fi/story/howto/dither/jy/ https://bisqwit.iki.fi/story/howto/dither/jy/

kingcharles 5y ago
That's an amazing article. I had to dither all the 24-bit color sprite assets from the artists, when I was a video game developer in the 90s, into a single 256-color palette that was hand-picked. I just used a simple euclidian-distance algorithm and thought I was clever. As always with HN I come out looking like an imposter!

newobj 5y ago

Subsampling and dithering the shit out of something does not pixel art make. It's called "art" for a reason

drdeca 5y ago

I wonder if one alternate approach could perhaps be to do image segmentation on the original image, decide what segments are important enough to be preserved, and then, using other techniques to do the downscaling and color choosing, but while enforcing an additional constraint of sufficient contrast between the downscaled versions of the different segments? Like, if some detail would, if scaled down proportionally with everything else would become too small to make out, but where by making it a bit bigger than it should be would remain both recognizable, while still fitting ok within whatever larger thing? E.g. converting lips into a row of pixels when necessary. Similarly with eyes.

aaronbrethorst 5y ago

Why does its logo evoke the SNES if it’s converting images to an 8-bit style?

b20000 5y ago

for all the cool machine learning stuff and libs involved the results look like what i could do in the 90s in photoshop. it doesn’t really look like hand drawn pixel art IMHO.

butz 5y ago

This effect happens naturally, after re-saving print screen of an image in JPG format a few times. More interesting might be pixray project (https://github.com/dribnet/pixray https://github.com/dribnet/pixray) where pixel art is generated using machine learning from keyword prompts.