12 ms·
Depixelizing Pixel Art
- digitalpacman 13y agoThis is pretty amazing of a demo.
- rsiqueira 13y agoAmazing image zoom algorithm, t creates a hires version of a low res image. I would love to see this image zoom feature implemented the in ImageMagick.
- adamnemecek 13y agoTechnology almost caught up with CSI.
- bradyd 13y agoThere is a technique that might actually be able to do this called compressed sensing[1] that I first heard about it in a Wired article from several years ago[2]. The technique allows reconstructing images, music, etc. by sampling a portion of the original and applying an algorithm that reconstructs the missing information and is apparently used in MRI image processing to get by with shorter scans. Researchers from Rice University constructed a camera with only a single pixel image sensor that, using this technique, produced legible images with as little as a 2% sampling of the final image resolution, although more complex images required 10-20% or more[3]. I don't see why this technique couldn't be used to take already good quality images and "enhance" areas of them like in CSI. [1] https://en.wikipedia.org/wiki/Compressed_sensing https://en.wikipedia.org/wiki/Compressed_sensing [2] http://www.wired.com/magazine/2010/02/ff_algorithm/ http://www.wired.com/magazine/2010/02/ff_algorithm/ [3] http://dsp.rice.edu/cscamera http://dsp.rice.edu/cscamera
- nivla 13y agoIs there a paper or code associated with this?
- adamnemecek 13y agohttp://research.microsoft.com/en-us/um/people/kopf/pixelart/ http://research.microsoft.com/en-us/um/people/kopf/pixelart/
- nivla 13y agoThank you.
- mxfh 13y agoWhich also has the original HN discussions https://news.ycombinator.com/item?id=2673271 https://news.ycombinator.com/item?id=2673271 and links to +1000 day old threads: https://news.ycombinator.com/item?id=2601347 https://news.ycombinator.com/item?id=2601347 https://news.ycombinator.com/item?id=2578706 https://news.ycombinator.com/item?id=2578706 yet those space invaders still haunt me.
- deleted 13y ago[deleted]
- computerslol 13y agoLooks great :) Makes me a bit less tense about the appearance of older programs in the 4k+ display future.
- joosters 13y agoThis would be awesome if it was available as a scaler for high-DPI screens and old apps. Or even bundled into televisions for upscaling. The paper does note that it is designed for hand-edited pixel art, where every different coloured pixel is treated as significant. So I wonder how it behaves when scaling photos? (Edit: the paper does note that it doesn't do well on anti-aliased graphics...)
- panzi 13y agoTelevisions have upscale algorithms. But nothing like this. But they don't need anything like this because they don't scale pixle art with sharp borders and very low resolution. They scale video footage (which one could say is "anti-aliased") with about 640x480 resolution.
- mrbuttons454 13y agoto me, hq4x looks better every time.
- tibbon 13y agoI was thinking that too, but then came to terms with 'ours' version
- Ricapar 13y agoI think it depends on what your end goal is. Are you trying to scale and hide the pixel-ness of the original? The "ours" does pretty darn well, especially if you look at the Boo example as mentioned above. However, if you're looking at in the case of an SNES emulator or something.. then hq4x seems like the best choice. It keeps true to the original pixel art, while appearing significantly better.
- fuzzywalrus 13y agoLinked earlier: Video of Ours vs HQ4X. Ours seems to have a slight advantage to my eyes, admittedly it is a personal preference. Text has a distinct advantage on HQ4X but Mario and Yoshi look better even if less "accurate". https://research.microsoft.com/en-us/um/people/kopf/pixelart/supplementary/video_multi_comparison_4x_h264.mp4 https://research.microsoft.com/en-us/um/people/kopf/pixelart...
- gverri 13y agoLooks like a normalized version of the EPX algorithm.
- th0ma5 13y agoI thought this looked familiar! It is from 2011 https://research.microsoft.com/en-us/um/people/kopf/pixelart/ https://research.microsoft.com/en-us/um/people/kopf/pixelart...
- tibbon 13y agoAre there any emulators that support this?
- cedel2k1 13y agoYay, tons of.. at least with the algo's they compare their stuff to.
- jrockway 13y agoI wish they would use this in Windows 8. As far as I can tell, their scaling algorithm for 4k displays is: scale up 2x using nearest neighbor, then randomly drop pixels to get back to 125% or 150%. The result is horrifyingly bad. Not just kind of bad, but unusably bad. (Adding insult to injury, they ClearType the text before scaling.)
- bluedino 13y agoI would guess it's for performance reasons.
- jrockway 13y agoI have 4 i7 cores and a $500 graphics card. I feel like it's possible :) TF2, for example, plays beautifully at 3840x2160@60Hz. Scaling up the graphics for an 800x600 visual novel that plays at 1 frame per 30 seconds is computationally possible. (The good news is that someone has written an emulator for one VN that I'm working through, so I should be able to play with scaling in that way. If I can get it to compile.)
- rkuykendall-com 13y agoDo you know if MS is working on Mac-Style resolution independence for W9? I'm currently running a Retina MBP, at 2x res I think, and it works _perfectly_. The first few months some apps didn't work well, but the community adapted very quickly.
- 13y ago
- rkuykendall-com 13y agoDoes anyone know where I can find an implementation of this? I can't even figure out what to call it. They just refer to it as "our method."
- jonathanj 13y agoHere's a pure Python implementation: https://github.com/jerith/depixel https://github.com/jerith/depixel
- jonathanj 13y agoA work in progress implementation, anyway.
- rkuykendall-com 13y agoThanks! See? Even he doesn't know what to call it!
- sp332 13y agoI wonder why they all fall down on Peach and Peach 2? None of them look nearly as good (to me) as bicubic.
- Lerc 13y agoI think it comes down to your point of focus. With peach and peach 2, The eyes change shape quite a bit. Humans notice that much more than other aspects. In general, I think the "ours" method could benefit by moving the boundaries between colours in and out until they get an average brightness match with the similar area in the nearest neighbour version. A lot of the "ours" images have a higher proportion of black.
- mistercow 13y agoSuper2xSaI looks better to me at 4x than bicubic, and EPX does an OK job but the others look pretty terrible. I think the problem is that those sprites use multiple strategies for handling diagonal outlines. Those look fine when the image is blurred together, but when you try to detect the outlines and upscale them, the differences between techniques are exaggerated.
- frik 13y agohq4x looks better (IMHO), sadly the website is offline (default blank wordpress site): http://hiend3d.com/hq4x.html http://hiend3d.com/hq4x.html backup on archive.org: https://web.archive.org/web/20120630044334/http://www.hiend3d.com/demos.html https://web.archive.org/web/20120630044334/http://www.hiend3... http://en.wikipedia.org/wiki/Hqx http://en.wikipedia.org/wiki/Hqx Source: https://code.google.com/p/hqx/source/browse/trunk/src/hq4x.c https://code.google.com/p/hqx/source/browse/trunk/src/hq4x.c
- lloeki 13y agoOnly if you look at the first ones. All of Boo, Dolphin, all of SoM, Invaders, Golden Axe, Windows assets and Lost Vikings look great and have vastly reduced artifacts compared to hqx. Also, according to the paper abstract[0], it outputs vector, not scaled up bitmap. [0]: http://research.microsoft.com/en-us/um/people/kopf/pixelart/ http://research.microsoft.com/en-us/um/people/kopf/pixelart/
- panzi 13y agoAs you say: It depends. E.g. the sword looks better with hq4x, because the result isn't rounded.
- panzi 13y agoThe Super Mario and Secret of Mana stuff works quite good, except it changes the facial expression on "Peach 2". I'd like to see Donkey Kong Country 1/2/3, Megaman (NES and SNES) and Chrono Trigger processed by this. And not just the sprites cut out of the background but embedded in a scene.
- lloeki 13y ago"ours" Peach 1 suddenly looks really sad.
- vacri 13y agoLook at Axe Battler 2 - if the sword isn't vertical, then the result is not superior. Unless, of course, you think the guy is wielding a sawtooth blade. Not to mention that muscles have a smoothly-graded tone under the 'ours' algorithm, whereas hq4x has an unlifelike blocky effect.
- mberning 13y agoBoo is the best example of 'ours'. Looks amazing.
- zem 13y agoyeah, it seems to handle patches of complex colouring badly (especially where two tones of the same colour were used to good effect in the icon, but scaled up as messy blobs), but boo and the invaders were very impressive.
- Trufa 13y agoI think bower looks badass! http://research.microsoft.com/en-us/um/people/kopf/pixelart/supplementary/results_ours/smw_bowser_ours_8x.png http://research.microsoft.com/en-us/um/people/kopf/pixelart/...
- smackfu 13y agoFrom when this first came out: https://news.ycombinator.com/item?id=2601347 https://news.ycombinator.com/item?id=2601347 https://news.ycombinator.com/item?id=2578706 https://news.ycombinator.com/item?id=2578706
- josh-wrale 13y agoI chose to embrace the Pixels, myself: http://vulariter.wordpress.com/tag/jewel-punk/ http://vulariter.wordpress.com/tag/jewel-punk/
- deleted 13y ago[deleted]
- rsiqueira 13y agoThe authors of this paper also have other amazing image processing/optimization algorithms. Eg: Deblurring, Optimizing Color Consistency, Content-aware Automatic Photo Enhancement, Object Discovery and Segmentation in Internet Images and more: Dani Lischinski's page: http://www.cs.huji.ac.il/~danix/ http://www.cs.huji.ac.il/~danix/ Johannes Kopf page: http://johanneskopf.de/ http://johanneskopf.de/
- vanderZwan 13y agoWow, the content-aware downscaling seems very useful! http://research.microsoft.com/en-us/um/people/kopf/downscaling/ http://research.microsoft.com/en-us/um/people/kopf/downscali... > This paper introduces a novel content-adaptive image downscaling method. The key idea is to optimize the shape and locations of the downsampling kernels to better align with local image features. [...] Besides natural images, our algorithm is also effective for creating pixel art images from vector graphics inputs, due to its ability to keep linear features sharp and connected. IIRC, this is one of the major reasons that vector graphics based icons have not taken off.
- JasonFruit 13y agoSometimes the Vector Magic results are truly magical; striking poses become even more striking when the figure is reduced to a collection of simpler shapes.
- vsviridov 13y agoI like a lot of VectorMagic results... Very abstract art!
- zimpenfish 13y agoYeah, I really like the VM ones - particularly how the space invader turns into a StarTrek badge and the Bombermans(men?).
- Orangeair 13y agoIt reminds me of old Microsoft Word clipart.
- zeidrich 13y agoThis is a pretty old paper, but I hadn't seen the live demo. That's pretty cool. A problem with this algorithm is that it's pretty slow. It would be hard to use it for real time scaling. I think that it's pretty cool though for any precalculated conversion to vector format. It would also be pretty cool if there were software that allowed you to create a pixel art image with a live preview in this format and the ability to export into a vector format. One of the benefits of pixel art IMO is that it's got a pretty distinct style and its simplicity and restrictions means it's easy to be aware of composition where if you are given an unrestricted canvas that freedom can make it hard to know where to start. If you are given a 8x8 pixel grid to indicate "happy" there's only so many ways to do it, a symbolic heart, a little smiley face, etc. Given a whole sheet of paper you'll have more difficulty. With the high res displays, there's more to consider than just the upscaling of old programs, the fact is people (especially indie game devs) still want to work with pixel art, and the number of titles with the term "8-bit" in their name that have come out recently. Part of that's because it's in vogue now, but a big part of it is simply that a poor artist can make a passable 24x24 sprite. But it's still a shame that on high res displays they end up looking like a blocky mess. If you had a program that let you author your sprites under the same conditions, but let you preview an upscaled version and left you with a vector image (or high res texture with mip maps) you can make use of the fidelity while maintaining the simplicity. I think that would be cool.
- ahassan 13y agoWell SuperEagle seems even slower for me, and I know for a fact that I can emulate with SuperEagle scaling on and it works fine (>30 FPS). So I don't think their algorithm is out of the question.
- lloeki 13y ago> If you had a program that let you author your sprites under the same conditions, but let you preview an upscaled version Combine Atwood's law with a proposed law of mentioning ideas on HN being implemented shortly thereafter and there may be a HTML5+JS implementation popping up as a "Show HN" soon.
- glhaynes 13y ago
- tumes 13y agoI must be a luddite, I prefer my pixilation like my soup: chunky :)
- 10098 13y agoBut the blocky versions actually look better than vectorized ones. Most of the vectorized sprites look hideous with the exception of Boo and dolphin.
- GhotiFish 13y agothey have a certain style to them that is lost at higher resolutions.
- userbinator 13y agoCount me as another one who likes the pixel aesthetic. The title made me ask myself "why would you want to do this?" (I also loathe most forms of antialiasing, including ClearType, so that might have something to do with it... I like sharp, clearly contrasted edges, including pixel edges.)
- Kiro 13y agoNo, I think the vectorized versions are much better but yeah, Boo is the best one.
- sz4kerto 13y agoI can't believe noone linked this so far. https://www.youtube.com/watch?v=3uoM5kfZIQ0&hd=1 https://www.youtube.com/watch?v=3uoM5kfZIQ0&hd=1
- panzi 13y agohttps://www.youtube.com/watch?v=JMIHNiR3CP8 https://www.youtube.com/watch?v=JMIHNiR3CP8 ;)
- pesenti 13y agoI can't believe you are linking to CSI instead of Blade Runner, the grand daddy of all enhance scenes http://www.youtube.com/watch?v=qHepKd38pr0 http://www.youtube.com/watch?v=qHepKd38pr0 makes me feel old...
- neals 13y agoFunny how I don't remember those vikings to be so low-detail and tiny.
- sitkack 13y agoTodo: Blend hq4x and the kopf techniques in a post processing step.
- Lambdanaut 13y agoThe Space Invaders "ours" were very impressive. The text was always really good too. The algorithm works well on more regular shapes.
- josephschmoe 13y agoDoesn't work well with geometric objects. Look at the swords, shields, the letter 'A'.
- kaeso 13y agoA C++ LGPL implementation of this has been as been developed as part of last year Inkscape GSoC and is available as a standalone library: https://launchpad.net/libdepixelize https://launchpad.net/libdepixelize It will be released in the next Inkscape major version, expected soon :)
- SimHacker 13y agoPerfect! Thanks for the link. I want to run this over the classic SimCity tile set (and the additional ones in the add-on graphics sets like the moon base, etc), to generate high resolution pixel and vector tiles for the JavaScript browser based version of Micropolis (GPL SimCity).
- panzi 13y agoIs there no 8x or 16x version of hq4x? I thought you can use it to get full HD resolution from 320x240 (in realtime using GLSL shader).
- dlsym 13y agoWell I guess it's retro week again on HN... Better check my bookmarks from 3 years ago...
- anon4 13y agoThe issue with depixelizing pixel art is that you need an AI that can identify semantically what the features in the sprite are. When you look at Mario you know you're looking at a stout little man with a hat, you can identify his eyes, ears, nose, hair, hat, moustache, shirt, gloves, overalls and shoes. Given that knowledge, if I were to depixelize Mario's sprite, I'll know the rough shape of his shoes and where the shading should go and what shape it should have, i.e. I can reason what the 3D shape is and how the 2D shape was derived from that. I'll also be able to identify which lines were put there to make sure separate elements can be legible and not make those thicker than needed. Or that his eyes blend with his hat simply because there aren't enough pixels to make them separate. For simple shapes, like the background, their algorithm works really well, but for complex objects it fails, because it distorts details that were put in with very careful thought and completely depend on the resolution. Such small sprites rely a lot on being looked at by someone who can identify semantically what they're looking at, and any really successful depixelization solution will need to be able to understand what basic shapes the sprite is made of based on what it's supposed to represent.
- Houshalter 13y agoI've been thinking about doing something like this. Compress images down to half resolution, then train a neural network to blow it back up and guess the pixels as accurately as possible.
- mistercow 13y agoI did something very much like this years ago (upscaling used to be an obsession of mine). The results were pretty disappointing. But there's a lot of space to explore there in terms of choosing and designing the neural net, choosing the right training set, and figuring out an initial transform for data that is input to the neural net.
- thesteamboat 13y agoNew deep learning techniques might help this approach a lot.
- NicoJuicy 13y agoThey forgot natural neighbour interpolation... Should show better results then their research ;)
- TepidSolarsoul 13y agoI have a hard time believing the use of Nintendo IP in this case (big corporation-backed research lab) falls under Fair Use.
- galactus 13y agoI wrote an implementation of this in common lisp when they published it: http://dev.crazyrobot.net/2011/07/12/pixelart-to-vector/ http://dev.crazyrobot.net/2011/07/12/pixelart-to-vector/
- fisherprice 13y agoThe 'ours' filter (I guess it's the one from MS) is mind-blowing.
- rangibaby 13y agoHQ4X and Ours are the clear winners if you must resample like this; the downsides are that HQ4X occasionally has jaggies and Ours makes makes a lot of fine details look "blobby". The Vector Magic results reminded me a lot of this: http://www.cowboybooks.com.au/html/acidtrip1.html http://www.cowboybooks.com.au/html/acidtrip1.html
- jatin085 13y agoRegarding HQ4X, they used the algorithm to create 4X image but used Nearest neighbor to create 8X and 16X images. I think doing HQ4X two times would have resulted into a better 16X images without jags.
- omegote 13y agoI'm amazed to find that no one has mentioned that ScummVM has implemented some of these algorithms for years and they work pretty nice with the classical games.
- baumgarn 13y agoand snes emulators ...10 years ago?
- granttimmerman 13y agoHere's the home page: http://research.microsoft.com/en-us/um/people/kopf/pixelart/ http://research.microsoft.com/en-us/um/people/kopf/pixelart/
- bart3r 13y agoAre there any ruby libraries that would depixel an image in a similar fashion?
- neona 13y agoIt would be interesting to see a pixelart/emulator scaler that was temporally aware - that is, able to use context from near or all relevant sprite frames to get additional detail/context. This would require a compositing emulator, as some others have discussed, but I could see it producing notably superior results, given it would have much more source data to work with.
- bovermyer 13y agoThe Secret of Mana ones made my eyes leak, not gonna lie. Awesome graphics technology update not withstanding, that game was an integral part of my teenage years.
- mjmahone17 13y agoKind of ironic that MSR is using an unregistered version of PhotoZoom.
- asd 13y agoCtr+f "Invader 1" and choose the Vector Magic version. Star Trek!
- Legend 13y agoThis seems pretty awesome. Can someone please tell me where this could be useful?
- mistercow 13y agoI think the most interesting aspect of this is the internal shading. I haven't seen any other algorithm that manages that while maintaining sharp edges overall. I imagine it would end up with pretty weird results though when a sprite was on top of a background that with similar enough colors to trigger the shading detection at its edges.
- rocky5 13y agoamazing.
- kumarrahul 13y agoProper Video here :- https://research.microsoft.com/en-us/um/people/kopf/pixelart/supplementary/video_multi_comparison_4x_h264.mp4 https://research.microsoft.com/en-us/um/people/kopf/pixelart...
- Udo 13y agoAt first I thought this was a live JS-based demo. That would have been amazing.
- NKCSS 13y agoJust awesome.
- yzh 13y agoI did a GPU version in 2012 for my course project. For small size sprite, it runs roughly the same or even slower than the original paper. Anyone interested in finishing this and maybe try this on realtime emulator? http://yzhwang.github.io/depixelization/ http://yzhwang.github.io/depixelization/
- pesenti 13y agoEnhance! http://www.youtube.com/watch?v=qHepKd38pr0 http://www.youtube.com/watch?v=qHepKd38pr0
- virtualritz 13y agoIt actually looks like as if you took the polylines from EPX and used the points as the CVs for a biquadratic or bicubic interpolating spline (e.g. Catmull-Rom), you'd be very close to their result already.
- actionsketch 13y agoI just learned that my favorite depixelizer is hq4x
- klh_dk 13y agoactually running the hqx algo on the art and then throwing it through ANY vectorisation engine with a slight roundness comes on top of any of these. And Microsoft is not really being fair to Vector Magic here, there's a ton of settings there to improve that rendering depending on what you want...
- aaronetz 13y agoSomething related that I stumbled upon some time ago (submitted to HN, got one upvote): http://gpuhacks.wordpress.com/2013/07/08/signed-distance-field-rendering-of-color-bit-planes/ http://gpuhacks.wordpress.com/2013/07/08/signed-distance-fie...
- whywhywhy5 13y agoI think hq4x is the best. Microsoft's curves too much - and while it can look good/stylized for some pieces, it's ultimately inaccurate and warps many of the pieces from their original form.
- dangayle 13y agoAm I the only one who prefers the Vector Magic versions? They're like mini Picassos.
- kstenerud 13y agoI needed some depixelation software for some graphics awhile back, so I threw together a quick program that depixelates using various technologies. https://github.com/kstenerud/depixelate https://github.com/kstenerud/depixelate So far it supports HQX, XBRZ, and Scale2x.