10 ms·
Depixelizing Pixel Art (2011)
- FreakyT 9y agoEvery time I see this paper I'm always disappointed that there doesn't seem to exist a usable (open source or otherwise) implementation of it anywhere.
- MBCook 9y ago(2011)
- ktpsns 9y agoThe longest discussion about this article on hacker news so far: https://news.ycombinator.com/item?id=2578706 https://news.ycombinator.com/item?id=2578706 I wonder why Microsoft payed him to do develop this kind of algorithm.
- jjcm 9y agoProbably as a research project to see if it's worth using in HD conversions of old games, such as the Age of Empires remake that's dropping tomorrow. The algorithm seems to work well on standalone sprites, but kinda merges in a gooey way when two sprites are next to each other. The super mario example from the previous discussion shows this slightly: http://files.jjcm.org/upscaling.mp4 http://files.jjcm.org/upscaling.mp4 Very curious to see this as it progresses however.
- ttoinou 9y agoTaking time into account is missing. I guess the graph reconstruction could also depends on some kind of pixel art optical flow
- simcop2387 9y agoI imagine it was also useful to see if it could be used to scale up low res artwork in older programs to hidpi displays.
- pradn 9y agoIt looks like the transformation in being applied to the whole image at once in your example. If each sprite was vectorized individually, you wouldn't see the gooeyness. If you are a game developer, this change should be easy enough.
- simooooo 9y agoI'm certain I've already seen this as a shader option in emulators
- c12 9y agoI'm still impressed with the super mario example, I can't personally see anything negative about it and would enjoy playing the game with that "art style." It's not perfect but seeing the alternatives it is a huge step in the right direction.
- LoSboccacc 9y agoI don't think it was about videogames, more likely upscaling algorithm would have saved a TON of bandwidth for videos. a a matter of fact some of the most recents algo available for video deinterlacing like nnedi3 are perfect for that, and of course also work for pixel art, and I think they blow away the example showed in their gallery - https://www.youtube.com/watch?v=0691zsXWbhA https://www.youtube.com/watch?v=0691zsXWbhA I wonder if now, in the tensorflow era, a better automatic approach using generative nn could be devised to reach beyond 8x
- bitmadness 9y agoMSR researchers have a lot of freedom, Microsoft doesn't micromanage exactly what they work on.
- drcode 9y agoA fantastic classic paper (but ancient history)
- oceanghost 9y agoIs there an implementation of this anywhere?
- ttoinou 9y agoI love this paper ! Has anyone come up with an interactive pixel art tool using this algorithm ?
- zhamisen 9y agoIs this work been already used in emulators filters?
- danbolt 9y agoFilters have achcieved similar results, but not quite with the same technique! Or, this algorithm wasn’t intended for real-time processing and creates an adjacency graph of similar pixels. Once it’s done that, it will try to fit/optimize splines over the regions. Emulator filters like hqx determine each pixel’s “upscaled shape” by the colours of its neighbours, usually indexed with a table.
- lwansbrough 9y agoCould you take this a step further? For example, a human knows this thing isn't supposed to be lumpy. What would it take to make this look more like it was interpreted by a human?
- Nition 9y agoIt would take an understanding of what the image actually represents.
- deleted 9y ago[deleted]
- supermdguy 9y agoSuper resolution with deep neural networks has gotten some decent results that incorporate knowledge of what the image is. However, it's probably not worth it. Running this algorithm on a single image is much simpler than finding thousands of pixel art -> image pairs and training a superresolution network.
- jcims 9y agoI like the aesthetic of hq4x in a lot of them, it’s a little less loopy.
- Tommakx 9y agoDoes anyone have an open source implementation of this algoritm? Or a just a demo? I would love to play around with it!
- phnofive 9y agohttps://github.com/vvanirudh/Pixel-Art https://github.com/vvanirudh/Pixel-Art can't vouch for it yet, but will be giving it a try later.
- cknight 9y agoA quick play around and I find myself suffering from this same issue: https://github.com/vvanirudh/Pixel-Art/issues/4 https://github.com/vvanirudh/Pixel-Art/issues/4 A shame, I'm really keen to spruce up some old 50x50px forum icons from where I used to hang around back in the early naughties.
- 0x00000000 9y agoVideo is quite neat. http://johanneskopf.de/publications/pixelart/supplementary/video_ours_4x_h264.mp4 http://johanneskopf.de/publications/pixelart/supplementary/v... Reminds me of some old newgrounds flash animations and games
- blt 9y agowaifu2x (http://waifu2x.udp.jp/ http://waifu2x.udp.jp/) is another often-posted superresolution approach, focusing on upscaling low-res anime images instead of extremely tiny pixel art. It uses a totally different learning-based approach - however, their approach would not work with pixel art because it depends on having a set of high resolution ground truth images, and downsampling them to generate training data pairs. No such high-resolution examples exist for pixel art.
- anonymfus 9y ago> No such high-resolution examples exist for pixel art. How about application icons?
- blt 9y agogood point!
- murukesh_s 9y agowaifu2x seems to be based on Deep Convolutional Neural Networks, while the article looks like a pure handwritten algorithm and at least in this case the algorithm looks like a winner.. I am wondering whether in the future we may forget about writing algorithms altogether and instead rely on ANNs to do tasks that could have done better using hand written algorithms?
- blt 9y agoYou mean tasks like computing the rank of a matrix? (http://oneweirdkerneltrick.com/rank.pdf http://oneweirdkerneltrick.com/rank.pdf) ;) Image super-resolution is an ill-posed inverse problem: many possible high-resolution images would be reduced to the same low-resolution image. However, some of the possibilities are more realistic than others. Consider a grey pixel in a photograph from real life. It's more likely that it was downsampled from a 2x2 block of grey pixels than from a 2x2 black-and-white checkerboard pattern. We apply such knowledge by adding regularization to the problem, or using a prior distribution in a Bayesian formulation. Deep learning is very good at memorizing those priors by looking at real world data. I think, in problems where the prior is important and complicated, data-driven approaches have a big advantage over hand-engineered approaches. That does not mean they will take over every kind of problem.
- larsiusprime 9y agoHow does this compare to XBR? https://github.com/Treeki/libxbr-standalone https://github.com/Treeki/libxbr-standalone
- bhouston 9y agoSomething incredibly similar is already built into the latest versions of Snes9x. hq4x and the #xBRZ algorithms are amazing and fast and look nearly identical to these results.
- freshyill 9y agoThey compare it to hq4x in their supplementary materials. Theirs is better. http://johanneskopf.de/publications/pixelart/supplementary/index.html http://johanneskopf.de/publications/pixelart/supplementary/i...
- lathiat 9y agohq4x looks a lot better in the video comparison: http://johanneskopf.de/publications/pixelart/supplementary/video_multi_comparison_4x_h264.mp4 http://johanneskopf.de/publications/pixelart/supplementary/v... This method still wins out though.. but interesting never the less.
- cooper12 9y agoI think hq4x has a slight edge because it doesn't go all the way trying to make everything smooth, still preserving some pixelation. There's actually an analog to this in denoising: if you denoise too heavily you end up losing detail. Smart denoising manages to preserve edges, giving a better result. In this case, pixels are the same as noise, undesirable, but containing useful information.
- mikewhy 9y agoxBRZ is pretty close: https://gfycat.com/FrequentWanCatbird https://gfycat.com/FrequentWanCatbird
- mikewhy 9y agoxBRZ can do wonders, especially for Link To The Past.
- samstave 9y agoIsnt that called "anti Aliasing" :-) (just kidding)
- EgoIncarnate 9y ago(2011)
- tptacek 9y agoIs there something that gets a really good result going the opposite direction?
- rangibaby 9y agofor both directions I would say a competent artist. Clever algorithms get you some of the way, but you will still need someone with good sense to edit the result to get something usable, which is going to cost you in either time or money. Downsizing isn't so bad now because of higher DPI displays (e.g. font hinting is obsolete now) but previously designers would need to make individual icons in a number of sizes removing elements as necessary.
- tjr225 9y agoIt seems pretty trivial but I could be way wrong as this is in no way my area of expertise...but- pixelization filters in photoshop and other image editing softwares have been around for a very long time. Even logically it is an easy leap as they're(the software/filters) most likely choosing the most common color in a given area and then converting that area to a square of that color..
- michaelmior 9y agoJust pixelating an image doesn't really give high quality pixel art. No examples handy but I'm confident you'd see this if you tried to reverse the images in the post with the approach you outlined.
- asmosoinio 9y agoIn case you missed it - from another top-level comment: http://pixelatorapp.com/ http://pixelatorapp.com/
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- kelukelugames 9y agoThis page is where all the fun is. But it's also very slow. http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html http://johanneskopf.de/publications/pixelart/supplementary/m...
- leni536 9y agoTheir method change too much on eyes and facial expressions in general. Also that broad sword is really blunt. When comparing only the 4x results hq4x wins half the time. On the other half hq4x is a close second and generally looks alright. However in some cases the vectorized ones are pretty bad.
- deleted 9y ago[deleted]
- Proven 9y agoSounds promising for my JAV collection!
- corysama 9y agoMeanwhile, over in r/gamedev, someone just released a tool that does a good job of pixel-arting regular images. http://pixelatorapp.com/ http://pixelatorapp.com/ https://www.reddit.com/r/gamedev/comments/7ylvt9/pixelator_a_gamedev_tool_i_wrote_to_convert_any/ https://www.reddit.com/r/gamedev/comments/7ylvt9/pixelator_a...
- donutmonger 9y agoI'd be interested to see the result of iterating on an image using these tools. Like go from high res to pixel art, back to high res etc.
- alanfalcon 9y agoLike translating a phrase into a foreign language and back repeatedly, I inagine you’d end up with something fun and impressionistic. I’d like to see it too.
- Franciscouzo 9y agoLike this? http://www.translationparty.com/ http://www.translationparty.com/
- warent 9y agoI've never heard of this before but just wasted 20 minutes on it. Very interesting concept with hilarious results
- setzer22 9y agoMost things I tried were slightly changed from english to japanese just once, then converged. Maybe it's just my lack of imagination!
- reificator 9y ago
- microcolonel 9y agoGotta say, I really like those PhotoZoom 4 results, aside from some of the fringing artifacts. They seem to preserve a lot more of the detail, whereas the presented approach focuses more on eliminating noise.
- knkhere 9y agoI want to go through the research paper but am unable to load or download it. If anyone has managed to do the impossible please share a drive link. BTW which journal is this published in?
- z3t4 9y agoTake an image, Resize 500% so you see the pixels. Save. Run it through the algorithm. Repeat.
- maggit 9y agoI was excited about this when it was all the rage. It speaks to me as a programmer. Some years later, I stumbled upon this excellent article by an artist considering the problem from the art perspective: http://www.dinofarmgames.com/a-pixel-artist-renounces-pixel-art/ http://www.dinofarmgames.com/a-pixel-artist-renounces-pixel-... It points to the problem with these upscaled versions: They aren't any good. In the end, the artist demonstrates how to get good results, draw it again: http://www.dinofarmgames.com/wp-content/uploads/2015/04/yoshi-hand-drawn.png http://www.dinofarmgames.com/wp-content/uploads/2015/04/yosh... Of course, the algorithmic approach lets you upscale any number of preexisting games. However, I have concluded that I like the original art better.
- yoz-y 9y agoIn their comparison http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html http://johanneskopf.de/publications/pixelart/supplementary/m... the Vector Magic results appeal the most to me precisely for this reason. For most images they look more like an artists rendition than an attempt to trace the image. Which is good.
- LoSboccacc 9y agowhen it works it's beautiful but worse cases like mario face are ruined. hq4x isn't as pretty but seems more consistent over the whole gallery of examples http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html http://johanneskopf.de/publications/pixelart/supplementary/m...
- annabellish 9y ago>Of course, the algorithmic approach lets you upscale any number of preexisting games. This is the point of these algorithms, for sure. Hand-crafted graphics will always beat algorithmic upscaling of lower quality hand crafted graphics, given the same amount of time to do each, but that isn't possible for most examples of games with pixel art.
- fenwick67 9y agoI had to look but there is a NES emulator that does sprite replacement which is interesting: https://github.com/mkwong98/HDNes https://github.com/mkwong98/HDNes https://www.youtube.com/watch?v=N57ApspYHVE https://www.youtube.com/watch?v=N57ApspYHVE
- snvzz 9y agoAs long as using the pixel art as-is remains an option. So many classic games have been destroyed in this process.
- TruthSHIFT 9y agoThis site has a great page that compares various algorithms: http://johanneskopf.de/publications/pixelart/supplementary/multi_comparison.html http://johanneskopf.de/publications/pixelart/supplementary/m... While none of them really produce usable results, I love the artistic quality of the Vector Magic algorithm.
- MaxBarraclough 9y ago> none of them really produce usable results I don't know about that. In my opinion the images that aren't game characters look best with their vectoriser/upscaler. Cursor 1 and Setup, for instance.
- mrguyorama 9y agoThe chest blew my mind with how it seemed to accurately capture the shading and reflectivity of everything
- bluedino 9y agoIt looks...weird? Cartoony and almost Picasso-y. Boo and Bowser look great. Mario and Yoshi look terrible. The detailed pixel art and gradients just don't carry over. The shapes of the outlines do, however. Scanlines IMO look the best for video games, because it also tries to simulate the medium they were originally presented in, not just the graphics data behind the images https://imgur.com/wtxyA2d https://imgur.com/wtxyA2d
- thriftwy 9y agoIs it included into DosBox already? hq4x is included I guess, but unfortunately when you get your games from e.g. GOG, you have to tune DosBox manually every time.
- galago 9y agoMany type designers deal with this as part of their workflow if they work with traditional drawing tools like pen and ink. My understanding is that for classically designed fonts its often easier to redraw vector objects over the scan than to clean up what the automatic tools produce. However, for certain types of "distressed" fonts its a kind of necessity.
- malkia 9y agoMy favourite all time game is Heroes of Might and Magic III. It was released in 1999, and the art is still good, but that maybe because I'm so used to it. Ubisoft did a great job, enhancing it for the HD release in 2015, there is video showing it: https://www.youtube.com/watch?v=AX_h3RIbSKk https://www.youtube.com/watch?v=AX_h3RIbSKk Though, one lesson I've learned (as a game developer myself), was that enhancing may introduce more visual problems. On a port of a game I worked (Playstation1 -> PC) we hired an artist to enhance the eye textures, but once the team behind the original game saw this, they told us that since there was no EYE animation in the game, and having such high-fidelity texture (for it's time, back in 2000) made all the characters look like toys.. Now if we left (and we did) the eye textures blurry, then your own eyes can't focus, and accept this better (without eye animation). I'm not an artist (just a coder), but this taught me "less is more".
- jehlakj 9y agoThe biggest advantage of pixel art is that it takes up less resources to create a solid retro type game. Animations suddenly become a lot easier as it relies on the abstract nature of putting the imaginative efforts to the users. By removing these artifacts, you essentially defeat that purpose. Animating vectors would take a lot more time to create something that seems professional, mostly because you have to be far more detail - analogous to moving from 2d to 3d. I personally don’t see the use cases for this in gaming.
- tomlogic 9y agoA real-world example of this technique shows up in ColorDMD, an LCD to replace DMD displays used in pinball machines from the early 90's until just recently. Most of the displays were 128x32 with 4 (and later 16) brightness levels. ColorDMD first had artists create color overlays for the images and animations, and then emulated the display with colored dots instead of the red/orange color from the original DMD. A later firmware release included upscaling, with some impressive results. This thread on Pinside (a pinball discussion site) shows some examples from the game "The Simpsons Pinball Party". I've linked directly to a comment that shows an original rendered DMD frame along with colorized versions with and without upscaling. https://pinside.com/pinball/forum/topic/colordmd-unveils-new-hd-color-upscaling#post-2218723 https://pinside.com/pinball/forum/topic/colordmd-unveils-new...
- InternetUser 9y agoFor the like pixel art, click here and scroll way down: http://noirlac.tumblr.com/archive http://noirlac.tumblr.com/archive