6 ms·
Extending Van Gogh’s Starry Night with Inpainting
- jwise0 12y agoI wish that the Wolfram articles would describe a little more about what their algorithms do, rather than the mechanics of how to use their primitives. The articles remind me some of the era of web engines optimized for a specific task -- there were plenty of MVC frameworks that work really really well for writing a todo list in 25 lines, but doing anything more complicated than a todo list meant dissecting the guts of the framework. It'd be great to hear how they did an efficient implementation of texture synthesis. But if you want to read at least about the basics, the dissertation they linked [1] gets into the details on page 18, anyway, in section 2.2 (and beyond). [1] http://www.logarithmic.net/pfh-files/thesis/dissertation.pdf http://www.logarithmic.net/pfh-files/thesis/dissertation.pdf
- devilsdounut 12y agoWolfram has a habit of creating blog posts about the test cases for their algorithms.
- jordigh 12y agoHe also has a habit of not talking about how things work. All the time. Source code, mathematical proofs, citations, all of these things are relatively unimportant in Wolfram's world. He even thinks you should not try to find out how things work: http://reference.wolfram.com/language/tutorial/WhyYouDoNotUsuallyNeedToKnowAboutInternals.html http://reference.wolfram.com/language/tutorial/WhyYouDoNotUs...
- JetSpiegel 12y agoParticularly in more advanced applications of the Wolfram System, it may sometimes seem worthwhile to try to analyze internal algorithms in order to predict which way of doing a given computation will be the most efficient. And there are indeed occasionally major improvements that you will be able to make in specific computations as a result of such analyses. But most often the analyses will not be worthwhile. For the internals of the Wolfram System are quite complicated, and even given a basic description of the algorithm used for a particular purpose, it is usually extremely difficult to reach a reliable conclusion about how the detailed implementation of this algorithm will actually behave in particular circumstances. Trust us, we're professionals!
- jordigh 12y agoYeah, I find this article very insulting and condescending. "Don't worry your pretty little head, our ideas are far too complicated for you to understand!"
- JacobEdelman 12y agoIts a They, not He. The post isn't written by Wolfram himself :)
- geofft 12y agoThe Principle of Computational Equivalence says that Wolfram and his employees are equivalent, right?
- jordigh 12y agoHow do you know? This article has been up for as long as I've known Wolfram to have a website. I strongly suspect he wrote it himself.
- carlob 12y agoYou'd be surprised to know what fraction of the documentation he's actually written himself.
- dahart 12y agoRe: basics, here is one of the least efficient texture synthesis algorithms, but is detailed, an excellent starting place for beginners, and produces very high quality texture synthesis: http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.html http://graphics.cs.cmu.edu/people/efros/research/NPS/alg.htm... That is from my favorite texture synthesis paper of all (Efros & Leung '99), for these reasons: - I know Efros, and he's a genius, and also an absolutely adorable human being. - This paper was inspired by Markov text generators, its the two-dimensional equivalent, e.g., http://en.wikipedia.org/wiki/Mark_V_Shaney http://en.wikipedia.org/wiki/Mark_V_Shaney. Its very simple, produces great results, - AND - this texture synthesis algorithm will synthesize written text from a picture of written text. Almost none of the faster texture synthesis algorithms do this, and it seems meta; I like to think there is some deep fundamental truth hiding in there somewhere. (edited for typos and formatting)
- spot 12y agoafaik this technique was first implemented in the early 90s: http://draves.org/fuse/ http://draves.org/fuse/
- jordigh 12y agoWolfram has a habit of not giving a lot of citations or hiding them under the small print in order to make it seem like he or his company came up with everything in a void.
- JacobEdelman 12y agoAlthough this article isn't by Wolfram I agree that the philosophy of his company seems to be to hype products like any normal consumer company does. The problem is this doesn't really work when your selling and academic product. No programming language management organization will claim they are "revolutionizing programming" without strong qualifiers.
- sp332 12y agoFTA: The “TextureSynthesis” method is based on the algorithm described in “Image Texture Tools,”[1] a PhD thesis by P. Harrison. This algorithm is an enhanced best-fit approach introduced in 1981 by D. Garber in “Computational Models for Texture Analysis and Texture Synthesis.”[2] [1] http://www.logarithmic.net/pfh-files/thesis/dissertation.pdf http://www.logarithmic.net/pfh-files/thesis/dissertation.pdf [2] http://dl.acm.org/citation.cfm?id=910609 http://dl.acm.org/citation.cfm?id=910609
- spot 12y agois Garber's thesis (or better yet, a paper summarizing it) available online?
- sp332 12y agoThere have been lots of advances since, under the term "content-aware fill". Lots of links from here https://www.adobe.com/technology/projects/content-aware-fill.html https://www.adobe.com/technology/projects/content-aware-fill... There's an oldish plugin for the GIMP called Resynthesizer, but there are better algorithms out now. G'MIC has one called "inpainting". http://gmic.eu/repository.shtml http://gmic.eu/repository.shtml
- thu 12y agoThat's really neat. This makes me think to image analogies[0], something that impressed me enough that I keep remembering it after 13 years! [0]: http://www.mrl.nyu.edu/projects/image-analogies/ http://www.mrl.nyu.edu/projects/image-analogies/
- adamb_ 12y agoThanks for sharing this. I was really impressed by this example: http://www.mrl.nyu.edu/projects/image-analogies/potomac.html http://www.mrl.nyu.edu/projects/image-analogies/potomac.html
- JoeAltmaier 12y agoIts tiling the painting seamlessly, which is cool. Not really Van Gogh; but Van-Gogh compatible?
- maho 12y agoMathematica demos sure look pretty [0]. But I recently tried to get actual work done with Mathematica and found the experience utterly ridiculous. The high level functions, while nice for demos, never quite do what you need. Want to display data vs. time with errorbars around the data? Sorry, no function for this completely exotic use case, please hack a DateListPlot on top of an ErrorListPlot [1]. The WolframAlpha integration "transparently" interprets code for you in the background. This interpretation can change any day [2] and leaves you guessing why your notebook doesn't run anymore. The bugs I encountered were severe: One of the output function swallows the "-" sign if the result is between -1 and 0 [3]. Currently I am battling file corruption of notebooks that are saved on network drives and open while the drive is disattached/reattached. Had it not been for the awesome support by the mathematica. stackexchange.com community, I would have quit Mathematica completely. Now I vowed to use it only for small hacks. And, if I want to impress someone, for demos. [0] https://www.youtube.com/watch?v=EjCWdsrVcBM https://www.youtube.com/watch?v=EjCWdsrVcBM [1] http://mathematica.stackexchange.com/questions/18962 http://mathematica.stackexchange.com/questions/18962 [2] The input Quantity[1, "Kelvins"] is interpreted by Mathematica itself and yields an object equivalent to "1 Kelvins". The input Quantity[1, "Kelvin"] however is interpreted "transparently" by Wolfram Alpha. A few weeks ago it evaluated to "1 KelvinsDifference", but now has changed to "1 Kelvins". Don't get me started on the KelvinsDifference vs. Kelvins thing, which are both represented by the same symbol to maximize your confusion when your unit conversions are failing... [3] http://mathematica.stackexchange.com/questions/65600 http://mathematica.stackexchange.com/questions/65600
- leephillips 12y agoReally interesting, detailed comment. The problem is not that Mathematica has bugs - all complex software has bugs (except TeX). The problem is that you can't fix it, or even see how it works, because the source is secret. Nothing will get fixed until the company decides to fix it. Access to source is not an ideological imperative, it's a practical necessity.
- cpach 12y agoI’m not convinced open source would really help. I get the feeling that Mathematica’s underlying philosophy just isn’t a good fit for practical programming.
- andrey-p 12y agoWorth checking out the rest of Yarin Gal (the PhD student whose work prompted this post)'s extrapolated works of art: http://extrapolated-art.com/ http://extrapolated-art.com/ Some of these look so seamless it's staggering.
- deleted 12y ago[deleted]
- pfh 12y agoNote there's also a Gimp plugin called Resynthesizer that does this. http://www.logarithmic.net/pfh/resynthesizer http://www.logarithmic.net/pfh/resynthesizer It's included in Ubuntu(/Debian?) package "gimp-plugin-registry".
- trop 12y agoNot to be totally grumpy here, but this is classic mathematicians-think-art-is-images-looking-like-famous-art. The expanded "starry night" isn't an interesting painting, of course. And it adds nothing to Mr. V. V. Gogh's work. It just mimics aspects of the extant work, in a spatially naive manner (e.g., what more do we learn except that the new rendition exists on a black mesa surrounded by more village-with-church?). SIGGRAPH-style research shouldn't validate itself merely because it teaches the computer to adopt a cliched mode of image-making.
- infinitone 12y agoHave you looked at the other examples? http://extrapolated-art.com/ http://extrapolated-art.com/ Some of these are pretty impressive and do more than just pattern repetition.
- TheOtherHobbes 12y agoI'm going to agree with the critics - I think you may be confusing technical problem-solving with artistic awareness. They're all interesting as technology demonstrations, but none of them improve on the original compositions, and some of them are almost completely unsuccessful as visual art.
- Rapzid 12y agoI think that makes the whole just as interesting. Photographers often frame a tiny portion of the entire view available to them. That frame is where their art is, not the expanded view. So in a sense the wall just highlights that what's inside the frame is special and it contrasts nicely.
- djloche 12y agoNot to make this any less impressive, but doesn't the latest versions of Photoshop have these types of features built in?
- special_ghost 12y agoSuch an approach might be useful to the print industry for generating seamless "bleed" around images, which is required to allow some margin for error when trimming the physical prints.
- mistagiggles 12y agoFrom the comments on the article: http://imgur.com/a/BfcK4 http://imgur.com/a/BfcK4 Some more tests of the algorithm with less impressive results (though still good looking)
- soperj 12y agoOnly problem is, I've been to the village where Starry night was painted, it's not that big.
- thenomad 12y agoExtension algorithms are absolutely amazing - and incredibly useful if you're a practical working visual artist. I've personally found Photoshop's Content-Aware Fill to be so useful on films and comics that it now gets its own credit, as: "Matte extension: Content-Aware Phil".